-- Logs begin at Thu 2022-05-19 18:35:46 UTC, end at Thu 2022-05-19 20:34:05 UTC. -- May 19 19:35:19.383710 dv-cinder-835881-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 19:35:19.392511 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 19:35:19 2022] *** May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: nodename: dv-cinder-835881-13-iscsi May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: machine: x86_64 May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: clock source: unix May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: pcre jit disabled May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: detected number of CPU cores: 4 May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: current working directory: / May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: detected binary path: /usr/bin/uwsgi-core May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: your processes number limit is 47680 May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: your memory page size is 4096 bytes May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: detected max file descriptor number: 2048 May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: lock engine: pthread robust mutexes May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: thunder lock: enabled May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 19:35:19.395510 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 19:35:19.439386 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: Python main interpreter initialized at 0x5571a1f789c0 May 19 19:35:19.439386 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: python threads support enabled May 19 19:35:19.439386 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: your server socket listen backlog is limited to 100 connections May 19 19:35:19.439386 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: your mercy for graceful operations on workers is 90 seconds May 19 19:35:19.439702 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: mapped 403077 bytes (393 KB) for 2 cores May 19 19:35:19.439773 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: *** Operational MODE: preforking *** May 19 19:35:19.440075 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: *** uWSGI is running in multiple interpreter mode *** May 19 19:35:19.440075 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: spawned uWSGI master process (pid: 86977) May 19 19:35:19.440293 dv-cinder-835881-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 19:35:19.440977 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: spawned uWSGI worker 1 (pid: 86981, cores: 1) May 19 19:35:19.441122 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: spawned uWSGI worker 2 (pid: 86982, cores: 1) May 19 19:35:19.441277 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 19:35:25.055154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:35:25.055154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: from cryptography.utils import int_from_bytes May 19 19:35:25.055958 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:35:25.055958 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: from cryptography.utils import int_from_bytes May 19 19:35:25.105002 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 19:35:25.105170 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 19:35:25.105365 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] command line args: [] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 19:35:25.105627 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 19:35:25.105755 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ================================================================================ {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 19:35:25.106031 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] admin_token = **** {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.106284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] config_dir = [] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.106438 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.106646 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] config_source = [] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.106802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] debug = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.106990 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.107370 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.107569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.107791 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.108006 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.108247 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] list_limit = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.108401 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] log_config_append = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.108590 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.108750 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] log_dir = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.108923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] log_file = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.109165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.109404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.109655 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.109882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.110050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.110223 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.110499 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.110741 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.110955 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.111179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.111409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.111636 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.111886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.112108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.112385 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.112869 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.84/identity {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.113055 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] publish_errors = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.113225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.113480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.113659 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.113882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.114104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.114351 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] standard_threads = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.114575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.114792 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.115088 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.115269 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] use_journal = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.115494 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] use_json = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.115562 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] use_stderr = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.115737 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] use_syslog = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.115962 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.116217 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.116452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.116705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.116940 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.117182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.117415 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.117656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.117887 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.118129 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.118399 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.118648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.118880 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.119121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.119353 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.119592 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.119832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.120063 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.120306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.120554 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.120793 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.121023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.121267 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.121522 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.121767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.122010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.122235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.122486 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.122712 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.122944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.123188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.123435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.123658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.123882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.124122 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.124362 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.124593 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.124834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.125059 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.125289 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.125516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.125740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.125961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.126183 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.126430 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.126660 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.126902 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.127126 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.127355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.127578 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.127806 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.128034 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.128273 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.128498 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.128726 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.128957 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.129184 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.129412 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.129638 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.129860 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.130100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.130352 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.130661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.130890 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.131146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.131377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.131601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.131829 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.132058 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.132319 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.132551 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.132791 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.133031 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.133262 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.133496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.133725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.133965 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.134195 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.134450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.134690 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.134921 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.135150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.135380 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.135609 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.135837 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.136094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.136331 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.136565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.136795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.137032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.137262 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.137491 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.137717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.137942 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.138165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.138412 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.138641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.138864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.139097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.139327 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.139567 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.139797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.140157 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.140415 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.140651 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.140877 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] auth.external = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.141102 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.141393 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.141622 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.141848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] auth.password = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.142073 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] auth.token = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.142331 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.142564 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.142805 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.143031 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.143273 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.143393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:35:25.143393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: from cryptography.utils import int_from_bytes May 19 19:35:25.143973 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.143973 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.144192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:35:25.144192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: from cryptography.utils import int_from_bytes May 19 19:35:25.144398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] credential.caching = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.144398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.144398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.144976 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.144976 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.145203 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.145321 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.145541 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.145788 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.146019 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.146521 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.146810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.147053 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.147313 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.147551 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.147803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.148034 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] federation.caching = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.148264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.148502 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.148742 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.149099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.149215 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.149423 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.149668 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.149893 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.150154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.150461 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.150630 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.150857 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity.caching = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.151082 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.151321 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.151550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.151779 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.152037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.152243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.152471 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.152711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.152970 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.153180 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.153391 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.153618 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.153854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.154095 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.154381 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.154613 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.154853 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.155119 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.155346 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.155576 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.155800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.156051 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.156254 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.156487 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.156723 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.156955 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.157196 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.157420 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.157657 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.157895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.158124 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.158387 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.158626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.158848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.159079 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.159305 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.159534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.159764 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.160029 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.160233 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.160461 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.160701 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.160941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.161164 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.161390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.161630 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.161954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.162183 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.162572 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.162804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.163029 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.163263 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.163518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.163747 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.163987 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.164230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.164469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.164699 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.164922 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.165152 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.165379 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.165607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.165832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.166072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.166332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.166572 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.166809 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.167047 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.167269 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.167506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.167737 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.167966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.168194 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.168421 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.168659 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.168897 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.169126 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.169370 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.169593 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.169832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.170060 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.170307 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.170540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.170779 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.171017 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.171473 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.171710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] resource.caching = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.171954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.172196 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.172425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.172667 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.172909 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.173137 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.173373 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.173601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.173839 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.174068 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] role.caching = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.174329 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] role.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.174554 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.174794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.175032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.175271 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.175509 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.175747 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.175984 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.176220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.176458 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.176684 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.176920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.177158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.177439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.177733 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.178141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.178392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.178636 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.178877 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.179117 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.179358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.179584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.179829 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.180071 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.180302 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.180525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.180748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.180970 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.181213 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.181457 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.181697 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.181928 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.182156 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.182403 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.182634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] token.caching = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.182863 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.183103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.183333 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.183580 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.183825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.184055 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.184292 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.184527 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.184767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.184995 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.185227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.185455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.185692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.185934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.186156 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.186414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.186701 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.186954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.connection = **** {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.187190 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.187428 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.187657 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.187888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.188117 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.188347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.188575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.188802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.189037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.189265 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.189505 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.189721 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.189959 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.190185 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.190415 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 19:35:25.190548 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.190666 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 19:35:25.190782 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.190894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] command line args: [] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 19:35:25.191005 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.191118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 19:35:25.191307 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.191424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ================================================================================ {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 19:35:25.191537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 19:35:25.191652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] admin_token = **** {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.191777 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] config_dir = [] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.191889 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.192013 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] config_source = [] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.192193 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] debug = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.192409 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.192647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.192825 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.193149 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.193428 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.193664 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] list_limit = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.193829 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] log_config_append = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.194009 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.194185 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] log_dir = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.194386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] log_file = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.194626 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.194979 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.195232 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.195483 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.195662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.195846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.196094 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.196340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.196579 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.196815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.197050 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.197285 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.197533 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.197779 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.198056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.198576 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.84/identity {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.198759 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] publish_errors = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.198993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.199223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.199400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.199651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.199887 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.200121 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] standard_threads = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.200355 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.200598 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.200774 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.200950 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] use_journal = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.201127 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] use_json = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.201303 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] use_stderr = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.201478 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] use_syslog = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.201710 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:35:25.201967 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.202207 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.202489 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.202728 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.202978 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.203217 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.203469 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.203708 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.203957 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.204207 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.204458 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.204693 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.204941 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.205179 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.205426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.205675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.205913 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.206169 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.206453 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.206701 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.206938 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.207192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.207456 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.207711 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.207961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.208195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.208428 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.208686 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.208922 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.209172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.209426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.209656 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.209887 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.210137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.210410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.210650 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.210901 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.211134 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.211372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.211608 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.211840 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.212072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.212303 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.212536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.212773 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.213023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.213254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.213491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.213724 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.213961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.214197 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.214471 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.214702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.214940 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.215178 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.215415 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.215652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.215889 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.216119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.216368 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.216603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.216842 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.217073 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.217335 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.217572 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.217803 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.218041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.218290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.218564 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.218806 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.219057 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.219309 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.219548 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.219786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.220026 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.220280 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.220529 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.220771 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.221020 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.221260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.221496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.221735 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.221972 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.222087 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.222214 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.222346 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.222461 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.222576 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.222725 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.222970 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.223208 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.223451 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.223691 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.223930 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.224166 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.224446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.224679 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.224912 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.225161 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.225393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.225633 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.225870 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.226119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.226383 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.226642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.226904 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.227146 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.227380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] auth.external = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.227612 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.227909 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.228478 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.228710 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] auth.password = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.228941 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] auth.token = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.229181 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.229426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.229674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.229905 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.230153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.230425 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.230662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.230899 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] credential.caching = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.231146 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.231396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.231512 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.231629 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.231868 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.232113 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.232348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.232609 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.232863 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.233102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.233356 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.233652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.233900 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.234165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.234431 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.234692 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.234932 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] federation.caching = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.235180 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.235430 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.235679 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.235911 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.236161 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.236402 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.236657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.236895 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.237150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.237389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.237639 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.237877 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity.caching = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.238126 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.238400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.238641 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.238879 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.239128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.239364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.239604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.239877 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.240710 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.240973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.241210 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.241445 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.241693 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.241944 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.242194 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.242479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.242730 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.243003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.243245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.243484 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.243970 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.244204 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.244445 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.244689 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.244935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.245184 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.245417 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.245919 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.245919 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.246145 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.246389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.246637 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.246871 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.247110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.247343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.247583 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.247819 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.248065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.248304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.248541 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.248788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.249046 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.249278 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.249508 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.249764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.250014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.250255 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.250513 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.250749 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.250979 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.251222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.251484 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.251716 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.251967 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.252217 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.252465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.252701 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.252933 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.253170 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.253404 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.253642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.253872 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.254119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.254393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.254641 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.254889 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.255133 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.255364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.255741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.256026 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.256316 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.256554 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.256793 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.257041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.257290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.257528 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.257784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.258015 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.258282 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.258521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.258760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.258999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.259246 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.259356 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.259488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.259586 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.259724 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.259960 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.260199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] resource.caching = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.260449 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.260699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.260935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.261185 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.261446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.261685 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.261939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.262204 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.262467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.262704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] role.caching = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.262952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] role.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.263184 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.263436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.263683 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.263933 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.264181 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.264430 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.264677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.264922 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.265171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.265403 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.265651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.265897 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.266145 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.266415 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.266828 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.267064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.267315 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.267565 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.267812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.268061 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.268296 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.268550 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.268803 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.269042 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.269276 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.269506 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.269744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.269997 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.270251 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.270528 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.270766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.271002 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.271373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.271647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] token.caching = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.271889 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.272140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.272377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.272634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.272881 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.273120 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.273364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.273607 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.273854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.274091 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.274354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.274593 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.274840 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.275088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.275329 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.275574 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.275869 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.276133 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.connection = **** {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.276379 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.276630 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.276870 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.277113 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.277351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.277587 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.277823 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.278061 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.278340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.278578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.278824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.279059 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.279307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.279541 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.279791 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.280022 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.280260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.280493 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:35:25.280662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86982) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 19:35:25.307760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.307914 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.308125 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.311676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.316837 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.335350 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.335562 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.362532 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.362770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.374938 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.411395 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.411638 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.427488 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.427718 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.427918 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.428145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.428326 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.428560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.451714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.451999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.452119 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.452346 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.452549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.452760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.452968 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.454537 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.454731 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.454975 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.455233 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.455411 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.455614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.464228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.470295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.470530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.470754 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.470978 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.471196 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.471420 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.471621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.479139 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.479366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.479584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.479808 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.480015 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.480139 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.480279 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.480392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.483005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.483232 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.483455 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.483671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.483881 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.484092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.484311 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86982) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:35:25.497458 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.497646 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:35:25.497998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.498122 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.498220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:35:25.498315 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:35:25.498447 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.498667 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.498792 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.498869 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:35:25.498952 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.499090 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.499173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.499382 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.499645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.499899 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.510305 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.510603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.510864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.511136 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.511277 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.511426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.511664 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.511944 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.519167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.520027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.522532 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.522802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.523184 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.523308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.523446 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.523702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.523828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.523963 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.524043 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.524217 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.524356 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.524491 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.524571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.524739 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.524865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.524983 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.525088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.525259 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.525385 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.525492 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.525792 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.526051 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.551262 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.552657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.552956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.553225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.554507 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.554787 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.558492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.558712 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.558983 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.560056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.560332 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.560611 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.566159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.566456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.567949 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.568309 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.571363 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.571634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.571906 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.572171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.572431 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.572697 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.572947 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.573210 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.573365 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.573618 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.573893 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.574160 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.574466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.574737 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.575018 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.591454 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.591722 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.594195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.594497 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.596203 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.599036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.599973 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.600100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.602848 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.603452 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.605340 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.608432 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.608884 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.612145 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.612636 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.612927 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.613206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.613497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.613817 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.614197 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.616378 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.616671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.616975 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.617267 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.617538 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.617829 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.633326 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.633696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.633962 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.634218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.638373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.638664 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.638931 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.639194 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.642077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.643806 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.644073 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.647180 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.647308 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.647537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.647809 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.648081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.648368 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.649282 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.649620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.652786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.653066 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.653353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.653628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.653902 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.661549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.661929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.662212 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.662518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.662823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.663090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.663365 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.667473 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.667753 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.668028 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.668369 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.668571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.668868 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.669132 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.681375 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.684608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.687891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.688429 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.689471 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.691134 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.691414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.691686 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.691812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.695319 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.696922 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.698570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.698851 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.699140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.699266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.700775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.703984 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.704269 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.704547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.704817 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.706515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.708117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.711563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.711857 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.712138 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.712418 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.714141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.714515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.714858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.721594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.721878 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.722159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.722668 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.722945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.723228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.723461 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.723754 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.724039 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.724350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.724579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.724852 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.725129 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.725414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.725676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.725939 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.726195 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.730399 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86982) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:35:25.730671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.730951 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.731225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.731491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.731756 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.732023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.732294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.732571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.732843 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.733094 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.733349 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.733715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.733989 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86982) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:35:25.764931 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5571a1f789c0 pid: 86981 (default app) May 19 19:35:25.768341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2f4f2-a94e-4d32-9016-a0c3f9dde383 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:25.768627 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2f4f2-a94e-4d32-9016-a0c3f9dde383 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:25.768892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2f4f2-a94e-4d32-9016-a0c3f9dde383 None None] PATH_INFO: `/v3` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:25.771762 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5571a1f789c0 pid: 86982 (default app) May 19 19:35:25.773259 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 1/1] 10.222.0.84 () {50 vars in 791 bytes} [Thu May 19 19:35:25 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:33.815143 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225e015-03e7-4f3c-beb4-8be7170becd8 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:33.816026 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225e015-03e7-4f3c-beb4-8be7170becd8 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:33.816026 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225e015-03e7-4f3c-beb4-8be7170becd8 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:33.823645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 1/2] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:33 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:33.829029 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f73df-1061-4f11-8baf-a315de20995a None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:33.829461 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f73df-1061-4f11-8baf-a315de20995a None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:33.829551 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f73df-1061-4f11-8baf-a315de20995a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:33.897140 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: INFO dbcounter [None req-667f73df-1061-4f11-8baf-a315de20995a None None] Registered counter for database keystone May 19 19:35:33.922012 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG oslo_db.sqlalchemy.engines [None req-667f73df-1061-4f11-8baf-a315de20995a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86981) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 19:35:33.930178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writer thread running {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 19:35:34.108800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG passlib.handlers.bcrypt [None req-667f73df-1061-4f11-8baf-a315de20995a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86981) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 19:35:34.109198 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG passlib.handlers.bcrypt [None req-667f73df-1061-4f11-8baf-a315de20995a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86981) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 19:35:34.165137 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-667f73df-1061-4f11-8baf-a315de20995a None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:34.166670 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-667f73df-1061-4f11-8baf-a315de20995a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:34.321664 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 2/3] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 19:35:34.521980 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225e015-03e7-4f3c-beb4-8be7170becd8 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:34.617530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: INFO dbcounter [None req-d225e015-03e7-4f3c-beb4-8be7170becd8 None None] Registered counter for database keystone May 19 19:35:34.640088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG oslo_db.sqlalchemy.engines [None req-d225e015-03e7-4f3c-beb4-8be7170becd8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86982) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 19:35:34.649887 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writer thread running {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 19:35:34.770213 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225e015-03e7-4f3c-beb4-8be7170becd8 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:34.775940 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916f9c7-17fe-4985-967e-820fa3b7e56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '526171dca7b844d9b1cd612ef681909d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:34.776548 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916f9c7-17fe-4985-967e-820fa3b7e56e admin admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:34.776828 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916f9c7-17fe-4985-967e-820fa3b7e56e admin admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:34.777103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916f9c7-17fe-4985-967e-820fa3b7e56e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:34.786223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916f9c7-17fe-4985-967e-820fa3b7e56e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:34.837907 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7916f9c7-17fe-4985-967e-820fa3b7e56e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 19:35:34.838525 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916f9c7-17fe-4985-967e-820fa3b7e56e admin admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:34.848604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 2/4] 10.222.0.84 () {58 vars in 1324 bytes} [Thu May 19 19:35:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 19:35:34.856512 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f73df-1061-4f11-8baf-a315de20995a None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:34.875345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f73df-1061-4f11-8baf-a315de20995a None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:34.881816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6f455-8ef4-4670-a34d-af416d9350ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '526171dca7b844d9b1cd612ef681909d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:34.883004 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6f455-8ef4-4670-a34d-af416d9350ce admin admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:34.883289 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6f455-8ef4-4670-a34d-af416d9350ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:34.883562 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6f455-8ef4-4670-a34d-af416d9350ce admin admin] PATH_INFO: `/v3/projects/526171dca7b844d9b1cd612ef681909d` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:34.885219 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6f455-8ef4-4670-a34d-af416d9350ce admin admin] RBAC: Authorizing `identity:get_project(project_id=526171dca7b844d9b1cd612ef681909d)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:35.050652 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: WARNING keystone.common.rbac_enforcer.enforcer [None req-28a6f455-8ef4-4670-a34d-af416d9350ce admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 19:35:35.051005 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6f455-8ef4-4670-a34d-af416d9350ce admin admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:35.056087 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 3/5] 10.222.0.84 () {56 vars in 1207 bytes} [Thu May 19 19:35:34 2022] GET /identity/v3/projects/526171dca7b844d9b1cd612ef681909d => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:37.250431 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa22c4-301e-473c-ba03-67fb53cf41c0 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:37.250989 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa22c4-301e-473c-ba03-67fb53cf41c0 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:37.250989 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa22c4-301e-473c-ba03-67fb53cf41c0 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:37.256856 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 3/6] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:37.263621 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a00c25-bd4c-47b6-9e67-e6834a6d5246 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:37.264033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a00c25-bd4c-47b6-9e67-e6834a6d5246 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:37.264217 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a00c25-bd4c-47b6-9e67-e6834a6d5246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:37.293525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-32a00c25-bd4c-47b6-9e67-e6834a6d5246 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:37.295126 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-32a00c25-bd4c-47b6-9e67-e6834a6d5246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:37.305508 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 4/7] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 19:35:37.329022 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa22c4-301e-473c-ba03-67fb53cf41c0 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:37.343626 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa22c4-301e-473c-ba03-67fb53cf41c0 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:37.349280 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f23a21-d0e6-44d6-9c75-135f45c79463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '526171dca7b844d9b1cd612ef681909d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:37.350090 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f23a21-d0e6-44d6-9c75-135f45c79463 admin admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:37.350405 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f23a21-d0e6-44d6-9c75-135f45c79463 admin admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:37.350672 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f23a21-d0e6-44d6-9c75-135f45c79463 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:37.359606 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f23a21-d0e6-44d6-9c75-135f45c79463 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:37.361396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f23a21-d0e6-44d6-9c75-135f45c79463 admin admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:37.370474 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 4/8] 10.222.0.84 () {58 vars in 1324 bytes} [Thu May 19 19:35:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 19:35:37.376015 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a00c25-bd4c-47b6-9e67-e6834a6d5246 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:37.385344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a00c25-bd4c-47b6-9e67-e6834a6d5246 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:37.392665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b3632-6ac3-48c1-858e-cfb9a17538a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '526171dca7b844d9b1cd612ef681909d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:37.393938 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b3632-6ac3-48c1-858e-cfb9a17538a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:37.394329 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b3632-6ac3-48c1-858e-cfb9a17538a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:37.394688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b3632-6ac3-48c1-858e-cfb9a17538a8 admin admin] PATH_INFO: `/v3/users/a5800b131869435eb3ae07f86d69929e` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:37.397389 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b3632-6ac3-48c1-858e-cfb9a17538a8 admin admin] RBAC: Authorizing `identity:get_user(user_id=a5800b131869435eb3ae07f86d69929e)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:37.400346 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b3632-6ac3-48c1-858e-cfb9a17538a8 admin admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:37.402971 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 5/9] 10.222.0.84 () {56 vars in 1198 bytes} [Thu May 19 19:35:37 2022] GET /identity/v3/users/a5800b131869435eb3ae07f86d69929e => generated 247 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:39.690708 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d44e89-c109-4ed4-8e45-d6227bb6cf57 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:39.691265 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d44e89-c109-4ed4-8e45-d6227bb6cf57 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:39.691265 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d44e89-c109-4ed4-8e45-d6227bb6cf57 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:39.692653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 5/10] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:39.697966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7d5f7-3138-41bc-8d71-92d273457bc6 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:39.698407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7d5f7-3138-41bc-8d71-92d273457bc6 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:39.698689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7d5f7-3138-41bc-8d71-92d273457bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:39.701396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e034cc-f09b-4a21-bb1d-a15f3ad62cc8 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:39.701814 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e034cc-f09b-4a21-bb1d-a15f3ad62cc8 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:39.702206 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e034cc-f09b-4a21-bb1d-a15f3ad62cc8 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:39.704209 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 6/11] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:39.709665 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a60bcb-3d2f-4e47-8290-73444a3ab179 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:39.710066 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a60bcb-3d2f-4e47-8290-73444a3ab179 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:39.710503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a60bcb-3d2f-4e47-8290-73444a3ab179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:39.723518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-37e7d5f7-3138-41bc-8d71-92d273457bc6 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:39.724877 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-37e7d5f7-3138-41bc-8d71-92d273457bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:39.755537 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG passlib.handlers.bcrypt [None req-31a60bcb-3d2f-4e47-8290-73444a3ab179 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86982) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 19:35:39.755957 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG passlib.handlers.bcrypt [None req-31a60bcb-3d2f-4e47-8290-73444a3ab179 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86982) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 19:35:39.801782 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-31a60bcb-3d2f-4e47-8290-73444a3ab179 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:39.803451 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-31a60bcb-3d2f-4e47-8290-73444a3ab179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:39.807052 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 6/12] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:35:39.830545 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7d5f7-3138-41bc-8d71-92d273457bc6 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:39.854240 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:39.908515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 7/13] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:35:39.931731 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a60bcb-3d2f-4e47-8290-73444a3ab179 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:39.933509 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7d5f7-3138-41bc-8d71-92d273457bc6 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:39.970902 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c6474-dc98-4917-8fc4-be45ced2925e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:39.971720 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c6474-dc98-4917-8fc4-be45ced2925e None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:39.971996 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c6474-dc98-4917-8fc4-be45ced2925e None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:39.972267 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c6474-dc98-4917-8fc4-be45ced2925e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:39.984184 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55c6474-dc98-4917-8fc4-be45ced2925e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-e55c6474-dc98-4917-8fc4-be45ced2925e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:35:39.995504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:35:39.996787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:35:39.997575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:35:39.997575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:35:39.997575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:35:39.997575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:35:39.997575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:35:39.997575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:35:39.997575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:35:39.997575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:35:39.997575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:35:39.997575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:35:39.997575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:35:39.997575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:35:39.997575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 7/14] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:35:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:35:40.002940 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c6474-dc98-4917-8fc4-be45ced2925e None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:40.044168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a60bcb-3d2f-4e47-8290-73444a3ab179 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:40.046481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:40.082388 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e5178-a2e2-4909-b9b6-22554093281e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:40.083388 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e5178-a2e2-4909-b9b6-22554093281e None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:40.083664 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e5178-a2e2-4909-b9b6-22554093281e None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:40.083939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e5178-a2e2-4909-b9b6-22554093281e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:40.088445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:40.111381 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c6474-dc98-4917-8fc4-be45ced2925e None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:40.145671 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a5d00-d46e-494a-bcee-ab61e7d3c946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:40.146392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a5d00-d46e-494a-bcee-ab61e7d3c946 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:40.146674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a5d00-d46e-494a-bcee-ab61e7d3c946 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:40.146950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a5d00-d46e-494a-bcee-ab61e7d3c946 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:40.147710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a5d00-d46e-494a-bcee-ab61e7d3c946 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:35:40.148015 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a5d00-d46e-494a-bcee-ab61e7d3c946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:40.149426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a5d00-d46e-494a-bcee-ab61e7d3c946 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:40.155272 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e5178-a2e2-4909-b9b6-22554093281e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:40.156799 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e5178-a2e2-4909-b9b6-22554093281e None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:40.161455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 8/15] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:35:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:40.167010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a5d00-d46e-494a-bcee-ab61e7d3c946 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:40.265175 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a5d00-d46e-494a-bcee-ab61e7d3c946 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:40.265664 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 8/16] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:35:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:35:40.271490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e5178-a2e2-4909-b9b6-22554093281e None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:40.275106 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:40.300454 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffc52c-6605-4b81-a922-b45f7f834fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:40.301346 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffc52c-6605-4b81-a922-b45f7f834fe8 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:40.301624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffc52c-6605-4b81-a922-b45f7f834fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:40.302042 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffc52c-6605-4b81-a922-b45f7f834fe8 None admin] PATH_INFO: `/v3/users/a5800b131869435eb3ae07f86d69929e` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:40.303781 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffc52c-6605-4b81-a922-b45f7f834fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=a5800b131869435eb3ae07f86d69929e)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:40.305270 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffc52c-6605-4b81-a922-b45f7f834fe8 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:40.307116 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 9/17] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:35:40 2022] GET /identity/v3/users/a5800b131869435eb3ae07f86d69929e => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:40.312586 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffc52c-6605-4b81-a922-b45f7f834fe8 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:40.331386 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:40.384584 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e5178-a2e2-4909-b9b6-22554093281e None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:40.418201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531325a7-5bbd-4e02-a809-736d8d2d538c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:40.418964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531325a7-5bbd-4e02-a809-736d8d2d538c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:40.419067 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffc52c-6605-4b81-a922-b45f7f834fe8 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:40.419279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531325a7-5bbd-4e02-a809-736d8d2d538c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:40.419524 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531325a7-5bbd-4e02-a809-736d8d2d538c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:40.438442 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531325a7-5bbd-4e02-a809-736d8d2d538c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:40.439924 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531325a7-5bbd-4e02-a809-736d8d2d538c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:40.453370 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196b788-ef9d-4337-9d64-4e581b63a365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:40.454168 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196b788-ef9d-4337-9d64-4e581b63a365 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:40.454480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196b788-ef9d-4337-9d64-4e581b63a365 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:40.454753 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196b788-ef9d-4337-9d64-4e581b63a365 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:40.455946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196b788-ef9d-4337-9d64-4e581b63a365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:40.457396 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196b788-ef9d-4337-9d64-4e581b63a365 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:40.459087 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 10/18] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:35:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:40.460527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-531325a7-5bbd-4e02-a809-736d8d2d538c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:35:40.460527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:35:40.460527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:35:40.460527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:35:40.460527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:35:40.460527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:35:40.460527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:35:40.460527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:35:40.460527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:35:40.460527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:35:40.460527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:35:40.460527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:35:40.460527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:35:40.460527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:35:40.460527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:35:40.461377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:35:40.462187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:35:40.462187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:35:40.462187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:35:40.462187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:35:40.462187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 9/19] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:35:40.464672 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196b788-ef9d-4337-9d64-4e581b63a365 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:40.467214 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531325a7-5bbd-4e02-a809-736d8d2d538c None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:40.524030 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:40.545349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:40.565482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196b788-ef9d-4337-9d64-4e581b63a365 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:40.566663 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531325a7-5bbd-4e02-a809-736d8d2d538c None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:40.599781 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b613f-25d3-4793-9d3a-e01ab8506f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:40.600031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c210e-a1c3-44ad-9d90-5efef21bfdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:40.600423 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b613f-25d3-4793-9d3a-e01ab8506f39 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:40.600700 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b613f-25d3-4793-9d3a-e01ab8506f39 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:40.600805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c210e-a1c3-44ad-9d90-5efef21bfdb7 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:40.600967 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b613f-25d3-4793-9d3a-e01ab8506f39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:40.601053 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c210e-a1c3-44ad-9d90-5efef21bfdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:40.601244 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c210e-a1c3-44ad-9d90-5efef21bfdb7 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:40.601787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b613f-25d3-4793-9d3a-e01ab8506f39 None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.domain.id=default, user.id=a5800b131869435eb3ae07f86d69929e) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:35:40.601966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c210e-a1c3-44ad-9d90-5efef21bfdb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:35:40.602089 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b613f-25d3-4793-9d3a-e01ab8506f39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:40.602280 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c210e-a1c3-44ad-9d90-5efef21bfdb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:40.603908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b613f-25d3-4793-9d3a-e01ab8506f39 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:40.604040 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c210e-a1c3-44ad-9d90-5efef21bfdb7 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:40.610719 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 11/20] 10.222.0.84 () {56 vars in 1322 bytes} [Thu May 19 19:35:40 2022] GET /identity/v3/role_assignments?user.id=a5800b131869435eb3ae07f86d69929e&scope.domain.id=default&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:35:40.626498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 10/21] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:43.384372 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394557c4-c72c-43f7-8fdf-3e91da878e1d None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:43.385398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394557c4-c72c-43f7-8fdf-3e91da878e1d None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:43.386065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394557c4-c72c-43f7-8fdf-3e91da878e1d None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:43.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 12/22] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:43.393445 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a5437-b591-41f6-8fe9-5a3d81b4bba2 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:43.394320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a5437-b591-41f6-8fe9-5a3d81b4bba2 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:43.394816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a5437-b591-41f6-8fe9-5a3d81b4bba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:43.422540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-d38a5437-b591-41f6-8fe9-5a3d81b4bba2 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:43.424057 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-d38a5437-b591-41f6-8fe9-5a3d81b4bba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:43.520506 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 11/23] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:35:43.538637 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d764b8-1f7f-47de-a7ad-5810383248df None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:43.539230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9ab09-bbb9-43a2-9905-caf9f6c342b2 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:43.539337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d764b8-1f7f-47de-a7ad-5810383248df None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:43.539519 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9ab09-bbb9-43a2-9905-caf9f6c342b2 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:43.539789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9ab09-bbb9-43a2-9905-caf9f6c342b2 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:43.540173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d764b8-1f7f-47de-a7ad-5810383248df None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:43.541144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 12/24] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:35:43.541927 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 13/25] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:43.546871 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32be5f-4e37-4a3f-9e83-e2a089958694 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:43.547152 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32be5f-4e37-4a3f-9e83-e2a089958694 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:43.547425 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32be5f-4e37-4a3f-9e83-e2a089958694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:43.557495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b290b-0f88-426c-837b-2598ac3a3d10 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:43.558102 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b290b-0f88-426c-837b-2598ac3a3d10 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:43.558605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b290b-0f88-426c-837b-2598ac3a3d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:43.572103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-1a32be5f-4e37-4a3f-9e83-e2a089958694 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:43.573267 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-1a32be5f-4e37-4a3f-9e83-e2a089958694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:43.590732 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-eb2b290b-0f88-426c-837b-2598ac3a3d10 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:43.592273 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-eb2b290b-0f88-426c-837b-2598ac3a3d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:43.640179 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:43.678724 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 13/26] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:35:43.679728 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32be5f-4e37-4a3f-9e83-e2a089958694 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:43.763676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 14/27] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 19:35:43.764556 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b290b-0f88-426c-837b-2598ac3a3d10 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:43.818432 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32be5f-4e37-4a3f-9e83-e2a089958694 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:43.901155 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:43.930349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b290b-0f88-426c-837b-2598ac3a3d10 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:43.967836 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00809dc3-f10f-4852-8186-bbe261af5e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:43.969141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00809dc3-f10f-4852-8186-bbe261af5e41 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:43.969676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00809dc3-f10f-4852-8186-bbe261af5e41 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:43.970142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00809dc3-f10f-4852-8186-bbe261af5e41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:43.980851 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f07d4-9a1a-4065-84d7-3b240f449f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:43.981653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f07d4-9a1a-4065-84d7-3b240f449f9f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:43.981932 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f07d4-9a1a-4065-84d7-3b240f449f9f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:43.982211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f07d4-9a1a-4065-84d7-3b240f449f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:43.983259 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00809dc3-f10f-4852-8186-bbe261af5e41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-00809dc3-f10f-4852-8186-bbe261af5e41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:35:43.991550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:35:43.992418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:35:43.993154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:35:43.993154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:35:43.993154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:35:43.993154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:35:43.993154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:35:43.993154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:35:43.993154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:35:43.993154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:35:43.993154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:35:43.993154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:35:43.993154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:35:43.993154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:35:43.993154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 15/28] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:35:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:35:43.993760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00809dc3-f10f-4852-8186-bbe261af5e41 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:44.000587 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f07d4-9a1a-4065-84d7-3b240f449f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:44.002189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f07d4-9a1a-4065-84d7-3b240f449f9f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:44.021364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-590f07d4-9a1a-4065-84d7-3b240f449f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:35:44.021364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:35:44.021364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:35:44.021364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:35:44.021364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:35:44.021364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:35:44.021364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:35:44.021364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:35:44.021364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:35:44.021364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:35:44.021364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:35:44.021364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:35:44.021364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:35:44.021364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:35:44.021364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:35:44.022419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:35:44.023218 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:35:44.023218 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:35:44.023218 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:35:44.023218 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:35:44.026346 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 14/29] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:35:44.027793 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03254a5c-b12b-4b1b-b4e7-84e597a9a5d8 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:44.028084 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03254a5c-b12b-4b1b-b4e7-84e597a9a5d8 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:44.028363 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03254a5c-b12b-4b1b-b4e7-84e597a9a5d8 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:44.033736 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 15/30] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:44.033736 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03254a5c-b12b-4b1b-b4e7-84e597a9a5d8 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:44.064559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:44.143002 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00809dc3-f10f-4852-8186-bbe261af5e41 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:44.148704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03254a5c-b12b-4b1b-b4e7-84e597a9a5d8 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:44.178698 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd2417-1f4a-46c6-baa7-9a50cf228d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:44.179592 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd2417-1f4a-46c6-baa7-9a50cf228d73 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:44.180013 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd2417-1f4a-46c6-baa7-9a50cf228d73 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:44.180449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd2417-1f4a-46c6-baa7-9a50cf228d73 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:44.181329 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd2417-1f4a-46c6-baa7-9a50cf228d73 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:44.182898 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd2417-1f4a-46c6-baa7-9a50cf228d73 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:44.185732 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:44.203568 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be275508-3b11-4b55-bcc0-47cfecc12f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:44.204245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be275508-3b11-4b55-bcc0-47cfecc12f36 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:44.204527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be275508-3b11-4b55-bcc0-47cfecc12f36 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:44.204792 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be275508-3b11-4b55-bcc0-47cfecc12f36 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:44.205551 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be275508-3b11-4b55-bcc0-47cfecc12f36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:35:44.205868 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be275508-3b11-4b55-bcc0-47cfecc12f36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:44.207295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be275508-3b11-4b55-bcc0-47cfecc12f36 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:44.221068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 16/31] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:35:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:44.222213 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be275508-3b11-4b55-bcc0-47cfecc12f36 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:44.327500 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:44.340657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be275508-3b11-4b55-bcc0-47cfecc12f36 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:44.348816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 16/32] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 19:35:43 2022] POST /identity/v3/roles => generated 219 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:35:44.351122 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9303304-bb60-4716-8f41-fc8a152232b6 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:44.351653 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9303304-bb60-4716-8f41-fc8a152232b6 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:44.352135 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9303304-bb60-4716-8f41-fc8a152232b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:44.387285 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab188a-e31d-4ee3-8792-cab15fc6e7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:44.388255 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab188a-e31d-4ee3-8792-cab15fc6e7fb None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:44.388687 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab188a-e31d-4ee3-8792-cab15fc6e7fb None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:44.389093 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab188a-e31d-4ee3-8792-cab15fc6e7fb None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:44.390072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab188a-e31d-4ee3-8792-cab15fc6e7fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:35:44.390526 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab188a-e31d-4ee3-8792-cab15fc6e7fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:44.392116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab188a-e31d-4ee3-8792-cab15fc6e7fb None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:44.395803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-c9303304-bb60-4716-8f41-fc8a152232b6 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:44.397635 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-c9303304-bb60-4716-8f41-fc8a152232b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:44.407656 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 17/33] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:44.409612 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab188a-e31d-4ee3-8792-cab15fc6e7fb None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:44.482204 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 17/34] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:35:44.483376 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9303304-bb60-4716-8f41-fc8a152232b6 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:44.557967 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab188a-e31d-4ee3-8792-cab15fc6e7fb None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:44.609573 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d6212-4b28-4ff2-ba77-5764b63dd1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:44.611000 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d6212-4b28-4ff2-ba77-5764b63dd1ba None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:44.611763 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d6212-4b28-4ff2-ba77-5764b63dd1ba None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:44.612358 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d6212-4b28-4ff2-ba77-5764b63dd1ba None admin] PATH_INFO: `/v3/users/a5800b131869435eb3ae07f86d69929e` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:44.614240 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d6212-4b28-4ff2-ba77-5764b63dd1ba None admin] RBAC: Authorizing `identity:get_user(user_id=a5800b131869435eb3ae07f86d69929e)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:44.617204 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d6212-4b28-4ff2-ba77-5764b63dd1ba None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:44.620345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 18/35] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:35:44 2022] GET /identity/v3/users/a5800b131869435eb3ae07f86d69929e => generated 247 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:44.621990 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064dbef6-2c3e-471a-982b-5d20ed521bbe None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:44.622489 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064dbef6-2c3e-471a-982b-5d20ed521bbe None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:44.622956 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064dbef6-2c3e-471a-982b-5d20ed521bbe None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:44.624840 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 19/36] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:44.625705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064dbef6-2c3e-471a-982b-5d20ed521bbe None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:44.642602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9303304-bb60-4716-8f41-fc8a152232b6 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:44.652650 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:44.679685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:44.691818 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ad6c4-3d41-4c2b-af9e-bec8556264a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:44.692682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ad6c4-3d41-4c2b-af9e-bec8556264a9 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:44.692959 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ad6c4-3d41-4c2b-af9e-bec8556264a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:44.693232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ad6c4-3d41-4c2b-af9e-bec8556264a9 None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:44.693974 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ad6c4-3d41-4c2b-af9e-bec8556264a9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:44.696028 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ad6c4-3d41-4c2b-af9e-bec8556264a9 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:44.729983 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 18/37] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:35:44 2022] POST /identity/v3/projects => generated 301 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:35:44.731126 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ad6c4-3d41-4c2b-af9e-bec8556264a9 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:44.765987 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064dbef6-2c3e-471a-982b-5d20ed521bbe None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:44.824121 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14eca0-dbf9-4183-86c5-566622ade0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:44.826053 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14eca0-dbf9-4183-86c5-566622ade0f5 None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:44.826643 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14eca0-dbf9-4183-86c5-566622ade0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:44.827117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14eca0-dbf9-4183-86c5-566622ade0f5 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:44.828777 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14eca0-dbf9-4183-86c5-566622ade0f5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:44.831507 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14eca0-dbf9-4183-86c5-566622ade0f5 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:44.857030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ad6c4-3d41-4c2b-af9e-bec8556264a9 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:44.897313 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1ae60-a0b0-4020-825e-7d7a5d289b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:44.898193 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1ae60-a0b0-4020-825e-7d7a5d289b46 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:44.898507 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1ae60-a0b0-4020-825e-7d7a5d289b46 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:44.898780 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1ae60-a0b0-4020-825e-7d7a5d289b46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:44.900160 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1ae60-a0b0-4020-825e-7d7a5d289b46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:44.901717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1ae60-a0b0-4020-825e-7d7a5d289b46 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:44.903803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 19/38] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:35:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:44.904953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae75ea-0a3e-48df-bea7-b4dfad1131a1 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:44.905239 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae75ea-0a3e-48df-bea7-b4dfad1131a1 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:44.905509 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae75ea-0a3e-48df-bea7-b4dfad1131a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:44.926177 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-cdae75ea-0a3e-48df-bea7-b4dfad1131a1 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:44.927923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-cdae75ea-0a3e-48df-bea7-b4dfad1131a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:44.980512 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:44.986772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 20/39] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 19:35:44 2022] POST /identity/v3/roles => generated 225 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:35:44.987809 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14eca0-dbf9-4183-86c5-566622ade0f5 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:45.022431 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 20/40] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:35:45.022431 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482982bd-835c-4e35-af99-993b771b7897 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:45.022431 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482982bd-835c-4e35-af99-993b771b7897 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:45.022431 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482982bd-835c-4e35-af99-993b771b7897 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:45.022431 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 21/41] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:45.028012 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7d00a-12fc-4a4f-a7d9-833dbdc08793 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:45.028373 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7d00a-12fc-4a4f-a7d9-833dbdc08793 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:45.028719 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7d00a-12fc-4a4f-a7d9-833dbdc08793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:45.046349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-81e7d00a-12fc-4a4f-a7d9-833dbdc08793 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:45.047749 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-81e7d00a-12fc-4a4f-a7d9-833dbdc08793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:45.088466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:45.117157 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14eca0-dbf9-4183-86c5-566622ade0f5 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:45.124904 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 22/42] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:35:45.126076 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7d00a-12fc-4a4f-a7d9-833dbdc08793 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:45.169634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646e85c-f45d-4723-9b3a-d98bd21e92ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:45.170794 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646e85c-f45d-4723-9b3a-d98bd21e92ff None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:45.170902 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646e85c-f45d-4723-9b3a-d98bd21e92ff None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:45.171125 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646e85c-f45d-4723-9b3a-d98bd21e92ff None admin] PATH_INFO: `/v3/domains/default/users/a5800b131869435eb3ae07f86d69929e/roles/a33caf2301fb44db90304eb3d9afc3a5` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:45.173332 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646e85c-f45d-4723-9b3a-d98bd21e92ff None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a5800b131869435eb3ae07f86d69929e, role_id=a33caf2301fb44db90304eb3d9afc3a5)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:45.175341 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646e85c-f45d-4723-9b3a-d98bd21e92ff None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:45.192904 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 21/43] 10.222.0.84 () {58 vars in 1361 bytes} [Thu May 19 19:35:44 2022] PUT /identity/v3/domains/default/users/a5800b131869435eb3ae07f86d69929e/roles/a33caf2301fb44db90304eb3d9afc3a5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:35:45.194159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646e85c-f45d-4723-9b3a-d98bd21e92ff None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:45.263306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7d00a-12fc-4a4f-a7d9-833dbdc08793 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:45.265585 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:45.303667 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315f807-f428-4e53-87a7-5624f2619c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:45.304374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315f807-f428-4e53-87a7-5624f2619c9a None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:45.304654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315f807-f428-4e53-87a7-5624f2619c9a None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:45.304933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315f807-f428-4e53-87a7-5624f2619c9a None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:45.305697 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315f807-f428-4e53-87a7-5624f2619c9a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:45.307169 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315f807-f428-4e53-87a7-5624f2619c9a None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:45.325557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 23/44] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 19:35:45 2022] POST /identity/v3/roles => generated 223 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:35:45.335348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646e85c-f45d-4723-9b3a-d98bd21e92ff None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:45.379167 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12c82c-a246-4129-b888-63b0d4af8a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:45.380076 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12c82c-a246-4129-b888-63b0d4af8a4f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:45.380844 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12c82c-a246-4129-b888-63b0d4af8a4f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:45.380844 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12c82c-a246-4129-b888-63b0d4af8a4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:45.381931 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12c82c-a246-4129-b888-63b0d4af8a4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:45.383487 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12c82c-a246-4129-b888-63b0d4af8a4f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:45.385456 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 22/45] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:35:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:45.391229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315f807-f428-4e53-87a7-5624f2619c9a None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:45.509651 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315f807-f428-4e53-87a7-5624f2619c9a None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:45.549053 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cc740-0c56-413c-ac83-17feb36399cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:45.549724 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cc740-0c56-413c-ac83-17feb36399cd None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:45.550011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cc740-0c56-413c-ac83-17feb36399cd None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:45.550316 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cc740-0c56-413c-ac83-17feb36399cd None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:45.551044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cc740-0c56-413c-ac83-17feb36399cd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:45.553254 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cc740-0c56-413c-ac83-17feb36399cd None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:45.581440 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 24/46] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:35:45 2022] POST /identity/v3/projects => generated 312 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:35:47.420386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6b216-0cf4-460c-940c-7d7a69879904 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:47.420930 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6b216-0cf4-460c-940c-7d7a69879904 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:47.420930 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6b216-0cf4-460c-940c-7d7a69879904 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:47.422382 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 23/47] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:47.427801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c243b-6d4d-4611-89bf-58d6d6516112 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:47.428215 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c243b-6d4d-4611-89bf-58d6d6516112 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:47.428426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c243b-6d4d-4611-89bf-58d6d6516112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:47.445684 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-171c243b-6d4d-4611-89bf-58d6d6516112 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:47.446901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-171c243b-6d4d-4611-89bf-58d6d6516112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:47.458455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:47.518153 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 25/48] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:35:47.541407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6b216-0cf4-460c-940c-7d7a69879904 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:47.551602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:47.651949 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6b216-0cf4-460c-940c-7d7a69879904 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:47.689567 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e74072-4baa-4979-a776-7a87fb19cd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:47.690302 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e74072-4baa-4979-a776-7a87fb19cd76 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:47.690597 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e74072-4baa-4979-a776-7a87fb19cd76 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:47.690874 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e74072-4baa-4979-a776-7a87fb19cd76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:47.703031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e74072-4baa-4979-a776-7a87fb19cd76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-07e74072-4baa-4979-a776-7a87fb19cd76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:35:47.711846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:35:47.712675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:35:47.713407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:35:47.713407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:35:47.713407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:35:47.713407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:35:47.713407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:35:47.713407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:35:47.713407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:35:47.713407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:35:47.713407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:35:47.713407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:35:47.713407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:35:47.713407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:35:47.713407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 24/49] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:35:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:35:47.719196 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c243b-6d4d-4611-89bf-58d6d6516112 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:47.819569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c243b-6d4d-4611-89bf-58d6d6516112 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:47.828197 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92466105-fdea-42f0-be65-d81ccff15655 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:47.828522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92466105-fdea-42f0-be65-d81ccff15655 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:47.828806 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92466105-fdea-42f0-be65-d81ccff15655 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:47.830239 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 25/50] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:47.835529 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a5f62-cabc-4447-9c06-e83d77f5abe2 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:47.835845 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a5f62-cabc-4447-9c06-e83d77f5abe2 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:47.836125 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a5f62-cabc-4447-9c06-e83d77f5abe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:47.854124 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-dd4a5f62-cabc-4447-9c06-e83d77f5abe2 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:47.855657 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc61ca-a221-45bb-9322-0a926fd4f765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:47.855802 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-dd4a5f62-cabc-4447-9c06-e83d77f5abe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:47.856335 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc61ca-a221-45bb-9322-0a926fd4f765 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:47.856618 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc61ca-a221-45bb-9322-0a926fd4f765 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:47.856893 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc61ca-a221-45bb-9322-0a926fd4f765 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:47.857656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc61ca-a221-45bb-9322-0a926fd4f765 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:35:47.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc61ca-a221-45bb-9322-0a926fd4f765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:47.859414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc61ca-a221-45bb-9322-0a926fd4f765 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:47.868418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 26/51] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:35:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:47.874010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc61ca-a221-45bb-9322-0a926fd4f765 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:47.884266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:47.917503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:47.925651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 26/52] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:35:47.948440 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a5f62-cabc-4447-9c06-e83d77f5abe2 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:47.972921 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc61ca-a221-45bb-9322-0a926fd4f765 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:48.006600 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7b527-e4ef-40b7-a76c-27e9c55538bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:48.007457 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7b527-e4ef-40b7-a76c-27e9c55538bc None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:48.007731 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7b527-e4ef-40b7-a76c-27e9c55538bc None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:48.008001 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7b527-e4ef-40b7-a76c-27e9c55538bc None admin] PATH_INFO: `/v3/users/a5800b131869435eb3ae07f86d69929e` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:48.009228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7b527-e4ef-40b7-a76c-27e9c55538bc None admin] RBAC: Authorizing `identity:get_user(user_id=a5800b131869435eb3ae07f86d69929e)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:48.010729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7b527-e4ef-40b7-a76c-27e9c55538bc None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:48.012329 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 27/53] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:35:47 2022] GET /identity/v3/users/a5800b131869435eb3ae07f86d69929e => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:35:48.017728 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7b527-e4ef-40b7-a76c-27e9c55538bc None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:48.056868 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a5f62-cabc-4447-9c06-e83d77f5abe2 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:48.089857 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c202aee-44f1-4285-b31b-50e4bd23ac51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:48.090691 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c202aee-44f1-4285-b31b-50e4bd23ac51 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:48.090972 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c202aee-44f1-4285-b31b-50e4bd23ac51 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:48.091242 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c202aee-44f1-4285-b31b-50e4bd23ac51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:48.092409 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c202aee-44f1-4285-b31b-50e4bd23ac51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:48.093881 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c202aee-44f1-4285-b31b-50e4bd23ac51 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:48.095543 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 27/54] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:35:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:35:48.098124 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:48.101326 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c202aee-44f1-4285-b31b-50e4bd23ac51 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:48.122619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7b527-e4ef-40b7-a76c-27e9c55538bc None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:48.155742 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a200e32-c0f3-4e29-9ed2-377ff044a9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:48.156544 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a200e32-c0f3-4e29-9ed2-377ff044a9d3 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:48.156822 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a200e32-c0f3-4e29-9ed2-377ff044a9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:48.157104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a200e32-c0f3-4e29-9ed2-377ff044a9d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:48.158360 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a200e32-c0f3-4e29-9ed2-377ff044a9d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:48.159914 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a200e32-c0f3-4e29-9ed2-377ff044a9d3 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:48.161749 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 28/55] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:35:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:35:48.168619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a200e32-c0f3-4e29-9ed2-377ff044a9d3 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:48.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:48.210659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c202aee-44f1-4285-b31b-50e4bd23ac51 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:48.246965 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e53f00-c007-4755-a33b-1ce94ff90675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:48.248153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e53f00-c007-4755-a33b-1ce94ff90675 None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:48.248153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e53f00-c007-4755-a33b-1ce94ff90675 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:48.248360 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e53f00-c007-4755-a33b-1ce94ff90675 None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:48.249070 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e53f00-c007-4755-a33b-1ce94ff90675 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:48.250720 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e53f00-c007-4755-a33b-1ce94ff90675 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:48.277461 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a200e32-c0f3-4e29-9ed2-377ff044a9d3 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:48.283977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 28/56] 10.222.0.84 () {60 vars in 1140 bytes} [Thu May 19 19:35:48 2022] POST /identity/v3/projects => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:35:48.314885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323ad49-8cad-4bcd-a59c-fe2a3f41d6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:48.315567 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323ad49-8cad-4bcd-a59c-fe2a3f41d6dd None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:48.315848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323ad49-8cad-4bcd-a59c-fe2a3f41d6dd None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:48.316115 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323ad49-8cad-4bcd-a59c-fe2a3f41d6dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:48.316962 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323ad49-8cad-4bcd-a59c-fe2a3f41d6dd None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.domain.id=default, user.id=a5800b131869435eb3ae07f86d69929e) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:35:48.317327 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323ad49-8cad-4bcd-a59c-fe2a3f41d6dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:48.319027 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323ad49-8cad-4bcd-a59c-fe2a3f41d6dd None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:48.326140 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 29/57] 10.222.0.84 () {56 vars in 1322 bytes} [Thu May 19 19:35:48 2022] GET /identity/v3/role_assignments?user.id=a5800b131869435eb3ae07f86d69929e&scope.domain.id=default&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:35:50.553702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c664477-cde5-4992-8b1c-3c5e99177eab None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:50.554641 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c664477-cde5-4992-8b1c-3c5e99177eab None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:50.554641 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c664477-cde5-4992-8b1c-3c5e99177eab None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:50.557054 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 29/58] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:50.563297 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfec8e-b40b-45d8-b8fc-93cca9e5cbb0 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:50.563981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfec8e-b40b-45d8-b8fc-93cca9e5cbb0 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:50.564166 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfec8e-b40b-45d8-b8fc-93cca9e5cbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:50.588350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-e3bfec8e-b40b-45d8-b8fc-93cca9e5cbb0 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:50.589526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-e3bfec8e-b40b-45d8-b8fc-93cca9e5cbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:50.591447 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:50.660743 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 30/59] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:35:50.685909 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c664477-cde5-4992-8b1c-3c5e99177eab None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:50.801812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c664477-cde5-4992-8b1c-3c5e99177eab None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:50.818546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:50.840387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398afc05-b86c-4f57-8180-1f13f229440f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:50.841182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398afc05-b86c-4f57-8180-1f13f229440f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:50.841466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398afc05-b86c-4f57-8180-1f13f229440f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:50.841742 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398afc05-b86c-4f57-8180-1f13f229440f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:50.843009 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398afc05-b86c-4f57-8180-1f13f229440f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:50.844508 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398afc05-b86c-4f57-8180-1f13f229440f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:50.846147 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 30/60] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:35:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:50.854159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfec8e-b40b-45d8-b8fc-93cca9e5cbb0 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:50.954821 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfec8e-b40b-45d8-b8fc-93cca9e5cbb0 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:50.990014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f8dee-c91b-48ec-ad3e-27668e13fee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:50.990697 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f8dee-c91b-48ec-ad3e-27668e13fee8 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:50.990975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f8dee-c91b-48ec-ad3e-27668e13fee8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:50.991244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f8dee-c91b-48ec-ad3e-27668e13fee8 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:50.991955 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f8dee-c91b-48ec-ad3e-27668e13fee8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:50.993421 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f8dee-c91b-48ec-ad3e-27668e13fee8 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:51.006219 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:51.023421 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 31/61] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:35:50 2022] POST /identity/v3/users => generated 275 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:35:54.286464 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc3c7a-920c-4c10-86d2-33b18b3a6ad7 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:54.286988 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc3c7a-920c-4c10-86d2-33b18b3a6ad7 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:54.287061 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc3c7a-920c-4c10-86d2-33b18b3a6ad7 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:54.288406 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 31/62] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:54.304393 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f7a6c-e29e-42d3-b760-b6525fe65a16 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:54.304890 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f7a6c-e29e-42d3-b760-b6525fe65a16 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:54.304961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f7a6c-e29e-42d3-b760-b6525fe65a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:54.331316 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-a24f7a6c-e29e-42d3-b760-b6525fe65a16 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:54.332675 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-a24f7a6c-e29e-42d3-b760-b6525fe65a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:54.405524 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 32/63] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:35:54.429298 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc3c7a-920c-4c10-86d2-33b18b3a6ad7 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:54.580014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc3c7a-920c-4c10-86d2-33b18b3a6ad7 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:54.587261 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f87448-0542-431e-86af-dd715f8c0e5e None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:54.587551 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f87448-0542-431e-86af-dd715f8c0e5e None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:54.587887 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f87448-0542-431e-86af-dd715f8c0e5e None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:54.589207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 33/64] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:35:54.595150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462f09b-3f95-460d-be32-59449190ceab None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:54.595434 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462f09b-3f95-460d-be32-59449190ceab None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:54.595707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462f09b-3f95-460d-be32-59449190ceab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:54.619473 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-2462f09b-3f95-460d-be32-59449190ceab None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:54.619473 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-2462f09b-3f95-460d-be32-59449190ceab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:54.698332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 34/65] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:35:54.719578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265cb58-da6e-42be-b0a3-f487d99051a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:54.720541 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265cb58-da6e-42be-b0a3-f487d99051a7 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:54.720830 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265cb58-da6e-42be-b0a3-f487d99051a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:54.721107 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265cb58-da6e-42be-b0a3-f487d99051a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:54.721457 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56e680-eb7a-422e-a954-59141aada099 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:54.722056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56e680-eb7a-422e-a954-59141aada099 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:54.722567 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56e680-eb7a-422e-a954-59141aada099 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:54.725076 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 35/66] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:54.729347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56e680-eb7a-422e-a954-59141aada099 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:54.735817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265cb58-da6e-42be-b0a3-f487d99051a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-c265cb58-da6e-42be-b0a3-f487d99051a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:35:54.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:35:54.744549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:35:54.745722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:35:54.745722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:35:54.745722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:35:54.745722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:35:54.745722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:35:54.745722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:35:54.745722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:35:54.745722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:35:54.745722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:35:54.745722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:35:54.745722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:35:54.745722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:35:54.745722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 32/67] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:35:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:35:54.746586 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040df5c-0dc7-405a-b51c-1f2ed08dd729 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:54.746586 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040df5c-0dc7-405a-b51c-1f2ed08dd729 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:54.746730 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040df5c-0dc7-405a-b51c-1f2ed08dd729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:54.767669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-c040df5c-0dc7-405a-b51c-1f2ed08dd729 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:54.768860 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-c040df5c-0dc7-405a-b51c-1f2ed08dd729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:54.814671 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:54.817103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:54.847661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 33/68] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:35:54.849359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040df5c-0dc7-405a-b51c-1f2ed08dd729 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:54.898608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56e680-eb7a-422e-a954-59141aada099 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:54.934959 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9793d-f4b3-4a7a-bc5b-90d93e288dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:54.936160 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9793d-f4b3-4a7a-bc5b-90d93e288dcc None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:54.936631 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9793d-f4b3-4a7a-bc5b-90d93e288dcc None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:54.937240 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9793d-f4b3-4a7a-bc5b-90d93e288dcc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:54.948691 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9793d-f4b3-4a7a-bc5b-90d93e288dcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-15a9793d-f4b3-4a7a-bc5b-90d93e288dcc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:35:54.957418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:35:54.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:35:54.959171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:35:54.959171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:35:54.959171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:35:54.959171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:35:54.959171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:35:54.959171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:35:54.959171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:35:54.959171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:35:54.959171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:35:54.959171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:35:54.959171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:35:54.959171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:35:54.961619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 36/69] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:35:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:35:54.962650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9793d-f4b3-4a7a-bc5b-90d93e288dcc None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:54.983946 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040df5c-0dc7-405a-b51c-1f2ed08dd729 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:55.019788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03dc04-8d3a-405d-9edb-6f5c795bf906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:55.020754 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03dc04-8d3a-405d-9edb-6f5c795bf906 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:55.021038 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03dc04-8d3a-405d-9edb-6f5c795bf906 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:55.021365 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03dc04-8d3a-405d-9edb-6f5c795bf906 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:55.022122 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03dc04-8d3a-405d-9edb-6f5c795bf906 None admin] RBAC: Adding query filter params (name=member) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:35:55.022497 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03dc04-8d3a-405d-9edb-6f5c795bf906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:55.023903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03dc04-8d3a-405d-9edb-6f5c795bf906 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:55.037570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 34/70] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:35:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:55.037570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03dc04-8d3a-405d-9edb-6f5c795bf906 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:55.068909 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:55.071038 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9793d-f4b3-4a7a-bc5b-90d93e288dcc None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:55.084538 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:55.106159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e4878-11cb-4d82-ac0f-3b172891e98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:55.106918 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e4878-11cb-4d82-ac0f-3b172891e98e None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:55.107198 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e4878-11cb-4d82-ac0f-3b172891e98e None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:55.107465 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e4878-11cb-4d82-ac0f-3b172891e98e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:55.117771 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e4878-11cb-4d82-ac0f-3b172891e98e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-321e4878-11cb-4d82-ac0f-3b172891e98e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:35:55.125328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:35:55.126150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:35:55.126950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:35:55.126950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:35:55.126950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:35:55.126950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:35:55.126950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:35:55.126950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:35:55.126950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:35:55.126950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:35:55.126950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:35:55.126950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:35:55.126950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:35:55.126950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:35:55.126950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 37/71] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:35:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:35:55.127802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d32e92-7757-4568-9b5d-2a669535404b None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:55.128108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d32e92-7757-4568-9b5d-2a669535404b None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:55.128350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d32e92-7757-4568-9b5d-2a669535404b None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:55.129622 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 38/72] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:55.130315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d32e92-7757-4568-9b5d-2a669535404b None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:55.140233 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03dc04-8d3a-405d-9edb-6f5c795bf906 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:55.173636 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5bbe5-95e1-42aa-8cf0-4ffa3c75f65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:55.174421 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5bbe5-95e1-42aa-8cf0-4ffa3c75f65e None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:55.174708 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5bbe5-95e1-42aa-8cf0-4ffa3c75f65e None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:55.174980 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5bbe5-95e1-42aa-8cf0-4ffa3c75f65e None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:55.175731 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5bbe5-95e1-42aa-8cf0-4ffa3c75f65e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:35:55.176038 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5bbe5-95e1-42aa-8cf0-4ffa3c75f65e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:55.177480 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5bbe5-95e1-42aa-8cf0-4ffa3c75f65e None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:55.185327 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 35/73] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:35:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:55.187525 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6160c-92a3-4e27-850b-fecd3ca220d2 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:55.187812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6160c-92a3-4e27-850b-fecd3ca220d2 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:55.188077 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6160c-92a3-4e27-850b-fecd3ca220d2 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:55.189349 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 36/74] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:55.191079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6160c-92a3-4e27-850b-fecd3ca220d2 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:55.234108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d32e92-7757-4568-9b5d-2a669535404b None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:55.266230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa94064-a207-4de0-9878-840d02dee7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:55.267128 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa94064-a207-4de0-9878-840d02dee7d4 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:55.267407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa94064-a207-4de0-9878-840d02dee7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:55.267676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa94064-a207-4de0-9878-840d02dee7d4 None admin] PATH_INFO: `/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:55.281324 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa94064-a207-4de0-9878-840d02dee7d4 None admin] RBAC: Authorizing `identity:get_user(user_id=6c5e8aab0eaa42288cf4b090c1e64ae7)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:55.282857 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa94064-a207-4de0-9878-840d02dee7d4 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:55.284519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 39/75] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:35:55 2022] GET /identity/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:55.285736 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace6320-14cc-4648-a9db-1913204accaa None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:55.286176 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace6320-14cc-4648-a9db-1913204accaa None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:55.286464 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace6320-14cc-4648-a9db-1913204accaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:55.291817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:55.297515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6160c-92a3-4e27-850b-fecd3ca220d2 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:55.302900 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-6ace6320-14cc-4648-a9db-1913204accaa None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:55.304074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-6ace6320-14cc-4648-a9db-1913204accaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:55.330094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:55.335736 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a12e79-3d21-4c45-b8c5-3105e49b7006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:55.336394 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a12e79-3d21-4c45-b8c5-3105e49b7006 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:55.336678 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a12e79-3d21-4c45-b8c5-3105e49b7006 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:55.336958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a12e79-3d21-4c45-b8c5-3105e49b7006 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:55.337735 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a12e79-3d21-4c45-b8c5-3105e49b7006 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:35:55.338044 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a12e79-3d21-4c45-b8c5-3105e49b7006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:55.339490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a12e79-3d21-4c45-b8c5-3105e49b7006 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:55.347714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 37/76] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:35:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:55.348551 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a12e79-3d21-4c45-b8c5-3105e49b7006 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:55.377718 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 40/77] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:35:55.384451 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e02202-e509-4d5c-b435-c2471a9745df None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:55.384745 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e02202-e509-4d5c-b435-c2471a9745df None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:55.385038 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e02202-e509-4d5c-b435-c2471a9745df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:55.404861 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-b1e02202-e509-4d5c-b435-c2471a9745df None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:55.406069 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-b1e02202-e509-4d5c-b435-c2471a9745df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:55.449288 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a12e79-3d21-4c45-b8c5-3105e49b7006 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:55.472692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 41/78] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:35:55.473525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e02202-e509-4d5c-b435-c2471a9745df None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:55.494026 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b921a8e-c2e4-4f07-8219-972009941bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:55.495399 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b921a8e-c2e4-4f07-8219-972009941bb4 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:55.495755 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b921a8e-c2e4-4f07-8219-972009941bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:55.496008 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b921a8e-c2e4-4f07-8219-972009941bb4 None admin] PATH_INFO: `/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:55.497183 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b921a8e-c2e4-4f07-8219-972009941bb4 None admin] RBAC: Authorizing `identity:get_user(user_id=6c5e8aab0eaa42288cf4b090c1e64ae7)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:55.498685 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b921a8e-c2e4-4f07-8219-972009941bb4 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:55.500462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 38/79] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:35:55 2022] GET /identity/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:55.501078 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b921a8e-c2e4-4f07-8219-972009941bb4 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:55.531894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:55.564209 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:55.574283 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e02202-e509-4d5c-b435-c2471a9745df None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:55.603337 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b921a8e-c2e4-4f07-8219-972009941bb4 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:55.609086 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc67e9f-0dec-407d-86fb-d41cbcc7b8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:55.609744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc67e9f-0dec-407d-86fb-d41cbcc7b8e8 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:55.610025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc67e9f-0dec-407d-86fb-d41cbcc7b8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:55.610730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc67e9f-0dec-407d-86fb-d41cbcc7b8e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:55.639743 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa2a12-7023-4718-97b5-cd092a6e6521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:55.640565 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa2a12-7023-4718-97b5-cd092a6e6521 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:55.640849 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa2a12-7023-4718-97b5-cd092a6e6521 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:55.641121 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa2a12-7023-4718-97b5-cd092a6e6521 None admin] PATH_INFO: `/v3/users/a5800b131869435eb3ae07f86d69929e` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:55.642374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa2a12-7023-4718-97b5-cd092a6e6521 None admin] RBAC: Authorizing `identity:get_user(user_id=a5800b131869435eb3ae07f86d69929e)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:55.643834 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa2a12-7023-4718-97b5-cd092a6e6521 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:55.645738 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 39/80] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:35:55 2022] GET /identity/v3/users/a5800b131869435eb3ae07f86d69929e => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:55.646535 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa2a12-7023-4718-97b5-cd092a6e6521 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:55.682524 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc67e9f-0dec-407d-86fb-d41cbcc7b8e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:55.684000 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc67e9f-0dec-407d-86fb-d41cbcc7b8e8 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:55.740896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:55.747411 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa2a12-7023-4718-97b5-cd092a6e6521 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:55.771792 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:55.781263 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db041d09-17a7-4491-82bd-14afd9a7c305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:55.781853 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 42/81] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:35:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:35:55.782113 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db041d09-17a7-4491-82bd-14afd9a7c305 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:55.782424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db041d09-17a7-4491-82bd-14afd9a7c305 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:55.782701 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db041d09-17a7-4491-82bd-14afd9a7c305 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:55.782808 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc67e9f-0dec-407d-86fb-d41cbcc7b8e8 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:55.795506 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db041d09-17a7-4491-82bd-14afd9a7c305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-db041d09-17a7-4491-82bd-14afd9a7c305 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:35:55.803192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:35:55.804003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:35:55.804740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:35:55.804740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:35:55.804740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:35:55.804740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:35:55.804740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:35:55.804740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:35:55.804740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:35:55.804740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:35:55.804740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:35:55.804740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:35:55.804740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:35:55.804740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:35:55.804740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 40/82] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:35:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:35:55.805353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db041d09-17a7-4491-82bd-14afd9a7c305 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:55.887785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc67e9f-0dec-407d-86fb-d41cbcc7b8e8 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:55.904890 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db041d09-17a7-4491-82bd-14afd9a7c305 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:55.922200 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7905d-bfe8-4f57-8747-d70e14cb3625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:55.923023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7905d-bfe8-4f57-8747-d70e14cb3625 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:55.923303 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7905d-bfe8-4f57-8747-d70e14cb3625 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:55.923565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7905d-bfe8-4f57-8747-d70e14cb3625 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:55.924734 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7905d-bfe8-4f57-8747-d70e14cb3625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:55.926200 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7905d-bfe8-4f57-8747-d70e14cb3625 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:55.928073 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 43/83] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:35:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:55.932668 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7905d-bfe8-4f57-8747-d70e14cb3625 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:55.942998 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1d586-94ac-4810-9372-102e26b663ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:55.943655 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1d586-94ac-4810-9372-102e26b663ac None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:55.943973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1d586-94ac-4810-9372-102e26b663ac None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:55.944214 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1d586-94ac-4810-9372-102e26b663ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:55.969394 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:56.002895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:56.016055 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1d586-94ac-4810-9372-102e26b663ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:56.017587 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1d586-94ac-4810-9372-102e26b663ac None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:56.045206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7905d-bfe8-4f57-8747-d70e14cb3625 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:56.079775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc8c7d-9333-498a-b838-b3acc72f9d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:56.080900 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc8c7d-9333-498a-b838-b3acc72f9d62 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:56.081174 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc8c7d-9333-498a-b838-b3acc72f9d62 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:56.081599 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc8c7d-9333-498a-b838-b3acc72f9d62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:56.115223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 41/84] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:35:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:35:56.116078 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1d586-94ac-4810-9372-102e26b663ac None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:56.152059 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc8c7d-9333-498a-b838-b3acc72f9d62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:56.153540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc8c7d-9333-498a-b838-b3acc72f9d62 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:56.169691 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:56.208745 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:56.212388 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1d586-94ac-4810-9372-102e26b663ac None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:56.245497 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5366af-cec8-4847-8ea6-b0a85bfafa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:56.246451 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5366af-cec8-4847-8ea6-b0a85bfafa3f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:56.246660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5366af-cec8-4847-8ea6-b0a85bfafa3f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:56.246931 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5366af-cec8-4847-8ea6-b0a85bfafa3f None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:56.248115 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5366af-cec8-4847-8ea6-b0a85bfafa3f None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:56.249602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5366af-cec8-4847-8ea6-b0a85bfafa3f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:56.250594 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 44/85] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:35:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:35:56.251466 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc8c7d-9333-498a-b838-b3acc72f9d62 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:56.256148 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 42/86] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:35:56 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:56.256921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5366af-cec8-4847-8ea6-b0a85bfafa3f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:56.353473 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc8c7d-9333-498a-b838-b3acc72f9d62 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:56.360368 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5366af-cec8-4847-8ea6-b0a85bfafa3f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:56.384352 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:56.388509 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4629a-bd6b-4c04-a487-abd1c7c1bd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:56.389270 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4629a-bd6b-4c04-a487-abd1c7c1bd9b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:56.389545 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4629a-bd6b-4c04-a487-abd1c7c1bd9b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:56.389846 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4629a-bd6b-4c04-a487-abd1c7c1bd9b None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:56.390627 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4629a-bd6b-4c04-a487-abd1c7c1bd9b None admin] RBAC: Adding query filter params (name=member) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:35:56.390930 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4629a-bd6b-4c04-a487-abd1c7c1bd9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:56.392342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4629a-bd6b-4c04-a487-abd1c7c1bd9b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:56.396213 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2209f1b-0180-456c-8c89-86f635a843e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:56.396885 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2209f1b-0180-456c-8c89-86f635a843e1 None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:56.397173 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2209f1b-0180-456c-8c89-86f635a843e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:56.397468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2209f1b-0180-456c-8c89-86f635a843e1 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:56.398164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2209f1b-0180-456c-8c89-86f635a843e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:56.399626 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2209f1b-0180-456c-8c89-86f635a843e1 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:56.400646 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 45/87] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:35:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:56.401429 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4629a-bd6b-4c04-a487-abd1c7c1bd9b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:56.425346 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 43/88] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:35:56 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:35:56.426097 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2209f1b-0180-456c-8c89-86f635a843e1 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:56.439174 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:56.530816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4629a-bd6b-4c04-a487-abd1c7c1bd9b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:56.561263 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2209f1b-0180-456c-8c89-86f635a843e1 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:56.569118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489c40b-78e9-4d9e-9e61-894f3f53605b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:56.570005 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489c40b-78e9-4d9e-9e61-894f3f53605b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:56.570300 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489c40b-78e9-4d9e-9e61-894f3f53605b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:56.570597 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489c40b-78e9-4d9e-9e61-894f3f53605b None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:56.571851 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489c40b-78e9-4d9e-9e61-894f3f53605b None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:56.573350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489c40b-78e9-4d9e-9e61-894f3f53605b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:56.575320 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 46/89] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:35:56 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:56.575956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489c40b-78e9-4d9e-9e61-894f3f53605b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:56.601870 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae224c63-8634-48df-8d07-fae7b4a2b785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:56.603118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae224c63-8634-48df-8d07-fae7b4a2b785 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:56.603408 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae224c63-8634-48df-8d07-fae7b4a2b785 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:56.603696 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae224c63-8634-48df-8d07-fae7b4a2b785 None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:56.604927 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae224c63-8634-48df-8d07-fae7b4a2b785 None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:56.606459 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae224c63-8634-48df-8d07-fae7b4a2b785 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:56.608199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 44/90] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:35:56 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:56.613970 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae224c63-8634-48df-8d07-fae7b4a2b785 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:56.683534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:56.684356 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:56.696489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489c40b-78e9-4d9e-9e61-894f3f53605b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:56.744024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fcfcb-378b-48c0-9088-702862ffb086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:56.744812 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fcfcb-378b-48c0-9088-702862ffb086 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:56.745144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fcfcb-378b-48c0-9088-702862ffb086 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:56.745482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fcfcb-378b-48c0-9088-702862ffb086 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:56.746468 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9fcfcb-378b-48c0-9088-702862ffb086 None admin] RBAC: Adding query filter params (role.id=fe332203b7e04d6084e26c5be292af35, scope.project.id=644a8176f5874355bc660ed98897307f, user.id=6c5e8aab0eaa42288cf4b090c1e64ae7) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:35:56.746846 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9fcfcb-378b-48c0-9088-702862ffb086 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:56.747584 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae224c63-8634-48df-8d07-fae7b4a2b785 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:56.748391 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9fcfcb-378b-48c0-9088-702862ffb086 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:56.755330 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 47/91] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:35:56 2022] GET /identity/v3/role_assignments?user.id=6c5e8aab0eaa42288cf4b090c1e64ae7&scope.project.id=644a8176f5874355bc660ed98897307f&role.id=fe332203b7e04d6084e26c5be292af35 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:56.756197 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fcfcb-378b-48c0-9088-702862ffb086 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:56.789473 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac49f9-9a47-4b98-be37-853e4d71cab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:56.790622 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac49f9-9a47-4b98-be37-853e4d71cab1 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:56.791280 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac49f9-9a47-4b98-be37-853e4d71cab1 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:56.791607 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac49f9-9a47-4b98-be37-853e4d71cab1 None admin] PATH_INFO: `/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:56.793599 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac49f9-9a47-4b98-be37-853e4d71cab1 None admin] RBAC: Authorizing `identity:get_user(user_id=6c5e8aab0eaa42288cf4b090c1e64ae7)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:56.796091 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac49f9-9a47-4b98-be37-853e4d71cab1 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:56.798050 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 45/92] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:35:56 2022] GET /identity/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:56.798960 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac49f9-9a47-4b98-be37-853e4d71cab1 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:56.886353 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fcfcb-378b-48c0-9088-702862ffb086 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:56.925502 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d49ad0-a3ee-4c1d-8827-6d2d55bce1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:56.926178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d49ad0-a3ee-4c1d-8827-6d2d55bce1ca None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:56.926522 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d49ad0-a3ee-4c1d-8827-6d2d55bce1ca None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:56.926857 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d49ad0-a3ee-4c1d-8827-6d2d55bce1ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:56.927829 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d49ad0-a3ee-4c1d-8827-6d2d55bce1ca None admin] RBAC: Adding query filter params (role.id=4a241925b6cc4218b4f2af580b151877, scope.project.id=644a8176f5874355bc660ed98897307f, user.id=6c5e8aab0eaa42288cf4b090c1e64ae7) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:35:56.928151 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d49ad0-a3ee-4c1d-8827-6d2d55bce1ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:56.929740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d49ad0-a3ee-4c1d-8827-6d2d55bce1ca None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:56.936050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 48/93] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:35:56 2022] GET /identity/v3/role_assignments?user.id=6c5e8aab0eaa42288cf4b090c1e64ae7&scope.project.id=644a8176f5874355bc660ed98897307f&role.id=4a241925b6cc4218b4f2af580b151877 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:56.936944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d49ad0-a3ee-4c1d-8827-6d2d55bce1ca None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:56.938723 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac49f9-9a47-4b98-be37-853e4d71cab1 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:56.959379 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:56.961191 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:56.991816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c24242-b714-43e8-9536-95f6527f1cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:56.992519 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c24242-b714-43e8-9536-95f6527f1cc1 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:56.992806 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c24242-b714-43e8-9536-95f6527f1cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:56.993088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c24242-b714-43e8-9536-95f6527f1cc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:56.994692 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c24242-b714-43e8-9536-95f6527f1cc1 None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=644a8176f5874355bc660ed98897307f, user.id=a5800b131869435eb3ae07f86d69929e) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:35:56.995016 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c24242-b714-43e8-9536-95f6527f1cc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:56.997286 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c24242-b714-43e8-9536-95f6527f1cc1 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:57.009315 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 46/94] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:35:56 2022] GET /identity/v3/role_assignments?user.id=a5800b131869435eb3ae07f86d69929e&scope.project.id=644a8176f5874355bc660ed98897307f&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:57.085214 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d49ad0-a3ee-4c1d-8827-6d2d55bce1ca None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:57.128085 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44f523-3ce9-4c43-ab6f-480c67ff13ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:57.128762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44f523-3ce9-4c43-ab6f-480c67ff13ce None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:57.129044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44f523-3ce9-4c43-ab6f-480c67ff13ce None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:57.129314 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44f523-3ce9-4c43-ab6f-480c67ff13ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:57.208680 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44f523-3ce9-4c43-ab6f-480c67ff13ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:57.210077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44f523-3ce9-4c43-ab6f-480c67ff13ce None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:57.232176 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:57.325951 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 49/95] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:35:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:35:57.331426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c24242-b714-43e8-9536-95f6527f1cc1 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:57.441257 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c24242-b714-43e8-9536-95f6527f1cc1 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:57.477170 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df555531-9c88-41f3-93ba-2ceff4203bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:57.478310 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df555531-9c88-41f3-93ba-2ceff4203bac None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:57.478598 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df555531-9c88-41f3-93ba-2ceff4203bac None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:57.478871 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df555531-9c88-41f3-93ba-2ceff4203bac None admin] PATH_INFO: `/v3/projects/07ad62c92dd548cb8b7de09cf8881d4b` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:57.480132 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df555531-9c88-41f3-93ba-2ceff4203bac None admin] RBAC: Authorizing `identity:get_project(project_id=07ad62c92dd548cb8b7de09cf8881d4b)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:57.481590 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df555531-9c88-41f3-93ba-2ceff4203bac None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:57.483532 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 47/96] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:35:57 2022] GET /identity/v3/projects/07ad62c92dd548cb8b7de09cf8881d4b => generated 312 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:57.489307 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44f523-3ce9-4c43-ab6f-480c67ff13ce None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:57.593184 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44f523-3ce9-4c43-ab6f-480c67ff13ce None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:57.612108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:57.632423 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35445c-9926-4a99-861d-a33ced6632ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:57.633098 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35445c-9926-4a99-861d-a33ced6632ba None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:57.633380 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35445c-9926-4a99-861d-a33ced6632ba None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:57.633655 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35445c-9926-4a99-861d-a33ced6632ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:57.634536 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35445c-9926-4a99-861d-a33ced6632ba None admin] RBAC: Adding query filter params (role.id=fe332203b7e04d6084e26c5be292af35, scope.project.id=07ad62c92dd548cb8b7de09cf8881d4b, user.id=6c5e8aab0eaa42288cf4b090c1e64ae7) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:35:57.634849 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35445c-9926-4a99-861d-a33ced6632ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:57.636323 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35445c-9926-4a99-861d-a33ced6632ba None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:57.642646 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 50/97] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:35:57 2022] GET /identity/v3/role_assignments?user.id=6c5e8aab0eaa42288cf4b090c1e64ae7&scope.project.id=07ad62c92dd548cb8b7de09cf8881d4b&role.id=fe332203b7e04d6084e26c5be292af35 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:59.701549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189980c-bbaf-47ae-9d0f-cd00532cad35 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:59.702590 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189980c-bbaf-47ae-9d0f-cd00532cad35 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:59.703080 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189980c-bbaf-47ae-9d0f-cd00532cad35 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:59.704702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 48/98] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:35:59.710548 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c19730-41aa-4dff-9fd6-35a1d8c756d8 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:59.711378 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c19730-41aa-4dff-9fd6-35a1d8c756d8 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:59.711872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c19730-41aa-4dff-9fd6-35a1d8c756d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:59.730196 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-d2c19730-41aa-4dff-9fd6-35a1d8c756d8 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:35:59.731759 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-d2c19730-41aa-4dff-9fd6-35a1d8c756d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:35:59.804951 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 51/99] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:35:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:35:59.828541 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189980c-bbaf-47ae-9d0f-cd00532cad35 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:35:59.884307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:35:59.941724 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189980c-bbaf-47ae-9d0f-cd00532cad35 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:35:59.978710 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2da0f-416f-4f49-aeed-4db19a985a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:35:59.979914 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2da0f-416f-4f49-aeed-4db19a985a31 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:35:59.980390 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2da0f-416f-4f49-aeed-4db19a985a31 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:35:59.980865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2da0f-416f-4f49-aeed-4db19a985a31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:35:59.982284 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2da0f-416f-4f49-aeed-4db19a985a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:35:59.983943 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2da0f-416f-4f49-aeed-4db19a985a31 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:35:59.986084 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 49/100] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:35:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:35:59.995410 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c19730-41aa-4dff-9fd6-35a1d8c756d8 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:00.053189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d493a-d6ad-4c2b-b5eb-36208073968e None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:00.053454 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d493a-d6ad-4c2b-b5eb-36208073968e None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:00.053741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d493a-d6ad-4c2b-b5eb-36208073968e None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:00.055445 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 50/101] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:36:00.063883 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fc9a0-0c71-4f23-84e9-5e97e1ab11bb None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:00.064174 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fc9a0-0c71-4f23-84e9-5e97e1ab11bb None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:00.064458 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fc9a0-0c71-4f23-84e9-5e97e1ab11bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:00.100594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-724fc9a0-0c71-4f23-84e9-5e97e1ab11bb None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:00.101802 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-724fc9a0-0c71-4f23-84e9-5e97e1ab11bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:00.209790 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c19730-41aa-4dff-9fd6-35a1d8c756d8 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:00.242625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:00.263608 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 51/102] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:00.264738 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75518daf-25a0-4a3c-9ace-e624f08a06a6 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:00.265022 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75518daf-25a0-4a3c-9ace-e624f08a06a6 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:00.265296 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75518daf-25a0-4a3c-9ace-e624f08a06a6 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:00.266616 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 52/103] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:00.275826 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88302bac-0717-470f-b5d0-05b749638888 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:00.276126 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88302bac-0717-470f-b5d0-05b749638888 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:00.276387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88302bac-0717-470f-b5d0-05b749638888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:00.277964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522519b-1f4c-4612-8d63-385f9687c6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:00.278890 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522519b-1f4c-4612-8d63-385f9687c6c4 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:00.279374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522519b-1f4c-4612-8d63-385f9687c6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:00.279859 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522519b-1f4c-4612-8d63-385f9687c6c4 None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:00.280754 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522519b-1f4c-4612-8d63-385f9687c6c4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:00.282540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522519b-1f4c-4612-8d63-385f9687c6c4 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:00.309549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-88302bac-0717-470f-b5d0-05b749638888 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:00.311108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-88302bac-0717-470f-b5d0-05b749638888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:00.313490 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 52/104] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:35:59 2022] POST /identity/v3/projects => generated 302 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:36:00.314773 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522519b-1f4c-4612-8d63-385f9687c6c4 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:00.447788 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522519b-1f4c-4612-8d63-385f9687c6c4 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:00.450547 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:00.491221 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02617d3-3343-45a5-a1c1-3a5795614999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:00.492487 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02617d3-3343-45a5-a1c1-3a5795614999 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:00.493043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02617d3-3343-45a5-a1c1-3a5795614999 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:00.493530 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02617d3-3343-45a5-a1c1-3a5795614999 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:00.507039 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02617d3-3343-45a5-a1c1-3a5795614999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-f02617d3-3343-45a5-a1c1-3a5795614999 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:00.516383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:00.517422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:00.518222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:00.518222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:00.518222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:00.518222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:00.518222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:00.518222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:00.518222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:00.518222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:00.518222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:00.518222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:00.518222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:00.518222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:00.521800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 53/105] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:36:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:00.524295 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11064b84-1f42-4553-b088-f9c688056ade None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:00.524869 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11064b84-1f42-4553-b088-f9c688056ade None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:00.525399 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11064b84-1f42-4553-b088-f9c688056ade None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:00.527501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 54/106] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:00.528489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11064b84-1f42-4553-b088-f9c688056ade None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:00.534658 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 53/107] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:00.541438 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab09a4-6e21-4ceb-b6a1-826d1da6397b None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:00.541438 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab09a4-6e21-4ceb-b6a1-826d1da6397b None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:00.541438 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab09a4-6e21-4ceb-b6a1-826d1da6397b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:00.567268 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-cfab09a4-6e21-4ceb-b6a1-826d1da6397b None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:00.568757 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-cfab09a4-6e21-4ceb-b6a1-826d1da6397b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:00.586643 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:00.665940 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 54/108] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:00.671151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab09a4-6e21-4ceb-b6a1-826d1da6397b None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:00.712591 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11064b84-1f42-4553-b088-f9c688056ade None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:00.757371 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdcf7b-3276-4185-9aec-d468914ca2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:00.759198 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdcf7b-3276-4185-9aec-d468914ca2d4 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:00.759631 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdcf7b-3276-4185-9aec-d468914ca2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:00.760069 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdcf7b-3276-4185-9aec-d468914ca2d4 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:00.760968 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdcf7b-3276-4185-9aec-d468914ca2d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:00.761423 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdcf7b-3276-4185-9aec-d468914ca2d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:00.762979 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdcf7b-3276-4185-9aec-d468914ca2d4 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:00.776332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 55/109] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:00.778540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0219ab2-2914-4cfd-a81e-45136f2e3a6e None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:00.778966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0219ab2-2914-4cfd-a81e-45136f2e3a6e None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:00.779394 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0219ab2-2914-4cfd-a81e-45136f2e3a6e None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:00.782165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 56/110] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:00.783171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0219ab2-2914-4cfd-a81e-45136f2e3a6e None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:00.796772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab09a4-6e21-4ceb-b6a1-826d1da6397b None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:00.808088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:00.840880 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0a727-513b-4b70-b607-43014d68fedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:00.841848 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0a727-513b-4b70-b607-43014d68fedb None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:00.842291 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0a727-513b-4b70-b607-43014d68fedb None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:00.842665 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0a727-513b-4b70-b607-43014d68fedb None admin] PATH_INFO: `/v3/roles/member` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:00.854632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c0a727-513b-4b70-b607-43014d68fedb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-a4c0a727-513b-4b70-b607-43014d68fedb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:00.863225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:00.864468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:00.865200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:00.865200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:00.865200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:00.865200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:00.865200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:00.865200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:00.865200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:00.865200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:00.865200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:00.865200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:00.865200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:00.865200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:00.867629 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 55/111] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:00.868696 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0a727-513b-4b70-b607-43014d68fedb None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:00.898733 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:00.908196 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0219ab2-2914-4cfd-a81e-45136f2e3a6e None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:00.945126 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f458f83-4915-4053-b7f2-8144d68c9ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:00.945915 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f458f83-4915-4053-b7f2-8144d68c9ab0 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:00.946193 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f458f83-4915-4053-b7f2-8144d68c9ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:00.946493 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f458f83-4915-4053-b7f2-8144d68c9ab0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:00.958680 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f458f83-4915-4053-b7f2-8144d68c9ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-9f458f83-4915-4053-b7f2-8144d68c9ab0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:00.968581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:00.969639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:00.970525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:00.970525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:00.970525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:00.970525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:00.970525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:00.970525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:00.970525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:00.970525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:00.970525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:00.970525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:00.970525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:00.970525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:00.970525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 57/112] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:00.971403 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6ad3a-18bf-4781-a0f1-dab5d94495a9 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:00.971695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6ad3a-18bf-4781-a0f1-dab5d94495a9 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:00.974635 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6ad3a-18bf-4781-a0f1-dab5d94495a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:00.983592 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0a727-513b-4b70-b607-43014d68fedb None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:00.993755 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-12d6ad3a-18bf-4781-a0f1-dab5d94495a9 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:00.994959 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-12d6ad3a-18bf-4781-a0f1-dab5d94495a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:01.022642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873bdab-3899-4dc8-847c-5f1eaa4299d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:01.023963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873bdab-3899-4dc8-847c-5f1eaa4299d8 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:01.024252 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873bdab-3899-4dc8-847c-5f1eaa4299d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:01.024532 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873bdab-3899-4dc8-847c-5f1eaa4299d8 None admin] PATH_INFO: `/v3/users/a5800b131869435eb3ae07f86d69929e` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:01.025823 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873bdab-3899-4dc8-847c-5f1eaa4299d8 None admin] RBAC: Authorizing `identity:get_user(user_id=a5800b131869435eb3ae07f86d69929e)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:01.027361 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873bdab-3899-4dc8-847c-5f1eaa4299d8 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:01.029245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 56/113] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:00 2022] GET /identity/v3/users/a5800b131869435eb3ae07f86d69929e => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:01.029927 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873bdab-3899-4dc8-847c-5f1eaa4299d8 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:01.072468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:01.081216 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 58/114] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:01.081216 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a0f95-9ed9-45a7-9320-f90a131a8f65 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:01.081216 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a0f95-9ed9-45a7-9320-f90a131a8f65 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:01.081216 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a0f95-9ed9-45a7-9320-f90a131a8f65 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:01.081216 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 59/115] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:01.081623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a0f95-9ed9-45a7-9320-f90a131a8f65 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:01.152618 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873bdab-3899-4dc8-847c-5f1eaa4299d8 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:01.164876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:01.187894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71107d34-aae6-408e-8147-1d1008424e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:01.188387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71107d34-aae6-408e-8147-1d1008424e4f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:01.188674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71107d34-aae6-408e-8147-1d1008424e4f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:01.188957 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71107d34-aae6-408e-8147-1d1008424e4f None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:01.189716 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71107d34-aae6-408e-8147-1d1008424e4f None admin] RBAC: Adding query filter params (name=member) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:01.190028 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71107d34-aae6-408e-8147-1d1008424e4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:01.191477 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71107d34-aae6-408e-8147-1d1008424e4f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:01.192027 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a0f95-9ed9-45a7-9320-f90a131a8f65 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:01.200265 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 57/116] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:01.200899 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71107d34-aae6-408e-8147-1d1008424e4f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:01.232360 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29395e-4676-4159-a8e6-90c925c38c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:01.233033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29395e-4676-4159-a8e6-90c925c38c71 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:01.233314 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29395e-4676-4159-a8e6-90c925c38c71 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:01.233592 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29395e-4676-4159-a8e6-90c925c38c71 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:01.234406 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29395e-4676-4159-a8e6-90c925c38c71 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:01.234715 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29395e-4676-4159-a8e6-90c925c38c71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:01.236131 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29395e-4676-4159-a8e6-90c925c38c71 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:01.244749 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 60/117] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:01.245943 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af64eb-b669-4731-8a49-259a1916f8fa None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:01.246226 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af64eb-b669-4731-8a49-259a1916f8fa None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:01.246536 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af64eb-b669-4731-8a49-259a1916f8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:01.266125 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-20af64eb-b669-4731-8a49-259a1916f8fa None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:01.267358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-20af64eb-b669-4731-8a49-259a1916f8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:01.305968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:01.311764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71107d34-aae6-408e-8147-1d1008424e4f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:01.339063 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 61/118] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:01.346493 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af64eb-b669-4731-8a49-259a1916f8fa None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:01.355072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9f5ba-c0d4-4496-b36e-f6a236dbe262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:01.355986 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9f5ba-c0d4-4496-b36e-f6a236dbe262 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:01.356274 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9f5ba-c0d4-4496-b36e-f6a236dbe262 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:01.356548 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9f5ba-c0d4-4496-b36e-f6a236dbe262 None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:01.357775 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9f5ba-c0d4-4496-b36e-f6a236dbe262 None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:01.359412 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9f5ba-c0d4-4496-b36e-f6a236dbe262 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:01.361163 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 58/119] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:01 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:01.361961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9f5ba-c0d4-4496-b36e-f6a236dbe262 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:01.421889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:01.456062 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af64eb-b669-4731-8a49-259a1916f8fa None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:01.475631 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9f5ba-c0d4-4496-b36e-f6a236dbe262 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:01.490509 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7bcaf-4092-4e82-a5e3-6db50873ed1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:01.491339 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7bcaf-4092-4e82-a5e3-6db50873ed1f None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:01.491631 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7bcaf-4092-4e82-a5e3-6db50873ed1f None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:01.491899 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7bcaf-4092-4e82-a5e3-6db50873ed1f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:01.501695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7bcaf-4092-4e82-a5e3-6db50873ed1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-dec7bcaf-4092-4e82-a5e3-6db50873ed1f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:01.509553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:01.510605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:01.511369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:01.511369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:01.511369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:01.511369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:01.511369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:01.511369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:01.511369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:01.511369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:01.511369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:01.511369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:01.511369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:01.511369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:01.511369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 62/120] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:01.511973 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7bcaf-4092-4e82-a5e3-6db50873ed1f None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:01.516255 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e7e81-9182-4e91-90b4-d85d7537d239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:01.517325 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e7e81-9182-4e91-90b4-d85d7537d239 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:01.517775 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e7e81-9182-4e91-90b4-d85d7537d239 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:01.518054 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e7e81-9182-4e91-90b4-d85d7537d239 None admin] PATH_INFO: `/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:01.519415 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e7e81-9182-4e91-90b4-d85d7537d239 None admin] RBAC: Authorizing `identity:get_user(user_id=6c5e8aab0eaa42288cf4b090c1e64ae7)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:01.521014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e7e81-9182-4e91-90b4-d85d7537d239 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:01.526908 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 59/121] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:01 2022] GET /identity/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:01.526908 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e7e81-9182-4e91-90b4-d85d7537d239 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:01.557539 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:01.626776 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7bcaf-4092-4e82-a5e3-6db50873ed1f None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:01.640543 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e7e81-9182-4e91-90b4-d85d7537d239 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:01.659911 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:01.671257 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0daa9-de64-4f55-b4bf-afb436eea754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:01.672733 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0daa9-de64-4f55-b4bf-afb436eea754 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:01.673243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0daa9-de64-4f55-b4bf-afb436eea754 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:01.673719 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0daa9-de64-4f55-b4bf-afb436eea754 None admin] PATH_INFO: `/v3/users/8340d97d04ba441bbf713dcd96777a4e` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:01.684958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4ee42-ec6d-4845-8814-0a6f82d36517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:01.685997 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4ee42-ec6d-4845-8814-0a6f82d36517 None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:01.686374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4ee42-ec6d-4845-8814-0a6f82d36517 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:01.686713 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4ee42-ec6d-4845-8814-0a6f82d36517 None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f/users/a5800b131869435eb3ae07f86d69929e/roles/a33caf2301fb44db90304eb3d9afc3a5` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:01.688817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a4ee42-ec6d-4845-8814-0a6f82d36517 None admin] RBAC: Authorizing `identity:create_grant(project_id=644a8176f5874355bc660ed98897307f, user_id=a5800b131869435eb3ae07f86d69929e, role_id=a33caf2301fb44db90304eb3d9afc3a5)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:01.689313 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0daa9-de64-4f55-b4bf-afb436eea754 None admin] RBAC: Authorizing `identity:get_user(user_id=8340d97d04ba441bbf713dcd96777a4e)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:01.690437 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a4ee42-ec6d-4845-8814-0a6f82d36517 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:01.690832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0daa9-de64-4f55-b4bf-afb436eea754 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:01.692917 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 63/122] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:01 2022] GET /identity/v3/users/8340d97d04ba441bbf713dcd96777a4e => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:01.692917 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0daa9-de64-4f55-b4bf-afb436eea754 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:01.708205 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 60/123] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:36:01 2022] PUT /identity/v3/projects/644a8176f5874355bc660ed98897307f/users/a5800b131869435eb3ae07f86d69929e/roles/a33caf2301fb44db90304eb3d9afc3a5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:36:01.709045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4ee42-ec6d-4845-8814-0a6f82d36517 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:01.825183 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0daa9-de64-4f55-b4bf-afb436eea754 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:01.854031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:01.868045 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f8a58-ab1e-4cfd-a687-038002138d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:01.868398 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4ee42-ec6d-4845-8814-0a6f82d36517 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:01.869387 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f8a58-ab1e-4cfd-a687-038002138d28 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:01.869889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f8a58-ab1e-4cfd-a687-038002138d28 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:01.870321 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f8a58-ab1e-4cfd-a687-038002138d28 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:01.883682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f8a58-ab1e-4cfd-a687-038002138d28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-241f8a58-ab1e-4cfd-a687-038002138d28 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:01.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:01.893848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:01.894692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:01.894692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:01.894692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:01.894692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:01.894692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:01.894692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:01.894692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:01.894692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:01.894692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:01.894692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:01.894692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:01.894692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:01.897177 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 64/124] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:36:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:01.898223 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f8a58-ab1e-4cfd-a687-038002138d28 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:01.907868 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a7c44-3486-4639-a885-6e7d7cf62174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:01.908551 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a7c44-3486-4639-a885-6e7d7cf62174 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:01.908835 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a7c44-3486-4639-a885-6e7d7cf62174 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:01.909112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a7c44-3486-4639-a885-6e7d7cf62174 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:01.909874 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a7c44-3486-4639-a885-6e7d7cf62174 None admin] RBAC: Adding query filter params (name=member) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:01.910184 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a7c44-3486-4639-a885-6e7d7cf62174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:01.912438 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a7c44-3486-4639-a885-6e7d7cf62174 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:01.922552 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 61/125] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:01.923384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a7c44-3486-4639-a885-6e7d7cf62174 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:01.963306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:02.043209 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f8a58-ab1e-4cfd-a687-038002138d28 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:02.044055 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a7c44-3486-4639-a885-6e7d7cf62174 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:02.079433 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af98ddd-16c3-4ae0-9abf-c56ee2901087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:02.080449 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af98ddd-16c3-4ae0-9abf-c56ee2901087 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:02.081041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af98ddd-16c3-4ae0-9abf-c56ee2901087 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:02.081501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af98ddd-16c3-4ae0-9abf-c56ee2901087 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:02.087852 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cfcf9-56f1-4d1e-bd02-c845da7c868c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:02.088743 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cfcf9-56f1-4d1e-bd02-c845da7c868c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:02.089024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cfcf9-56f1-4d1e-bd02-c845da7c868c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:02.089296 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cfcf9-56f1-4d1e-bd02-c845da7c868c None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:02.090536 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cfcf9-56f1-4d1e-bd02-c845da7c868c None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:02.092581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cfcf9-56f1-4d1e-bd02-c845da7c868c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:02.094805 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 65/126] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:01 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:02.095618 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cfcf9-56f1-4d1e-bd02-c845da7c868c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:02.110771 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:02.161893 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af98ddd-16c3-4ae0-9abf-c56ee2901087 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:02.163423 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af98ddd-16c3-4ae0-9abf-c56ee2901087 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:02.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cfcf9-56f1-4d1e-bd02-c845da7c868c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:02.236325 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:02.260655 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f24a4-e04d-4941-9921-0163f080fed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:02.261322 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f24a4-e04d-4941-9921-0163f080fed8 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:02.261599 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f24a4-e04d-4941-9921-0163f080fed8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:02.261868 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f24a4-e04d-4941-9921-0163f080fed8 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:02.262655 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f24a4-e04d-4941-9921-0163f080fed8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:02.262962 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f24a4-e04d-4941-9921-0163f080fed8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:02.264380 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f24a4-e04d-4941-9921-0163f080fed8 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:02.278231 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 66/127] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:02.278231 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f24a4-e04d-4941-9921-0163f080fed8 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:02.278992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 62/128] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:02.280155 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af98ddd-16c3-4ae0-9abf-c56ee2901087 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:02.355740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:02.393296 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f24a4-e04d-4941-9921-0163f080fed8 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:02.407068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af98ddd-16c3-4ae0-9abf-c56ee2901087 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:02.439610 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3d624-7db3-4111-bec2-83af8ad6fa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:02.441182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3d624-7db3-4111-bec2-83af8ad6fa40 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:02.441787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3d624-7db3-4111-bec2-83af8ad6fa40 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:02.442359 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3d624-7db3-4111-bec2-83af8ad6fa40 None admin] PATH_INFO: `/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:02.444058 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3d624-7db3-4111-bec2-83af8ad6fa40 None admin] RBAC: Authorizing `identity:get_user(user_id=6c5e8aab0eaa42288cf4b090c1e64ae7)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:02.444176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f941e1-a03c-439a-93c3-d2e621766c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:02.445075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f941e1-a03c-439a-93c3-d2e621766c99 None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:02.445359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f941e1-a03c-439a-93c3-d2e621766c99 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:02.445628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f941e1-a03c-439a-93c3-d2e621766c99 None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f/users/6c5e8aab0eaa42288cf4b090c1e64ae7/roles/fe332203b7e04d6084e26c5be292af35` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:02.446629 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3d624-7db3-4111-bec2-83af8ad6fa40 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:02.447640 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f941e1-a03c-439a-93c3-d2e621766c99 None admin] RBAC: Authorizing `identity:create_grant(project_id=644a8176f5874355bc660ed98897307f, user_id=6c5e8aab0eaa42288cf4b090c1e64ae7, role_id=fe332203b7e04d6084e26c5be292af35)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:02.448879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 67/129] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:02 2022] GET /identity/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:02.453554 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f941e1-a03c-439a-93c3-d2e621766c99 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:02.455010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3d624-7db3-4111-bec2-83af8ad6fa40 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:02.470384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 63/130] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:36:02 2022] PUT /identity/v3/projects/644a8176f5874355bc660ed98897307f/users/6c5e8aab0eaa42288cf4b090c1e64ae7/roles/fe332203b7e04d6084e26c5be292af35 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:36:02.472188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f941e1-a03c-439a-93c3-d2e621766c99 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:02.511866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:02.599330 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3d624-7db3-4111-bec2-83af8ad6fa40 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:02.621144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f941e1-a03c-439a-93c3-d2e621766c99 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:02.640262 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:02.662316 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a436b-03be-470b-bd6f-3241ba77a76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:02.662316 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a436b-03be-470b-bd6f-3241ba77a76b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:02.662316 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a436b-03be-470b-bd6f-3241ba77a76b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:02.662316 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a436b-03be-470b-bd6f-3241ba77a76b None admin] PATH_INFO: `/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:02.662748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a436b-03be-470b-bd6f-3241ba77a76b None admin] RBAC: Authorizing `identity:get_user(user_id=6c5e8aab0eaa42288cf4b090c1e64ae7)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:02.662748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a436b-03be-470b-bd6f-3241ba77a76b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:02.664791 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 68/131] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:02 2022] GET /identity/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:02.665631 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a436b-03be-470b-bd6f-3241ba77a76b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:02.691172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81510e52-6012-41f8-92d4-68b88f8579a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:02.692497 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81510e52-6012-41f8-92d4-68b88f8579a2 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:02.692916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81510e52-6012-41f8-92d4-68b88f8579a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:02.693323 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81510e52-6012-41f8-92d4-68b88f8579a2 None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:02.698056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81510e52-6012-41f8-92d4-68b88f8579a2 None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:02.699833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81510e52-6012-41f8-92d4-68b88f8579a2 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:02.706291 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 64/132] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:02 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:02.707566 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81510e52-6012-41f8-92d4-68b88f8579a2 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:02.794855 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a436b-03be-470b-bd6f-3241ba77a76b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:02.809415 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:02.832709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754ca85-76f2-4f1b-9c50-82136d3d5092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:02.833854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754ca85-76f2-4f1b-9c50-82136d3d5092 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:02.834140 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754ca85-76f2-4f1b-9c50-82136d3d5092 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:02.834494 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754ca85-76f2-4f1b-9c50-82136d3d5092 None admin] PATH_INFO: `/v3/projects/07ad62c92dd548cb8b7de09cf8881d4b` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:02.835774 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754ca85-76f2-4f1b-9c50-82136d3d5092 None admin] RBAC: Authorizing `identity:get_project(project_id=07ad62c92dd548cb8b7de09cf8881d4b)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:02.837335 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754ca85-76f2-4f1b-9c50-82136d3d5092 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:02.839081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 69/133] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:02 2022] GET /identity/v3/projects/07ad62c92dd548cb8b7de09cf8881d4b => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:02.839852 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754ca85-76f2-4f1b-9c50-82136d3d5092 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:02.872333 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81510e52-6012-41f8-92d4-68b88f8579a2 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:02.907309 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea3302-b148-49cc-b7bb-abf64c089e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:02.908120 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea3302-b148-49cc-b7bb-abf64c089e40 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:02.908407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea3302-b148-49cc-b7bb-abf64c089e40 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:02.908682 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea3302-b148-49cc-b7bb-abf64c089e40 None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:02.909875 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea3302-b148-49cc-b7bb-abf64c089e40 None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:02.911680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea3302-b148-49cc-b7bb-abf64c089e40 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:02.913581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 65/134] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:02 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:02.914619 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73038677-6d42-489c-81ff-5af2b90cd193 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:02.914907 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73038677-6d42-489c-81ff-5af2b90cd193 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:02.915183 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73038677-6d42-489c-81ff-5af2b90cd193 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:02.916422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 66/135] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:02.917232 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73038677-6d42-489c-81ff-5af2b90cd193 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:02.958499 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754ca85-76f2-4f1b-9c50-82136d3d5092 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:02.976903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:02.998405 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79082e-d113-4177-95df-a824101fc1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:02.999540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79082e-d113-4177-95df-a824101fc1a1 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:03.000022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79082e-d113-4177-95df-a824101fc1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:03.000553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79082e-d113-4177-95df-a824101fc1a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:03.001679 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79082e-d113-4177-95df-a824101fc1a1 None admin] RBAC: Adding query filter params (role.id=6ab02cf5cb1b4d3595ea8ea883ff6f2d, scope.project.id=644a8176f5874355bc660ed98897307f, user.id=8340d97d04ba441bbf713dcd96777a4e) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:03.002180 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79082e-d113-4177-95df-a824101fc1a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:03.003904 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79082e-d113-4177-95df-a824101fc1a1 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:03.010568 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 70/136] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:02 2022] GET /identity/v3/role_assignments?user.id=8340d97d04ba441bbf713dcd96777a4e&scope.project.id=644a8176f5874355bc660ed98897307f&role.id=6ab02cf5cb1b4d3595ea8ea883ff6f2d => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:03.012395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2b9d3-8740-438f-8d7d-4ce31d83e7b0 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:03.012873 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2b9d3-8740-438f-8d7d-4ce31d83e7b0 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:03.013365 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2b9d3-8740-438f-8d7d-4ce31d83e7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:03.032722 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-57a2b9d3-8740-438f-8d7d-4ce31d83e7b0 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:03.034361 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-57a2b9d3-8740-438f-8d7d-4ce31d83e7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:03.037339 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73038677-6d42-489c-81ff-5af2b90cd193 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:03.076979 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1a47f-9527-41cc-b5ee-fde19cf43755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:03.077387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1a47f-9527-41cc-b5ee-fde19cf43755 None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:03.077651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1a47f-9527-41cc-b5ee-fde19cf43755 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:03.077936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1a47f-9527-41cc-b5ee-fde19cf43755 None admin] PATH_INFO: `/v3/projects/07ad62c92dd548cb8b7de09cf8881d4b/users/6c5e8aab0eaa42288cf4b090c1e64ae7/roles/fe332203b7e04d6084e26c5be292af35` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:03.080074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1a47f-9527-41cc-b5ee-fde19cf43755 None admin] RBAC: Authorizing `identity:create_grant(project_id=07ad62c92dd548cb8b7de09cf8881d4b, user_id=6c5e8aab0eaa42288cf4b090c1e64ae7, role_id=fe332203b7e04d6084e26c5be292af35)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:03.081564 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1a47f-9527-41cc-b5ee-fde19cf43755 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:03.084510 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:03.102352 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 67/137] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:36:02 2022] PUT /identity/v3/projects/07ad62c92dd548cb8b7de09cf8881d4b/users/6c5e8aab0eaa42288cf4b090c1e64ae7/roles/fe332203b7e04d6084e26c5be292af35 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:36:03.103801 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1a47f-9527-41cc-b5ee-fde19cf43755 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:03.131147 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 71/138] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:03.167743 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2b9d3-8740-438f-8d7d-4ce31d83e7b0 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:03.236584 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1a47f-9527-41cc-b5ee-fde19cf43755 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:03.252625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:03.283111 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bea996-c9fe-483e-8bc4-a9ee42f90c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:03.284765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bea996-c9fe-483e-8bc4-a9ee42f90c6d None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:03.285197 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bea996-c9fe-483e-8bc4-a9ee42f90c6d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:03.285779 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bea996-c9fe-483e-8bc4-a9ee42f90c6d None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f/users/6c5e8aab0eaa42288cf4b090c1e64ae7/roles/4a241925b6cc4218b4f2af580b151877` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:03.289007 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bea996-c9fe-483e-8bc4-a9ee42f90c6d None admin] RBAC: Authorizing `identity:create_grant(project_id=644a8176f5874355bc660ed98897307f, user_id=6c5e8aab0eaa42288cf4b090c1e64ae7, role_id=4a241925b6cc4218b4f2af580b151877)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:03.290901 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bea996-c9fe-483e-8bc4-a9ee42f90c6d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:03.311093 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 68/139] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:36:03 2022] PUT /identity/v3/projects/644a8176f5874355bc660ed98897307f/users/6c5e8aab0eaa42288cf4b090c1e64ae7/roles/4a241925b6cc4218b4f2af580b151877 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:36:03.381910 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2b9d3-8740-438f-8d7d-4ce31d83e7b0 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:03.436243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f241961-d29a-462f-bebe-5fd065f09377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:03.437733 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f241961-d29a-462f-bebe-5fd065f09377 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:03.438288 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f241961-d29a-462f-bebe-5fd065f09377 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:03.438761 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f241961-d29a-462f-bebe-5fd065f09377 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:03.440257 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f241961-d29a-462f-bebe-5fd065f09377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:03.442077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f241961-d29a-462f-bebe-5fd065f09377 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:03.444017 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 72/140] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:03.449800 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bea996-c9fe-483e-8bc4-a9ee42f90c6d None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:03.563737 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bea996-c9fe-483e-8bc4-a9ee42f90c6d None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:03.601468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158977a-d69c-4f02-bbd2-ba2ce9d6596f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:03.602994 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158977a-d69c-4f02-bbd2-ba2ce9d6596f None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:03.603505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158977a-d69c-4f02-bbd2-ba2ce9d6596f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:03.603991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158977a-d69c-4f02-bbd2-ba2ce9d6596f None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:03.604920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158977a-d69c-4f02-bbd2-ba2ce9d6596f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:03.606654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158977a-d69c-4f02-bbd2-ba2ce9d6596f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:03.635494 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 69/141] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:36:03 2022] POST /identity/v3/users => generated 283 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:36:04.502102 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad10ef6-6ca4-4fda-aa91-b6f5a6dcc05a None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:04.502627 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad10ef6-6ca4-4fda-aa91-b6f5a6dcc05a None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:04.502725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad10ef6-6ca4-4fda-aa91-b6f5a6dcc05a None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:04.504297 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 73/142] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:04.512367 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b4791-b26a-4c88-bbea-1e3a92d41f04 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:04.513267 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b4791-b26a-4c88-bbea-1e3a92d41f04 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:04.513741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b4791-b26a-4c88-bbea-1e3a92d41f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:04.535434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-820b4791-b26a-4c88-bbea-1e3a92d41f04 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:04.536971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-820b4791-b26a-4c88-bbea-1e3a92d41f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:04.561254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:04.620400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 70/143] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:04.661207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad10ef6-6ca4-4fda-aa91-b6f5a6dcc05a None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:04.712200 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:04.782076 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad10ef6-6ca4-4fda-aa91-b6f5a6dcc05a None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:04.821032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2843d2-2800-469a-a94f-f47efd1dd9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:04.821844 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2843d2-2800-469a-a94f-f47efd1dd9f9 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:04.822123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2843d2-2800-469a-a94f-f47efd1dd9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:04.826462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2843d2-2800-469a-a94f-f47efd1dd9f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:04.837757 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2843d2-2800-469a-a94f-f47efd1dd9f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-ef2843d2-2800-469a-a94f-f47efd1dd9f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:04.845656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:04.846535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:04.847725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:04.847725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:04.847725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:04.847725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:04.847725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:04.847725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:04.847725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:04.847725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:04.847725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:04.847725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:04.847725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:04.847725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:04.847725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 74/144] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:36:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:36:04.855318 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b4791-b26a-4c88-bbea-1e3a92d41f04 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:04.998634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b4791-b26a-4c88-bbea-1e3a92d41f04 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:05.044794 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad717ea1-10fa-4769-a9de-521473f39d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:05.045854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad717ea1-10fa-4769-a9de-521473f39d98 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:05.046378 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad717ea1-10fa-4769-a9de-521473f39d98 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:05.046846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad717ea1-10fa-4769-a9de-521473f39d98 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:05.047842 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad717ea1-10fa-4769-a9de-521473f39d98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:05.048383 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad717ea1-10fa-4769-a9de-521473f39d98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:05.050044 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad717ea1-10fa-4769-a9de-521473f39d98 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:05.062993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 71/145] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:05.069178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2843d2-2800-469a-a94f-f47efd1dd9f9 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:05.199156 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2843d2-2800-469a-a94f-f47efd1dd9f9 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:05.200739 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:05.244445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de04e2-8186-413e-ba51-7682becbb831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:05.245354 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de04e2-8186-413e-ba51-7682becbb831 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:05.245686 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de04e2-8186-413e-ba51-7682becbb831 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:05.245910 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de04e2-8186-413e-ba51-7682becbb831 None admin] PATH_INFO: `/v3/users/a5800b131869435eb3ae07f86d69929e` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:05.247151 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de04e2-8186-413e-ba51-7682becbb831 None admin] RBAC: Authorizing `identity:get_user(user_id=a5800b131869435eb3ae07f86d69929e)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:05.248652 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de04e2-8186-413e-ba51-7682becbb831 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:05.250328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 75/146] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:05 2022] GET /identity/v3/users/a5800b131869435eb3ae07f86d69929e => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:05.256462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad717ea1-10fa-4769-a9de-521473f39d98 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:05.292442 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:05.410139 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad717ea1-10fa-4769-a9de-521473f39d98 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:05.448595 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd4b4a-97ec-408e-90fe-7097d718f42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:05.449260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd4b4a-97ec-408e-90fe-7097d718f42d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:05.449538 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd4b4a-97ec-408e-90fe-7097d718f42d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:05.449813 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd4b4a-97ec-408e-90fe-7097d718f42d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:05.535857 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd4b4a-97ec-408e-90fe-7097d718f42d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:05.537331 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd4b4a-97ec-408e-90fe-7097d718f42d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:05.565383 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:05.682449 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 72/147] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:05.688043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de04e2-8186-413e-ba51-7682becbb831 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:05.818945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de04e2-8186-413e-ba51-7682becbb831 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:05.861959 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3576d915-cc96-4a17-a0cf-9cbd07db22a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:05.862860 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576d915-cc96-4a17-a0cf-9cbd07db22a6 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:05.863139 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576d915-cc96-4a17-a0cf-9cbd07db22a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:05.863430 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576d915-cc96-4a17-a0cf-9cbd07db22a6 None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:05.864615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576d915-cc96-4a17-a0cf-9cbd07db22a6 None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:05.866076 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576d915-cc96-4a17-a0cf-9cbd07db22a6 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:05.868117 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 76/148] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:05 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:05.873725 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd4b4a-97ec-408e-90fe-7097d718f42d None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:06.015164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd4b4a-97ec-408e-90fe-7097d718f42d None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:06.016224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a3dbe-d1bd-4ac6-b72c-0fa7f93549b6 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:06.016509 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a3dbe-d1bd-4ac6-b72c-0fa7f93549b6 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:06.016785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a3dbe-d1bd-4ac6-b72c-0fa7f93549b6 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:06.018170 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 77/149] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:36:06.024033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f9b71-dce0-403c-8c25-a024cae47ea3 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:06.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f9b71-dce0-403c-8c25-a024cae47ea3 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:06.024591 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f9b71-dce0-403c-8c25-a024cae47ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:06.047177 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:06.049434 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-e38f9b71-dce0-403c-8c25-a024cae47ea3 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:06.051135 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-e38f9b71-dce0-403c-8c25-a024cae47ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:06.115899 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500936a-b410-4ddf-a1bc-87ca7d766be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:06.116594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500936a-b410-4ddf-a1bc-87ca7d766be2 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:06.116882 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500936a-b410-4ddf-a1bc-87ca7d766be2 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:06.117161 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500936a-b410-4ddf-a1bc-87ca7d766be2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:06.117705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:06.118032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500936a-b410-4ddf-a1bc-87ca7d766be2 None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=644a8176f5874355bc660ed98897307f, user.id=a5800b131869435eb3ae07f86d69929e) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:06.118372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500936a-b410-4ddf-a1bc-87ca7d766be2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:06.120015 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500936a-b410-4ddf-a1bc-87ca7d766be2 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:06.136647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 73/150] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:05 2022] GET /identity/v3/role_assignments?user.id=a5800b131869435eb3ae07f86d69929e&scope.project.id=644a8176f5874355bc660ed98897307f&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:06.155392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 78/151] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:06.211854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500936a-b410-4ddf-a1bc-87ca7d766be2 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:06.361392 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500936a-b410-4ddf-a1bc-87ca7d766be2 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:06.406417 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828e0d3-ac49-44c3-b34a-a18e5b1cd3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:06.407274 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828e0d3-ac49-44c3-b34a-a18e5b1cd3f5 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:06.407531 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828e0d3-ac49-44c3-b34a-a18e5b1cd3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:06.407803 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828e0d3-ac49-44c3-b34a-a18e5b1cd3f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:06.424091 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4828e0d3-ac49-44c3-b34a-a18e5b1cd3f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-4828e0d3-ac49-44c3-b34a-a18e5b1cd3f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:06.432973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:06.433865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:06.434621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:06.434621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:06.434621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:06.434621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:06.434621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:06.434621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:06.434621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:06.434621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:06.434621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:06.434621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:06.434621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:06.434621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:06.434621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 74/152] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:06.440700 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f9b71-dce0-403c-8c25-a024cae47ea3 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:06.614839 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f9b71-dce0-403c-8c25-a024cae47ea3 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:06.667940 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0abed2-c119-41ae-b979-adb669a41296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:06.668635 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0abed2-c119-41ae-b979-adb669a41296 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:06.668910 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0abed2-c119-41ae-b979-adb669a41296 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:06.669178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0abed2-c119-41ae-b979-adb669a41296 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:06.669940 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0abed2-c119-41ae-b979-adb669a41296 None admin] RBAC: Adding query filter params (name=member) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:06.670244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0abed2-c119-41ae-b979-adb669a41296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:06.671685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0abed2-c119-41ae-b979-adb669a41296 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:06.686323 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 79/153] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:06.692244 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828e0d3-ac49-44c3-b34a-a18e5b1cd3f5 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:06.754360 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:06.824090 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828e0d3-ac49-44c3-b34a-a18e5b1cd3f5 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:06.863528 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0534d5-649b-4b9a-a43a-b94d93a47781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:06.864428 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0534d5-649b-4b9a-a43a-b94d93a47781 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:06.864693 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0534d5-649b-4b9a-a43a-b94d93a47781 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:06.865004 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0534d5-649b-4b9a-a43a-b94d93a47781 None admin] PATH_INFO: `/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:06.866170 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0534d5-649b-4b9a-a43a-b94d93a47781 None admin] RBAC: Authorizing `identity:get_user(user_id=6c5e8aab0eaa42288cf4b090c1e64ae7)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:06.867716 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0534d5-649b-4b9a-a43a-b94d93a47781 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:06.869352 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 75/154] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:06 2022] GET /identity/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:06.875404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0abed2-c119-41ae-b979-adb669a41296 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:06.969034 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:06.992515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b0656-d9f4-4e50-98d8-35477128c8c0 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:06.992847 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b0656-d9f4-4e50-98d8-35477128c8c0 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:06.993067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b0656-d9f4-4e50-98d8-35477128c8c0 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:06.994705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 76/155] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:07.009723 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe8a4d-d5fd-44dc-a161-0c3b2c0dfd4a None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:07.009996 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe8a4d-d5fd-44dc-a161-0c3b2c0dfd4a None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:07.010458 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe8a4d-d5fd-44dc-a161-0c3b2c0dfd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:07.043919 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-0afe8a4d-d5fd-44dc-a161-0c3b2c0dfd4a None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:07.045122 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-0afe8a4d-d5fd-44dc-a161-0c3b2c0dfd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:07.136953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0abed2-c119-41ae-b979-adb669a41296 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:07.220740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 77/156] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:07.236697 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cb261-5bd0-4686-8da0-70a9445e0a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:07.237384 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cb261-5bd0-4686-8da0-70a9445e0a22 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:07.237659 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cb261-5bd0-4686-8da0-70a9445e0a22 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:07.237926 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cb261-5bd0-4686-8da0-70a9445e0a22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:07.261128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe8a4d-d5fd-44dc-a161-0c3b2c0dfd4a None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:07.376649 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:07.384563 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cb261-5bd0-4686-8da0-70a9445e0a22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:07.386382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cb261-5bd0-4686-8da0-70a9445e0a22 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:07.461797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:07.552335 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe8a4d-d5fd-44dc-a161-0c3b2c0dfd4a None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:07.618439 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b75435-e939-44eb-af38-407ae4b0696f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:07.619845 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b75435-e939-44eb-af38-407ae4b0696f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:07.619845 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b75435-e939-44eb-af38-407ae4b0696f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:07.619845 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b75435-e939-44eb-af38-407ae4b0696f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:07.632170 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 80/157] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 757 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:07.633615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f43a7-05a4-4743-bd8d-e592863022d9 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:07.633902 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f43a7-05a4-4743-bd8d-e592863022d9 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:07.635759 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f43a7-05a4-4743-bd8d-e592863022d9 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:07.637146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 81/158] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:07.637905 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f43a7-05a4-4743-bd8d-e592863022d9 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:07.646798 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b75435-e939-44eb-af38-407ae4b0696f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-75b75435-e939-44eb-af38-407ae4b0696f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:07.654921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:07.655764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:07.656503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:07.656503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:07.656503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:07.656503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:07.656503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:07.656503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:07.656503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:07.656503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:07.656503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:07.656503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:07.656503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:07.656503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:07.663994 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 78/159] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:36:07.665380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a25d13-74a7-42d6-91a2-e81ab9b244bb None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:07.665668 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a25d13-74a7-42d6-91a2-e81ab9b244bb None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:07.665938 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a25d13-74a7-42d6-91a2-e81ab9b244bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:07.694770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-f9a25d13-74a7-42d6-91a2-e81ab9b244bb None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:07.696623 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-f9a25d13-74a7-42d6-91a2-e81ab9b244bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:07.811158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f43a7-05a4-4743-bd8d-e592863022d9 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:07.829964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 79/160] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:07.830859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a25d13-74a7-42d6-91a2-e81ab9b244bb None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:07.857422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:07.864545 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1eb090-fc91-425b-835f-f03f212b216a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:07.871252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1eb090-fc91-425b-835f-f03f212b216a None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:07.871753 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1eb090-fc91-425b-835f-f03f212b216a None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:07.872229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1eb090-fc91-425b-835f-f03f212b216a None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:07.873745 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1eb090-fc91-425b-835f-f03f212b216a None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:07.875488 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1eb090-fc91-425b-835f-f03f212b216a None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:07.877962 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 82/161] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:07 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:07.879512 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d76c7e-5c8f-4ae8-b009-5709598e9ca9 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:07.880045 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d76c7e-5c8f-4ae8-b009-5709598e9ca9 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:07.880554 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d76c7e-5c8f-4ae8-b009-5709598e9ca9 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:07.881994 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 83/162] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:07.884839 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d76c7e-5c8f-4ae8-b009-5709598e9ca9 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:07.898645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:08.012152 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a25d13-74a7-42d6-91a2-e81ab9b244bb None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:08.016207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d76c7e-5c8f-4ae8-b009-5709598e9ca9 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:08.049639 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472a333-f268-48f1-bf0a-dba21a2b47d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:08.050340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472a333-f268-48f1-bf0a-dba21a2b47d4 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:08.050621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472a333-f268-48f1-bf0a-dba21a2b47d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:08.050920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472a333-f268-48f1-bf0a-dba21a2b47d4 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:08.051679 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5472a333-f268-48f1-bf0a-dba21a2b47d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:08.051980 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5472a333-f268-48f1-bf0a-dba21a2b47d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:08.053362 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5472a333-f268-48f1-bf0a-dba21a2b47d4 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:08.072077 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 80/163] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:08.074235 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472a333-f268-48f1-bf0a-dba21a2b47d4 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:08.081645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d9866-6243-4404-9bbd-f9885aef619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:08.083754 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d9866-6243-4404-9bbd-f9885aef619e None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:08.085284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d9866-6243-4404-9bbd-f9885aef619e None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:08.085578 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d9866-6243-4404-9bbd-f9885aef619e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:08.086458 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d9866-6243-4404-9bbd-f9885aef619e None admin] RBAC: Adding query filter params (role.id=fe332203b7e04d6084e26c5be292af35, scope.project.id=644a8176f5874355bc660ed98897307f, user.id=6c5e8aab0eaa42288cf4b090c1e64ae7) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:08.086821 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d9866-6243-4404-9bbd-f9885aef619e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:08.088399 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d9866-6243-4404-9bbd-f9885aef619e None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:08.116695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 84/164] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:07 2022] GET /identity/v3/role_assignments?user.id=6c5e8aab0eaa42288cf4b090c1e64ae7&scope.project.id=644a8176f5874355bc660ed98897307f&role.id=fe332203b7e04d6084e26c5be292af35 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:08.118022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c62b95-22c5-4da9-bad3-86f4c2b3103f None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:08.118335 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c62b95-22c5-4da9-bad3-86f4c2b3103f None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:08.118605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c62b95-22c5-4da9-bad3-86f4c2b3103f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:08.164044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-45c62b95-22c5-4da9-bad3-86f4c2b3103f None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:08.165254 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-45c62b95-22c5-4da9-bad3-86f4c2b3103f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:08.243716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:08.247813 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:08.256671 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 85/165] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:08.257664 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c62b95-22c5-4da9-bad3-86f4c2b3103f None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:08.257768 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472a333-f268-48f1-bf0a-dba21a2b47d4 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:08.320036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c661e1-913d-4987-a431-ce27a5bb2efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:08.320874 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c661e1-913d-4987-a431-ce27a5bb2efc None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:08.321154 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c661e1-913d-4987-a431-ce27a5bb2efc None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:08.321426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c661e1-913d-4987-a431-ce27a5bb2efc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:08.338196 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c661e1-913d-4987-a431-ce27a5bb2efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-56c661e1-913d-4987-a431-ce27a5bb2efc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:08.346921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:08.347977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:08.348811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:08.348811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:08.348811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:08.348811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:08.348811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:08.348811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:08.348811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:08.348811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:08.348811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:08.348811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:08.348811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:08.348811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:08.348811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 81/166] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:36:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:08.350415 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844abe8c-0c58-4bd2-b670-0d398f9e76c1 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:08.350415 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844abe8c-0c58-4bd2-b670-0d398f9e76c1 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:08.350415 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844abe8c-0c58-4bd2-b670-0d398f9e76c1 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:08.351912 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 82/167] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:08.357270 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844abe8c-0c58-4bd2-b670-0d398f9e76c1 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:08.432633 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c62b95-22c5-4da9-bad3-86f4c2b3103f None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:08.473748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6fedd-16a5-42c0-810c-854dee6174fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:08.474683 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6fedd-16a5-42c0-810c-854dee6174fa None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:08.474962 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6fedd-16a5-42c0-810c-854dee6174fa None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:08.475232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6fedd-16a5-42c0-810c-854dee6174fa None admin] PATH_INFO: `/v3/users/8340d97d04ba441bbf713dcd96777a4e` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:08.476609 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6fedd-16a5-42c0-810c-854dee6174fa None admin] RBAC: Authorizing `identity:get_user(user_id=8340d97d04ba441bbf713dcd96777a4e)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:08.478529 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6fedd-16a5-42c0-810c-854dee6174fa None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:08.480416 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 86/168] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:08 2022] GET /identity/v3/users/8340d97d04ba441bbf713dcd96777a4e => generated 289 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:08.481274 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6fedd-16a5-42c0-810c-854dee6174fa None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:08.509372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844abe8c-0c58-4bd2-b670-0d398f9e76c1 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:08.545511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b7969-c230-4f31-a81a-f328e10d9127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:08.546334 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b7969-c230-4f31-a81a-f328e10d9127 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:08.546614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b7969-c230-4f31-a81a-f328e10d9127 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:08.546922 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b7969-c230-4f31-a81a-f328e10d9127 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:08.548434 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:08.560484 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b7969-c230-4f31-a81a-f328e10d9127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-f72b7969-c230-4f31-a81a-f328e10d9127 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:08.569128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:08.570188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:08.571074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:08.571074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:08.571074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:08.571074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:08.571074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:08.571074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:08.571074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:08.571074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:08.571074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:08.571074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:08.571074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:08.571074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:08.573096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 83/169] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:08.574513 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943edf53-d78c-4791-a743-b6b43f2bfbae None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:08.574967 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943edf53-d78c-4791-a743-b6b43f2bfbae None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:08.575504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943edf53-d78c-4791-a743-b6b43f2bfbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:08.595685 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-943edf53-d78c-4791-a743-b6b43f2bfbae None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:08.597117 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6fedd-16a5-42c0-810c-854dee6174fa None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:08.597367 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-943edf53-d78c-4791-a743-b6b43f2bfbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:08.609385 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:08.653590 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75905464-39d4-4ac5-bbe9-1c73b5c8cd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:08.654512 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75905464-39d4-4ac5-bbe9-1c73b5c8cd44 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:08.654663 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75905464-39d4-4ac5-bbe9-1c73b5c8cd44 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:08.654934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75905464-39d4-4ac5-bbe9-1c73b5c8cd44 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:08.655684 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75905464-39d4-4ac5-bbe9-1c73b5c8cd44 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:08.655986 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75905464-39d4-4ac5-bbe9-1c73b5c8cd44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:08.657489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75905464-39d4-4ac5-bbe9-1c73b5c8cd44 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:08.668820 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 87/170] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:08.669753 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75905464-39d4-4ac5-bbe9-1c73b5c8cd44 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:08.683495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 84/171] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:08.684621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943edf53-d78c-4791-a743-b6b43f2bfbae None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:08.792519 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943edf53-d78c-4791-a743-b6b43f2bfbae None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:08.800811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75905464-39d4-4ac5-bbe9-1c73b5c8cd44 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:08.825145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:08.829897 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54de08d-1837-4672-addd-b0fd6ca74721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:08.830637 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54de08d-1837-4672-addd-b0fd6ca74721 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:08.830911 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54de08d-1837-4672-addd-b0fd6ca74721 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:08.831177 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54de08d-1837-4672-addd-b0fd6ca74721 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:08.831919 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54de08d-1837-4672-addd-b0fd6ca74721 None admin] RBAC: Adding query filter params (name=member) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:08.832542 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54de08d-1837-4672-addd-b0fd6ca74721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:08.833938 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54de08d-1837-4672-addd-b0fd6ca74721 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:08.836773 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a08bb2-9920-4402-a93e-9ae2c2132ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:08.837639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a08bb2-9920-4402-a93e-9ae2c2132ba8 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:08.837915 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a08bb2-9920-4402-a93e-9ae2c2132ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:08.838184 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a08bb2-9920-4402-a93e-9ae2c2132ba8 None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:08.839387 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a08bb2-9920-4402-a93e-9ae2c2132ba8 None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:08.840827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a08bb2-9920-4402-a93e-9ae2c2132ba8 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:08.842354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 85/172] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:08.846830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 88/173] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:08 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:08.846961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54de08d-1837-4672-addd-b0fd6ca74721 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:08.847063 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a08bb2-9920-4402-a93e-9ae2c2132ba8 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:08.851100 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:08.943959 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54de08d-1837-4672-addd-b0fd6ca74721 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:08.946014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a08bb2-9920-4402-a93e-9ae2c2132ba8 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:08.977772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800c506-4812-4511-9d84-2e74b04d566c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:08.978696 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800c506-4812-4511-9d84-2e74b04d566c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:08.978830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07e22e-08f2-40d4-83e1-c3f5e5442dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:08.979003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800c506-4812-4511-9d84-2e74b04d566c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:08.979239 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800c506-4812-4511-9d84-2e74b04d566c None admin] PATH_INFO: `/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:08.979604 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07e22e-08f2-40d4-83e1-c3f5e5442dda None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:08.979876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07e22e-08f2-40d4-83e1-c3f5e5442dda None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:08.980127 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07e22e-08f2-40d4-83e1-c3f5e5442dda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:08.980460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800c506-4812-4511-9d84-2e74b04d566c None admin] RBAC: Authorizing `identity:get_user(user_id=6c5e8aab0eaa42288cf4b090c1e64ae7)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:08.981909 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800c506-4812-4511-9d84-2e74b04d566c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:08.988065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 86/174] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:08 2022] GET /identity/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:08.988065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a58986-67e1-4cc8-8d5d-a462d739800b None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:08.988065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a58986-67e1-4cc8-8d5d-a462d739800b None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:08.988065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a58986-67e1-4cc8-8d5d-a462d739800b None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:08.988065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 87/175] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:08.988065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5e74d-d555-44d5-a69d-772fdab06eed None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:08.988893 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5e74d-d555-44d5-a69d-772fdab06eed None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:08.988893 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5e74d-d555-44d5-a69d-772fdab06eed None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:08.989861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 88/176] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:08.991972 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5e74d-d555-44d5-a69d-772fdab06eed None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:08.998842 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07e22e-08f2-40d4-83e1-c3f5e5442dda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-0c07e22e-08f2-40d4-83e1-c3f5e5442dda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:09.006920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:09.007748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:09.013492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:09.013492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:09.013492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:09.013492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:09.013492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:09.013492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:09.013492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:09.013492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:09.013492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:09.013492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:09.013492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:09.013492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:09.013492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 89/177] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:36:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:09.013492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07e22e-08f2-40d4-83e1-c3f5e5442dda None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:09.062793 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:09.067728 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:09.099758 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5e74d-d555-44d5-a69d-772fdab06eed None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:09.114811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07e22e-08f2-40d4-83e1-c3f5e5442dda None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:09.133269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40492b17-f512-45e1-b553-c8b53c4bdbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:09.134163 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40492b17-f512-45e1-b553-c8b53c4bdbf7 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:09.134463 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40492b17-f512-45e1-b553-c8b53c4bdbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:09.134729 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40492b17-f512-45e1-b553-c8b53c4bdbf7 None admin] PATH_INFO: `/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:09.135882 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40492b17-f512-45e1-b553-c8b53c4bdbf7 None admin] RBAC: Authorizing `identity:get_user(user_id=6c5e8aab0eaa42288cf4b090c1e64ae7)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:09.137326 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40492b17-f512-45e1-b553-c8b53c4bdbf7 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:09.139001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 89/178] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:08 2022] GET /identity/v3/users/6c5e8aab0eaa42288cf4b090c1e64ae7 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:09.143533 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40492b17-f512-45e1-b553-c8b53c4bdbf7 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:09.148868 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62661072-310f-417e-99f7-b68478da11cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:09.149760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62661072-310f-417e-99f7-b68478da11cb None admin] REQUEST_METHOD: `PUT` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:09.150039 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62661072-310f-417e-99f7-b68478da11cb None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:09.150345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62661072-310f-417e-99f7-b68478da11cb None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f/users/8340d97d04ba441bbf713dcd96777a4e/roles/6ab02cf5cb1b4d3595ea8ea883ff6f2d` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:09.152365 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62661072-310f-417e-99f7-b68478da11cb None admin] RBAC: Authorizing `identity:create_grant(project_id=644a8176f5874355bc660ed98897307f, user_id=8340d97d04ba441bbf713dcd96777a4e, role_id=6ab02cf5cb1b4d3595ea8ea883ff6f2d)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:09.153854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62661072-310f-417e-99f7-b68478da11cb None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:09.168317 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 90/179] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:36:09 2022] PUT /identity/v3/projects/644a8176f5874355bc660ed98897307f/users/8340d97d04ba441bbf713dcd96777a4e/roles/6ab02cf5cb1b4d3595ea8ea883ff6f2d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:36:09.169553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec25829-ca57-48af-921d-658b5fad1e3b None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:09.169837 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec25829-ca57-48af-921d-658b5fad1e3b None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:09.170112 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec25829-ca57-48af-921d-658b5fad1e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:09.199012 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-1ec25829-ca57-48af-921d-658b5fad1e3b None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:09.200563 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-1ec25829-ca57-48af-921d-658b5fad1e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:09.254194 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40492b17-f512-45e1-b553-c8b53c4bdbf7 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:09.279696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 91/180] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:09.280991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708797c-cbfc-4ada-baaf-8fd42a83c0e4 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:09.281281 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708797c-cbfc-4ada-baaf-8fd42a83c0e4 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:09.281552 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708797c-cbfc-4ada-baaf-8fd42a83c0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:09.299313 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-e708797c-cbfc-4ada-baaf-8fd42a83c0e4 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:09.300524 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-e708797c-cbfc-4ada-baaf-8fd42a83c0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:09.315919 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f04656-995c-4a10-b013-2499a6d2e968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:09.317046 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f04656-995c-4a10-b013-2499a6d2e968 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:09.317535 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f04656-995c-4a10-b013-2499a6d2e968 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:09.317940 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f04656-995c-4a10-b013-2499a6d2e968 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:09.330680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:09.339704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:09.377034 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 92/181] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:09.377810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708797c-cbfc-4ada-baaf-8fd42a83c0e4 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:09.406926 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f04656-995c-4a10-b013-2499a6d2e968 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:09.409035 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f04656-995c-4a10-b013-2499a6d2e968 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:09.482073 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708797c-cbfc-4ada-baaf-8fd42a83c0e4 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:09.517752 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 90/182] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:09.518421 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f04656-995c-4a10-b013-2499a6d2e968 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:09.520746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea4eb5-0426-473f-a31a-d67844c76e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:09.521727 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea4eb5-0426-473f-a31a-d67844c76e0b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:09.522759 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea4eb5-0426-473f-a31a-d67844c76e0b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:09.523039 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea4eb5-0426-473f-a31a-d67844c76e0b None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:09.523814 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea4eb5-0426-473f-a31a-d67844c76e0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:09.524124 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea4eb5-0426-473f-a31a-d67844c76e0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:09.525543 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea4eb5-0426-473f-a31a-d67844c76e0b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:09.534681 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 93/183] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:09.535379 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea4eb5-0426-473f-a31a-d67844c76e0b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:09.569580 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:09.572914 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:09.639599 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f04656-995c-4a10-b013-2499a6d2e968 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:09.648233 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea4eb5-0426-473f-a31a-d67844c76e0b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:09.675084 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060aff0b-ff4d-4b97-ab2c-da82bc0c4df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:09.675781 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060aff0b-ff4d-4b97-ab2c-da82bc0c4df3 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:09.676302 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060aff0b-ff4d-4b97-ab2c-da82bc0c4df3 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:09.676586 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060aff0b-ff4d-4b97-ab2c-da82bc0c4df3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:09.684324 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c01ce4-50f3-4e9a-9769-ee7521bfa101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:09.685140 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c01ce4-50f3-4e9a-9769-ee7521bfa101 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:09.685420 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c01ce4-50f3-4e9a-9769-ee7521bfa101 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:09.685697 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c01ce4-50f3-4e9a-9769-ee7521bfa101 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:09.696878 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c01ce4-50f3-4e9a-9769-ee7521bfa101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-c0c01ce4-50f3-4e9a-9769-ee7521bfa101 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:09.705225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:09.706083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:09.706885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:09.706885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:09.706885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:09.706885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:09.706885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:09.706885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:09.706885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:09.706885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:09.706885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:09.706885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:09.706885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:09.706885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:09.706885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 94/184] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:36:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:09.707558 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c01ce4-50f3-4e9a-9769-ee7521bfa101 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:09.757510 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060aff0b-ff4d-4b97-ab2c-da82bc0c4df3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:09.759619 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060aff0b-ff4d-4b97-ab2c-da82bc0c4df3 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:09.805588 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:09.807420 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:09.817694 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c01ce4-50f3-4e9a-9769-ee7521bfa101 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:09.856739 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212bdc7-b6e8-4450-b69d-b4e52f0cd8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:09.857863 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212bdc7-b6e8-4450-b69d-b4e52f0cd8cb None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:09.858184 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212bdc7-b6e8-4450-b69d-b4e52f0cd8cb None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:09.858445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212bdc7-b6e8-4450-b69d-b4e52f0cd8cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:09.859620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212bdc7-b6e8-4450-b69d-b4e52f0cd8cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:09.861089 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212bdc7-b6e8-4450-b69d-b4e52f0cd8cb None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:09.862904 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 95/185] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:09.867954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212bdc7-b6e8-4450-b69d-b4e52f0cd8cb None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:09.881601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 91/186] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:09.881601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060aff0b-ff4d-4b97-ab2c-da82bc0c4df3 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:09.969942 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212bdc7-b6e8-4450-b69d-b4e52f0cd8cb None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:09.989982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060aff0b-ff4d-4b97-ab2c-da82bc0c4df3 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:10.005373 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a839f99-37e8-4d68-b8b7-556025c518b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:10.006243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a839f99-37e8-4d68-b8b7-556025c518b2 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:10.006561 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a839f99-37e8-4d68-b8b7-556025c518b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:10.006840 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a839f99-37e8-4d68-b8b7-556025c518b2 None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:10.008048 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a839f99-37e8-4d68-b8b7-556025c518b2 None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:10.009518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a839f99-37e8-4d68-b8b7-556025c518b2 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:10.016145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 96/187] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:09 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:10.016145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a839f99-37e8-4d68-b8b7-556025c518b2 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:10.035393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc29d2a-af79-497b-bd29-8e6f572839b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:10.036401 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc29d2a-af79-497b-bd29-8e6f572839b0 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:10.036685 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc29d2a-af79-497b-bd29-8e6f572839b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:10.036960 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc29d2a-af79-497b-bd29-8e6f572839b0 None admin] PATH_INFO: `/v3/users/b417bc81fb754260ba335d2c48fa86a7` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:10.048849 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:10.052642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:10.055162 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc29d2a-af79-497b-bd29-8e6f572839b0 None admin] RBAC: Authorizing `identity:get_user(user_id=b417bc81fb754260ba335d2c48fa86a7)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:10.056813 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc29d2a-af79-497b-bd29-8e6f572839b0 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:10.059350 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 92/188] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:09 2022] GET /identity/v3/users/b417bc81fb754260ba335d2c48fa86a7 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:10.060604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc29d2a-af79-497b-bd29-8e6f572839b0 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:10.128644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a839f99-37e8-4d68-b8b7-556025c518b2 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:10.162303 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f803f7-2bef-4b6b-b3e6-54124dc684fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:10.162978 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f803f7-2bef-4b6b-b3e6-54124dc684fa None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:10.163262 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f803f7-2bef-4b6b-b3e6-54124dc684fa None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:10.163527 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f803f7-2bef-4b6b-b3e6-54124dc684fa None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:10.164279 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f803f7-2bef-4b6b-b3e6-54124dc684fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:10.164581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f803f7-2bef-4b6b-b3e6-54124dc684fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:10.165973 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f803f7-2bef-4b6b-b3e6-54124dc684fa None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:10.166972 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc29d2a-af79-497b-bd29-8e6f572839b0 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:10.174193 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 97/189] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:10.175006 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f803f7-2bef-4b6b-b3e6-54124dc684fa None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:10.205191 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f252a28-1cbe-4442-a508-8a42756abad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:10.205867 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f252a28-1cbe-4442-a508-8a42756abad5 None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:10.206141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f252a28-1cbe-4442-a508-8a42756abad5 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:10.206448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f252a28-1cbe-4442-a508-8a42756abad5 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:10.207210 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f252a28-1cbe-4442-a508-8a42756abad5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:10.208665 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f252a28-1cbe-4442-a508-8a42756abad5 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:10.238375 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 93/190] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:36:10 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:36:10.238375 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f252a28-1cbe-4442-a508-8a42756abad5 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:10.286338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:10.301222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f803f7-2bef-4b6b-b3e6-54124dc684fa None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:10.316067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:10.350968 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50489163-f0c7-4db8-8db9-f5c08fcbe6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:10.351861 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50489163-f0c7-4db8-8db9-f5c08fcbe6ee None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:10.352204 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50489163-f0c7-4db8-8db9-f5c08fcbe6ee None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:10.352790 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50489163-f0c7-4db8-8db9-f5c08fcbe6ee None admin] PATH_INFO: `/v3/projects/07ad62c92dd548cb8b7de09cf8881d4b` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:10.354284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50489163-f0c7-4db8-8db9-f5c08fcbe6ee None admin] RBAC: Authorizing `identity:get_project(project_id=07ad62c92dd548cb8b7de09cf8881d4b)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:10.355810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50489163-f0c7-4db8-8db9-f5c08fcbe6ee None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:10.358384 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 98/191] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:10 2022] GET /identity/v3/projects/07ad62c92dd548cb8b7de09cf8881d4b => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:10.359252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50489163-f0c7-4db8-8db9-f5c08fcbe6ee None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:10.384065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f252a28-1cbe-4442-a508-8a42756abad5 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:10.434759 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d257b4-247e-4a35-9d48-8fdc0f888964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:10.435429 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d257b4-247e-4a35-9d48-8fdc0f888964 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:10.435948 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d257b4-247e-4a35-9d48-8fdc0f888964 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:10.436222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d257b4-247e-4a35-9d48-8fdc0f888964 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:10.438841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d257b4-247e-4a35-9d48-8fdc0f888964 None admin] RBAC: Adding query filter params (role.id=4a241925b6cc4218b4f2af580b151877, scope.project.id=644a8176f5874355bc660ed98897307f, user.id=6c5e8aab0eaa42288cf4b090c1e64ae7) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:10.438841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d257b4-247e-4a35-9d48-8fdc0f888964 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:10.440971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d257b4-247e-4a35-9d48-8fdc0f888964 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:10.448410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 94/192] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:10 2022] GET /identity/v3/role_assignments?user.id=6c5e8aab0eaa42288cf4b090c1e64ae7&scope.project.id=644a8176f5874355bc660ed98897307f&role.id=4a241925b6cc4218b4f2af580b151877 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:10.454997 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d257b4-247e-4a35-9d48-8fdc0f888964 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:10.486138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50489163-f0c7-4db8-8db9-f5c08fcbe6ee None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:10.535867 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdc80e-3c61-4eb5-8691-466daf6933bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:10.537636 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdc80e-3c61-4eb5-8691-466daf6933bf None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:10.538404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdc80e-3c61-4eb5-8691-466daf6933bf None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:10.538908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdc80e-3c61-4eb5-8691-466daf6933bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:10.578063 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:10.581022 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d257b4-247e-4a35-9d48-8fdc0f888964 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:10.602006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:10.640086 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338867d-f182-4983-b6b6-1d160aa868ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:10.641148 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338867d-f182-4983-b6b6-1d160aa868ed None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:10.641464 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338867d-f182-4983-b6b6-1d160aa868ed None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:10.641924 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338867d-f182-4983-b6b6-1d160aa868ed None admin] PATH_INFO: `/v3/users/b417bc81fb754260ba335d2c48fa86a7` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:10.643201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338867d-f182-4983-b6b6-1d160aa868ed None admin] RBAC: Authorizing `identity:get_user(user_id=b417bc81fb754260ba335d2c48fa86a7)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:10.644868 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338867d-f182-4983-b6b6-1d160aa868ed None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:10.646897 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 95/193] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:10 2022] GET /identity/v3/users/b417bc81fb754260ba335d2c48fa86a7 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:10.647547 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338867d-f182-4983-b6b6-1d160aa868ed None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:10.663275 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fdc80e-3c61-4eb5-8691-466daf6933bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:10.665874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fdc80e-3c61-4eb5-8691-466daf6933bf None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:10.779498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338867d-f182-4983-b6b6-1d160aa868ed None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:10.797580 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 99/194] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:10.798788 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdc80e-3c61-4eb5-8691-466daf6933bf None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:10.824194 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e69da-932b-4c79-bf16-fde9196161df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:10.824859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e69da-932b-4c79-bf16-fde9196161df None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:10.825138 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e69da-932b-4c79-bf16-fde9196161df None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:10.825409 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e69da-932b-4c79-bf16-fde9196161df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:10.826240 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e69da-932b-4c79-bf16-fde9196161df None admin] RBAC: Adding query filter params (role.id=fe332203b7e04d6084e26c5be292af35, scope.project.id=07ad62c92dd548cb8b7de09cf8881d4b, user.id=6c5e8aab0eaa42288cf4b090c1e64ae7) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:10.826576 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e69da-932b-4c79-bf16-fde9196161df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:10.828031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e69da-932b-4c79-bf16-fde9196161df None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:10.835629 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 96/195] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:10 2022] GET /identity/v3/role_assignments?user.id=6c5e8aab0eaa42288cf4b090c1e64ae7&scope.project.id=07ad62c92dd548cb8b7de09cf8881d4b&role.id=fe332203b7e04d6084e26c5be292af35 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:10.836390 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e69da-932b-4c79-bf16-fde9196161df None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:10.886076 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:10.906722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:10.954329 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdc80e-3c61-4eb5-8691-466daf6933bf None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:10.998236 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c6339-2c06-408e-bd0a-2ac80e1f9185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:10.999946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c6339-2c06-408e-bd0a-2ac80e1f9185 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:11.000822 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c6339-2c06-408e-bd0a-2ac80e1f9185 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:11.001304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c6339-2c06-408e-bd0a-2ac80e1f9185 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:11.013547 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e69da-932b-4c79-bf16-fde9196161df None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:11.091628 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c6339-2c06-408e-bd0a-2ac80e1f9185 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:11.091628 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c6339-2c06-408e-bd0a-2ac80e1f9185 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:11.105410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543fd3c-f15b-4139-9764-27b9997aa4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:11.106650 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543fd3c-f15b-4139-9764-27b9997aa4fc None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:11.107125 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543fd3c-f15b-4139-9764-27b9997aa4fc None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:11.108244 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543fd3c-f15b-4139-9764-27b9997aa4fc None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:11.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543fd3c-f15b-4139-9764-27b9997aa4fc None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:11.111378 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543fd3c-f15b-4139-9764-27b9997aa4fc None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:11.113881 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 97/196] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:10 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:11.120771 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543fd3c-f15b-4139-9764-27b9997aa4fc None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:11.219424 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:11.304682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 100/197] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:11.311532 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c6339-2c06-408e-bd0a-2ac80e1f9185 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:11.337483 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543fd3c-f15b-4139-9764-27b9997aa4fc None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:11.368540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:11.402973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934cda9-c4e4-40c9-a85f-bb190c93652a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:11.403900 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934cda9-c4e4-40c9-a85f-bb190c93652a None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:11.404345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934cda9-c4e4-40c9-a85f-bb190c93652a None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:11.404755 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934cda9-c4e4-40c9-a85f-bb190c93652a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:11.405732 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934cda9-c4e4-40c9-a85f-bb190c93652a None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=7c06d53d4c434698958a453f9500feaa, user.id=b417bc81fb754260ba335d2c48fa86a7) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:11.406171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934cda9-c4e4-40c9-a85f-bb190c93652a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:11.407799 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934cda9-c4e4-40c9-a85f-bb190c93652a None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:11.424339 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 98/198] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:11 2022] GET /identity/v3/role_assignments?user.id=b417bc81fb754260ba335d2c48fa86a7&scope.project.id=7c06d53d4c434698958a453f9500feaa&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:11.461323 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c6339-2c06-408e-bd0a-2ac80e1f9185 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:11.509007 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1d88a-9165-4a59-a986-424459c69933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:11.509891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1d88a-9165-4a59-a986-424459c69933 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:11.510171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1d88a-9165-4a59-a986-424459c69933 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:11.510493 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1d88a-9165-4a59-a986-424459c69933 None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:11.511768 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1d88a-9165-4a59-a986-424459c69933 None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:11.513253 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1d88a-9165-4a59-a986-424459c69933 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:11.515040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 101/199] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:11 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:11.521330 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934cda9-c4e4-40c9-a85f-bb190c93652a None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:11.661968 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b900f7-3dea-4820-a06c-e07510a102d0 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:11.662141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b900f7-3dea-4820-a06c-e07510a102d0 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:11.662450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b900f7-3dea-4820-a06c-e07510a102d0 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:11.663995 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 102/200] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:11.670428 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b0b9e-f9fc-4364-9e54-e8f9085f3dfb None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:11.670714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b0b9e-f9fc-4364-9e54-e8f9085f3dfb None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:11.670986 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b0b9e-f9fc-4364-9e54-e8f9085f3dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:11.694797 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934cda9-c4e4-40c9-a85f-bb190c93652a None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:11.710622 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-235b0b9e-f9fc-4364-9e54-e8f9085f3dfb None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:11.712192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-235b0b9e-f9fc-4364-9e54-e8f9085f3dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:11.751954 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748583fe-5832-4619-b2c0-1fd219531aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:11.752989 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748583fe-5832-4619-b2c0-1fd219531aab None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:11.753460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748583fe-5832-4619-b2c0-1fd219531aab None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:11.753975 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748583fe-5832-4619-b2c0-1fd219531aab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:11.755072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748583fe-5832-4619-b2c0-1fd219531aab None admin] RBAC: Adding query filter params (role.id=4a241925b6cc4218b4f2af580b151877, scope.project.id=7c06d53d4c434698958a453f9500feaa, user.id=b417bc81fb754260ba335d2c48fa86a7) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:11.755587 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748583fe-5832-4619-b2c0-1fd219531aab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:11.755865 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:11.757458 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748583fe-5832-4619-b2c0-1fd219531aab None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:11.764173 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 99/201] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:11 2022] GET /identity/v3/role_assignments?user.id=b417bc81fb754260ba335d2c48fa86a7&scope.project.id=7c06d53d4c434698958a453f9500feaa&role.id=4a241925b6cc4218b4f2af580b151877 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:11.820741 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 103/202] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:11.859021 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748583fe-5832-4619-b2c0-1fd219531aab None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:11.916151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:11.975509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748583fe-5832-4619-b2c0-1fd219531aab None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:12.012262 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0eee52-bad0-4693-b57d-8190e94f733a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:12.013449 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0eee52-bad0-4693-b57d-8190e94f733a None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:12.013952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0eee52-bad0-4693-b57d-8190e94f733a None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:12.014450 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0eee52-bad0-4693-b57d-8190e94f733a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:12.027231 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0eee52-bad0-4693-b57d-8190e94f733a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-db0eee52-bad0-4693-b57d-8190e94f733a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:12.036305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:12.037345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:12.038109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:12.038109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:12.038109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:12.038109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:12.038109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:12.038109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:12.038109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:12.038109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:12.038109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:12.038109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:12.038109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:12.038109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:12.041137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 100/203] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:12.047071 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b0b9e-f9fc-4364-9e54-e8f9085f3dfb None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:12.161333 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b0b9e-f9fc-4364-9e54-e8f9085f3dfb None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:12.207645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ea49c-2c50-4c08-a838-09796786eecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:12.208730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ea49c-2c50-4c08-a838-09796786eecc None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:12.209202 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ea49c-2c50-4c08-a838-09796786eecc None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:12.209661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ea49c-2c50-4c08-a838-09796786eecc None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:12.210639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ea49c-2c50-4c08-a838-09796786eecc None admin] RBAC: Adding query filter params (name=reader) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:12.211136 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ea49c-2c50-4c08-a838-09796786eecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:12.212719 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ea49c-2c50-4c08-a838-09796786eecc None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:12.226880 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 104/204] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:12.232934 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0eee52-bad0-4693-b57d-8190e94f733a None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:12.382156 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0eee52-bad0-4693-b57d-8190e94f733a None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:12.389659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:12.438775 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863f70d-e6bc-4560-9dbf-0daa66245c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:12.439677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863f70d-e6bc-4560-9dbf-0daa66245c45 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:12.439958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863f70d-e6bc-4560-9dbf-0daa66245c45 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:12.440232 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863f70d-e6bc-4560-9dbf-0daa66245c45 None admin] PATH_INFO: `/v3/users/8340d97d04ba441bbf713dcd96777a4e` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:12.443465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863f70d-e6bc-4560-9dbf-0daa66245c45 None admin] RBAC: Authorizing `identity:get_user(user_id=8340d97d04ba441bbf713dcd96777a4e)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:12.445439 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863f70d-e6bc-4560-9dbf-0daa66245c45 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:12.447149 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 101/205] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:12 2022] GET /identity/v3/users/8340d97d04ba441bbf713dcd96777a4e => generated 289 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:12.453086 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ea49c-2c50-4c08-a838-09796786eecc None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:12.490034 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:12.575865 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ea49c-2c50-4c08-a838-09796786eecc None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:12.632288 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfe04d-f0f4-4985-ac0f-ee162564dd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:12.632288 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfe04d-f0f4-4985-ac0f-ee162564dd18 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:12.632288 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfe04d-f0f4-4985-ac0f-ee162564dd18 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:12.632288 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfe04d-f0f4-4985-ac0f-ee162564dd18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:12.730209 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bfe04d-f0f4-4985-ac0f-ee162564dd18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:12.731879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bfe04d-f0f4-4985-ac0f-ee162564dd18 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:12.753415 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:12.864466 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 105/206] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:12.870154 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863f70d-e6bc-4560-9dbf-0daa66245c45 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:12.989200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863f70d-e6bc-4560-9dbf-0daa66245c45 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:13.029846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2d6f9-4b12-4465-b62b-51c319c7b1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:13.031130 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2d6f9-4b12-4465-b62b-51c319c7b1df None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:13.031625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2d6f9-4b12-4465-b62b-51c319c7b1df None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:13.032089 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2d6f9-4b12-4465-b62b-51c319c7b1df None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:13.033481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2d6f9-4b12-4465-b62b-51c319c7b1df None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:13.035261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2d6f9-4b12-4465-b62b-51c319c7b1df None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:13.037543 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 102/207] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:12 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:13.043031 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfe04d-f0f4-4985-ac0f-ee162564dd18 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:13.201181 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfe04d-f0f4-4985-ac0f-ee162564dd18 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:13.220354 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:13.240778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1348f-3179-4f06-81f4-18f604032c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:13.241438 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1348f-3179-4f06-81f4-18f604032c5a None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:13.241710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1348f-3179-4f06-81f4-18f604032c5a None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:13.241972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1348f-3179-4f06-81f4-18f604032c5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:13.242853 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1348f-3179-4f06-81f4-18f604032c5a None admin] RBAC: Adding query filter params (role.id=6ab02cf5cb1b4d3595ea8ea883ff6f2d, scope.project.id=644a8176f5874355bc660ed98897307f, user.id=8340d97d04ba441bbf713dcd96777a4e) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:13.243159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1348f-3179-4f06-81f4-18f604032c5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:13.244630 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1348f-3179-4f06-81f4-18f604032c5a None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:13.251327 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 106/208] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:13 2022] GET /identity/v3/role_assignments?user.id=8340d97d04ba441bbf713dcd96777a4e&scope.project.id=644a8176f5874355bc660ed98897307f&role.id=6ab02cf5cb1b4d3595ea8ea883ff6f2d => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:13.488791 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b056c2-e48f-4b4d-a97f-7b72304e6ad1 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:13.489176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b056c2-e48f-4b4d-a97f-7b72304e6ad1 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:13.489367 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b056c2-e48f-4b4d-a97f-7b72304e6ad1 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:13.490842 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 103/209] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:13.507463 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457d1e9-cf6c-45df-a31c-380b9866b6c4 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:13.507754 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457d1e9-cf6c-45df-a31c-380b9866b6c4 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:13.508003 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457d1e9-cf6c-45df-a31c-380b9866b6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:13.527581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-5457d1e9-cf6c-45df-a31c-380b9866b6c4 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:13.528806 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-5457d1e9-cf6c-45df-a31c-380b9866b6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:13.605809 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 107/210] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:13.612448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ede645-532b-4272-a5f6-e66b2da6ddcc None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:13.612726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ede645-532b-4272-a5f6-e66b2da6ddcc None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:13.612997 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ede645-532b-4272-a5f6-e66b2da6ddcc None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:13.616813 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 104/211] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:36:13.620248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7a3a1-5677-414b-9b37-8199892d37fc None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:13.620528 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7a3a1-5677-414b-9b37-8199892d37fc None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:13.620851 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7a3a1-5677-414b-9b37-8199892d37fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:13.634994 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ede645-532b-4272-a5f6-e66b2da6ddcc None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:13.639009 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-68d7a3a1-5677-414b-9b37-8199892d37fc None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:13.640326 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-68d7a3a1-5677-414b-9b37-8199892d37fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:13.700037 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:13.733359 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 108/212] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:13.760643 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7a3a1-5677-414b-9b37-8199892d37fc None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:13.776280 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ede645-532b-4272-a5f6-e66b2da6ddcc None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:13.813698 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd23294-0af6-4010-b6ac-b07a00eff6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:13.814897 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd23294-0af6-4010-b6ac-b07a00eff6c9 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:13.815469 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd23294-0af6-4010-b6ac-b07a00eff6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:13.815938 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd23294-0af6-4010-b6ac-b07a00eff6c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:13.817000 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:13.818617 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd23294-0af6-4010-b6ac-b07a00eff6c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:13.820286 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd23294-0af6-4010-b6ac-b07a00eff6c9 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:13.822447 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 105/213] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:13.827679 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd23294-0af6-4010-b6ac-b07a00eff6c9 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:13.898435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7a3a1-5677-414b-9b37-8199892d37fc None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:13.933275 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd917781-4ce6-4b12-b798-0546aba68dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:13.934339 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd917781-4ce6-4b12-b798-0546aba68dbc None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:13.934992 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd917781-4ce6-4b12-b798-0546aba68dbc None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:13.935462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd917781-4ce6-4b12-b798-0546aba68dbc None admin] PATH_INFO: `/v3/roles/member` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:13.946115 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd917781-4ce6-4b12-b798-0546aba68dbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:13.948291 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd23294-0af6-4010-b6ac-b07a00eff6c9 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-dd917781-4ce6-4b12-b798-0546aba68dbc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:13.954571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:13.955602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:13.956552 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:13.956552 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:13.956552 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:13.956552 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:13.956552 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:13.956552 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:13.956552 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:13.956552 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:13.956552 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:13.956552 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:13.956552 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:13.956552 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:13.959302 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 109/214] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:13.966016 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:13.967008 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd917781-4ce6-4b12-b798-0546aba68dbc None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:14.000058 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d18880-1cd6-40ca-90af-742f37cf247e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:14.001301 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d18880-1cd6-40ca-90af-742f37cf247e None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:14.001766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d18880-1cd6-40ca-90af-742f37cf247e None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:14.002182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d18880-1cd6-40ca-90af-742f37cf247e None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:14.003703 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d18880-1cd6-40ca-90af-742f37cf247e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:14.005986 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d18880-1cd6-40ca-90af-742f37cf247e None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:14.043208 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 106/215] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:36:13 2022] POST /identity/v3/users => generated 297 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:36:14.096797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd917781-4ce6-4b12-b798-0546aba68dbc None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:14.099577 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:14.139292 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15ab2f-4d7a-4dba-89e7-9fdc2f82645e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:14.139977 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15ab2f-4d7a-4dba-89e7-9fdc2f82645e None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:14.140252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15ab2f-4d7a-4dba-89e7-9fdc2f82645e None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:14.140525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15ab2f-4d7a-4dba-89e7-9fdc2f82645e None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:14.141301 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15ab2f-4d7a-4dba-89e7-9fdc2f82645e None admin] RBAC: Adding query filter params (name=member) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:14.141612 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15ab2f-4d7a-4dba-89e7-9fdc2f82645e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:14.143075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15ab2f-4d7a-4dba-89e7-9fdc2f82645e None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:14.153113 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 110/216] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:14.158911 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d18880-1cd6-40ca-90af-742f37cf247e None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:14.167132 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde593cc-98f3-4ee9-bdd9-69a32ef790ba None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:14.167435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde593cc-98f3-4ee9-bdd9-69a32ef790ba None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:14.167680 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde593cc-98f3-4ee9-bdd9-69a32ef790ba None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:14.169456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 111/217] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:14.175251 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659d9a3-a1e6-4c85-9e0d-b704ed1c34af None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:14.175549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659d9a3-a1e6-4c85-9e0d-b704ed1c34af None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:14.175956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659d9a3-a1e6-4c85-9e0d-b704ed1c34af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:14.202023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-1659d9a3-a1e6-4c85-9e0d-b704ed1c34af None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:14.204186 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-1659d9a3-a1e6-4c85-9e0d-b704ed1c34af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:14.279264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 112/218] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:14.286784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d18880-1cd6-40ca-90af-742f37cf247e None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:14.302116 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659d9a3-a1e6-4c85-9e0d-b704ed1c34af None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:14.336960 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7290cb1-6f9b-42eb-9db4-10fdb6828af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:14.338500 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7290cb1-6f9b-42eb-9db4-10fdb6828af3 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:14.338975 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7290cb1-6f9b-42eb-9db4-10fdb6828af3 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:14.339259 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7290cb1-6f9b-42eb-9db4-10fdb6828af3 None admin] PATH_INFO: `/v3/users/3cce36ca0d794853b17a35ca0b7494d8` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:14.357072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7290cb1-6f9b-42eb-9db4-10fdb6828af3 None admin] RBAC: Authorizing `identity:get_user(user_id=3cce36ca0d794853b17a35ca0b7494d8)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:14.357741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7290cb1-6f9b-42eb-9db4-10fdb6828af3 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:14.359510 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 107/219] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:14 2022] GET /identity/v3/users/3cce36ca0d794853b17a35ca0b7494d8 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:14.365211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7290cb1-6f9b-42eb-9db4-10fdb6828af3 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:14.404350 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:14.429707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:14.437100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659d9a3-a1e6-4c85-9e0d-b704ed1c34af None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:14.481238 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6b8ca-bae8-41bd-997c-ead8234205ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:14.482052 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6b8ca-bae8-41bd-997c-ead8234205ce None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:14.482356 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6b8ca-bae8-41bd-997c-ead8234205ce None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:14.482633 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6b8ca-bae8-41bd-997c-ead8234205ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:14.495039 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6b8ca-bae8-41bd-997c-ead8234205ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:14.503288 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7290cb1-6f9b-42eb-9db4-10fdb6828af3 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-90b6b8ca-bae8-41bd-997c-ead8234205ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:14.504266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:14.505092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:14.505954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:14.505954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:14.505954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:14.505954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:14.505954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:14.505954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:14.505954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:14.505954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:14.505954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:14.505954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:14.505954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:14.505954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:14.507014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 113/220] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:36:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:36:14.515730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6b8ca-bae8-41bd-997c-ead8234205ce None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:14.544346 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3908df-a54b-4219-93d1-5d380a1b6408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:14.545301 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3908df-a54b-4219-93d1-5d380a1b6408 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:14.545715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3908df-a54b-4219-93d1-5d380a1b6408 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:14.546117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3908df-a54b-4219-93d1-5d380a1b6408 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:14.620326 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3908df-a54b-4219-93d1-5d380a1b6408 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:14.622094 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3908df-a54b-4219-93d1-5d380a1b6408 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:14.630111 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6b8ca-bae8-41bd-997c-ead8234205ce None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:14.650415 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:14.669377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0480b-0709-48de-9e24-d1d2dc8c59ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:14.670331 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0480b-0709-48de-9e24-d1d2dc8c59ef None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:14.670677 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0480b-0709-48de-9e24-d1d2dc8c59ef None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:14.671014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0480b-0709-48de-9e24-d1d2dc8c59ef None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:14.671948 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0480b-0709-48de-9e24-d1d2dc8c59ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:14.672338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0480b-0709-48de-9e24-d1d2dc8c59ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:14.673886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0480b-0709-48de-9e24-d1d2dc8c59ef None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:14.683422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 114/221] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:14.689306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11f02a-aaa3-48eb-bfd4-51ff4fbe42f4 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:14.689667 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11f02a-aaa3-48eb-bfd4-51ff4fbe42f4 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:14.690011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11f02a-aaa3-48eb-bfd4-51ff4fbe42f4 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:14.692534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 115/222] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:14.693437 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11f02a-aaa3-48eb-bfd4-51ff4fbe42f4 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:14.713704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:14.751426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 108/223] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:14.753008 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82406a8-3f03-4fd6-8939-4efc9e9be939 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:14.753498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82406a8-3f03-4fd6-8939-4efc9e9be939 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:14.754050 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82406a8-3f03-4fd6-8939-4efc9e9be939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:14.775393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-f82406a8-3f03-4fd6-8939-4efc9e9be939 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:14.777238 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-f82406a8-3f03-4fd6-8939-4efc9e9be939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:14.814078 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11f02a-aaa3-48eb-bfd4-51ff4fbe42f4 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:14.854739 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5256296-0997-4e64-818c-38b9e9c2bd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:14.856212 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5256296-0997-4e64-818c-38b9e9c2bd05 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:14.856656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5256296-0997-4e64-818c-38b9e9c2bd05 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:14.857119 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5256296-0997-4e64-818c-38b9e9c2bd05 None admin] PATH_INFO: `/v3/users/b417bc81fb754260ba335d2c48fa86a7` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:14.858801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5256296-0997-4e64-818c-38b9e9c2bd05 None admin] RBAC: Authorizing `identity:get_user(user_id=b417bc81fb754260ba335d2c48fa86a7)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:14.860551 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5256296-0997-4e64-818c-38b9e9c2bd05 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:14.863045 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 116/224] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:14 2022] GET /identity/v3/users/b417bc81fb754260ba335d2c48fa86a7 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:14.863994 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5256296-0997-4e64-818c-38b9e9c2bd05 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:14.871524 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 109/225] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:14.872429 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82406a8-3f03-4fd6-8939-4efc9e9be939 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:14.940737 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:14.961809 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:14.979765 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5256296-0997-4e64-818c-38b9e9c2bd05 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:14.989935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82406a8-3f03-4fd6-8939-4efc9e9be939 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:15.014352 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5daa2-9734-4971-a557-5d42f6d1440a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:15.015224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5daa2-9734-4971-a557-5d42f6d1440a None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:15.015504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5daa2-9734-4971-a557-5d42f6d1440a None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:15.015940 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5daa2-9734-4971-a557-5d42f6d1440a None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:15.016966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5daa2-9734-4971-a557-5d42f6d1440a None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:15.018418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5daa2-9734-4971-a557-5d42f6d1440a None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:15.020275 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 117/226] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:14 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:15.021044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5daa2-9734-4971-a557-5d42f6d1440a None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:15.027778 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711f1ed-86b2-48da-9438-c08c7ad5147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:15.028925 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711f1ed-86b2-48da-9438-c08c7ad5147c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:15.029401 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711f1ed-86b2-48da-9438-c08c7ad5147c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:15.029857 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711f1ed-86b2-48da-9438-c08c7ad5147c None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:15.031234 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711f1ed-86b2-48da-9438-c08c7ad5147c None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:15.032937 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711f1ed-86b2-48da-9438-c08c7ad5147c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:15.034873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 110/227] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:14 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:15.036108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711f1ed-86b2-48da-9438-c08c7ad5147c None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:15.124606 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5daa2-9734-4971-a557-5d42f6d1440a None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:15.143534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711f1ed-86b2-48da-9438-c08c7ad5147c None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:15.160587 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53f98f-a446-4be2-9cc6-4950c6cfbb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:15.161403 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53f98f-a446-4be2-9cc6-4950c6cfbb0e None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:15.161683 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53f98f-a446-4be2-9cc6-4950c6cfbb0e None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:15.161951 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53f98f-a446-4be2-9cc6-4950c6cfbb0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:15.169092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:15.173071 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53f98f-a446-4be2-9cc6-4950c6cfbb0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:15.182304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2c1cc-3424-4fa3-9e81-643def939cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-5c53f98f-a446-4be2-9cc6-4950c6cfbb0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:15.182479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:15.183282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:15.184081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:15.184081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:15.184081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:15.184081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:15.184081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:15.184081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:15.184081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:15.184081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:15.184081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:15.184081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:15.184081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:15.184081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:15.184081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 118/228] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:36:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:15.184710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53f98f-a446-4be2-9cc6-4950c6cfbb0e None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:15.185931 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2c1cc-3424-4fa3-9e81-643def939cc1 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:15.186416 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2c1cc-3424-4fa3-9e81-643def939cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:15.186834 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2c1cc-3424-4fa3-9e81-643def939cc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:15.187867 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2c1cc-3424-4fa3-9e81-643def939cc1 None admin] RBAC: Adding query filter params (role.id=fe332203b7e04d6084e26c5be292af35, scope.project.id=7c06d53d4c434698958a453f9500feaa, user.id=3cce36ca0d794853b17a35ca0b7494d8) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:15.188358 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2c1cc-3424-4fa3-9e81-643def939cc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:15.190154 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2c1cc-3424-4fa3-9e81-643def939cc1 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:15.196773 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 111/229] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:15 2022] GET /identity/v3/role_assignments?user.id=3cce36ca0d794853b17a35ca0b7494d8&scope.project.id=7c06d53d4c434698958a453f9500feaa&role.id=fe332203b7e04d6084e26c5be292af35 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:15.197924 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2c1cc-3424-4fa3-9e81-643def939cc1 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:15.207039 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:15.305366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53f98f-a446-4be2-9cc6-4950c6cfbb0e None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:15.307563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2c1cc-3424-4fa3-9e81-643def939cc1 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:15.343434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c146f1-4a90-485d-8705-2b09c2b71674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:15.344640 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c146f1-4a90-485d-8705-2b09c2b71674 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:15.345092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c146f1-4a90-485d-8705-2b09c2b71674 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:15.345544 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c146f1-4a90-485d-8705-2b09c2b71674 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:15.345691 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922d3ae-20c7-4ecb-bcf2-69ad0a47c30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:15.346670 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c146f1-4a90-485d-8705-2b09c2b71674 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:15.346771 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922d3ae-20c7-4ecb-bcf2-69ad0a47c30c None admin] REQUEST_METHOD: `PUT` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:15.347129 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922d3ae-20c7-4ecb-bcf2-69ad0a47c30c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:15.347420 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c146f1-4a90-485d-8705-2b09c2b71674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:15.347546 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922d3ae-20c7-4ecb-bcf2-69ad0a47c30c None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa/users/b417bc81fb754260ba335d2c48fa86a7/roles/a33caf2301fb44db90304eb3d9afc3a5` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:15.349399 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c146f1-4a90-485d-8705-2b09c2b71674 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:15.349721 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922d3ae-20c7-4ecb-bcf2-69ad0a47c30c None admin] RBAC: Authorizing `identity:create_grant(project_id=7c06d53d4c434698958a453f9500feaa, user_id=b417bc81fb754260ba335d2c48fa86a7, role_id=a33caf2301fb44db90304eb3d9afc3a5)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:15.351438 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922d3ae-20c7-4ecb-bcf2-69ad0a47c30c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:15.360451 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 112/230] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:15.366764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c146f1-4a90-485d-8705-2b09c2b71674 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:15.386391 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 119/231] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:36:15 2022] PUT /identity/v3/projects/7c06d53d4c434698958a453f9500feaa/users/b417bc81fb754260ba335d2c48fa86a7/roles/a33caf2301fb44db90304eb3d9afc3a5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:36:15.442653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:15.494109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c146f1-4a90-485d-8705-2b09c2b71674 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:15.531652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001899d-2e72-45f7-bf50-db969d8d0e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:15.532576 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001899d-2e72-45f7-bf50-db969d8d0e6c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:15.532859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001899d-2e72-45f7-bf50-db969d8d0e6c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:15.533134 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001899d-2e72-45f7-bf50-db969d8d0e6c None admin] PATH_INFO: `/v3/users/b417bc81fb754260ba335d2c48fa86a7` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:15.534465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001899d-2e72-45f7-bf50-db969d8d0e6c None admin] RBAC: Authorizing `identity:get_user(user_id=b417bc81fb754260ba335d2c48fa86a7)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:15.536140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001899d-2e72-45f7-bf50-db969d8d0e6c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:15.537902 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 113/232] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:15 2022] GET /identity/v3/users/b417bc81fb754260ba335d2c48fa86a7 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:15.543734 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922d3ae-20c7-4ecb-bcf2-69ad0a47c30c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:15.638342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:15.673314 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922d3ae-20c7-4ecb-bcf2-69ad0a47c30c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:15.711103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeda497-0fd0-4a2a-b808-defbf7bbd2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:15.712133 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeda497-0fd0-4a2a-b808-defbf7bbd2b9 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:15.712502 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeda497-0fd0-4a2a-b808-defbf7bbd2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:15.712866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeda497-0fd0-4a2a-b808-defbf7bbd2b9 None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:15.714146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeda497-0fd0-4a2a-b808-defbf7bbd2b9 None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:15.715737 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeda497-0fd0-4a2a-b808-defbf7bbd2b9 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:15.717542 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 120/233] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:15 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:15.723587 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001899d-2e72-45f7-bf50-db969d8d0e6c None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:15.827728 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001899d-2e72-45f7-bf50-db969d8d0e6c None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:15.844242 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:15.862891 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca47a6-e295-49f4-87a8-01d26422ad49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:15.863977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca47a6-e295-49f4-87a8-01d26422ad49 None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:15.864159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca47a6-e295-49f4-87a8-01d26422ad49 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:15.864427 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca47a6-e295-49f4-87a8-01d26422ad49 None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa/users/b417bc81fb754260ba335d2c48fa86a7/roles/4a241925b6cc4218b4f2af580b151877` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:15.866673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca47a6-e295-49f4-87a8-01d26422ad49 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c06d53d4c434698958a453f9500feaa, user_id=b417bc81fb754260ba335d2c48fa86a7, role_id=4a241925b6cc4218b4f2af580b151877)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:15.868126 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca47a6-e295-49f4-87a8-01d26422ad49 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:15.885066 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 114/234] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:36:15 2022] PUT /identity/v3/projects/7c06d53d4c434698958a453f9500feaa/users/b417bc81fb754260ba335d2c48fa86a7/roles/4a241925b6cc4218b4f2af580b151877 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:36:16.862477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757d054-aa01-45ca-90f3-32614dc0152b None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:16.862983 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757d054-aa01-45ca-90f3-32614dc0152b None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:16.863161 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757d054-aa01-45ca-90f3-32614dc0152b None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:16.864753 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 121/235] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:16.870142 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae488d-a39f-44f3-81b1-6e05faff7579 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:16.870575 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae488d-a39f-44f3-81b1-6e05faff7579 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:16.870740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae488d-a39f-44f3-81b1-6e05faff7579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:16.899315 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-03ae488d-a39f-44f3-81b1-6e05faff7579 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:16.900850 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-03ae488d-a39f-44f3-81b1-6e05faff7579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:16.952267 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e32af-4cdf-4d49-be03-8975d4012377 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:16.952575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e32af-4cdf-4d49-be03-8975d4012377 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:16.952879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e32af-4cdf-4d49-be03-8975d4012377 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:16.954613 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 122/236] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:16.967806 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5650470-78f5-45bb-b68a-aed9941cab52 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:16.968098 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5650470-78f5-45bb-b68a-aed9941cab52 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:16.968357 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5650470-78f5-45bb-b68a-aed9941cab52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:17.002971 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-a5650470-78f5-45bb-b68a-aed9941cab52 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:17.004215 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-a5650470-78f5-45bb-b68a-aed9941cab52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:17.014510 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 115/237] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:17.044220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae488d-a39f-44f3-81b1-6e05faff7579 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:17.124639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 123/238] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:17.171796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5650470-78f5-45bb-b68a-aed9941cab52 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:17.175374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae488d-a39f-44f3-81b1-6e05faff7579 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:17.186812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:17.229064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507847de-c945-4757-bfb3-cadec93fc6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:17.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507847de-c945-4757-bfb3-cadec93fc6ef None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:17.230722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507847de-c945-4757-bfb3-cadec93fc6ef None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:17.231234 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507847de-c945-4757-bfb3-cadec93fc6ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:17.246879 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507847de-c945-4757-bfb3-cadec93fc6ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-507847de-c945-4757-bfb3-cadec93fc6ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:17.254861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:17.255680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:17.256433 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:17.256433 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:17.256433 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:17.256433 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:17.256433 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:17.256433 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:17.256433 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:17.256433 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:17.256433 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:17.256433 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:17.256433 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:17.256433 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:17.256433 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 116/239] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:36:17.262974 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507847de-c945-4757-bfb3-cadec93fc6ef None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:17.308183 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:17.373051 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5650470-78f5-45bb-b68a-aed9941cab52 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:17.415544 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507847de-c945-4757-bfb3-cadec93fc6ef None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:17.450740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb62297-6d80-483a-8786-6346e2aaf7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:17.451419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb62297-6d80-483a-8786-6346e2aaf7d3 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:17.451696 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb62297-6d80-483a-8786-6346e2aaf7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:17.451964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb62297-6d80-483a-8786-6346e2aaf7d3 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:17.452713 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb62297-6d80-483a-8786-6346e2aaf7d3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:17.458284 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb62297-6d80-483a-8786-6346e2aaf7d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:17.458284 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb62297-6d80-483a-8786-6346e2aaf7d3 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:17.464756 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 117/240] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:17.470375 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb62297-6d80-483a-8786-6346e2aaf7d3 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:17.471411 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd74737-779d-4c6f-9683-37532016a622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:17.472544 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd74737-779d-4c6f-9683-37532016a622 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:17.473107 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd74737-779d-4c6f-9683-37532016a622 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:17.473723 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd74737-779d-4c6f-9683-37532016a622 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:17.475350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd74737-779d-4c6f-9683-37532016a622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:17.477111 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd74737-779d-4c6f-9683-37532016a622 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:17.479769 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 124/241] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:17.487302 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd74737-779d-4c6f-9683-37532016a622 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:17.502636 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:17.580093 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb62297-6d80-483a-8786-6346e2aaf7d3 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:17.631101 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d064f84-c219-4958-b008-19ecfc249905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:17.632290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d064f84-c219-4958-b008-19ecfc249905 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:17.632741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d064f84-c219-4958-b008-19ecfc249905 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:17.633210 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d064f84-c219-4958-b008-19ecfc249905 None admin] PATH_INFO: `/v3/users/2cd044daf056476e8ee8ffa58b8aaffe` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:17.648705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d064f84-c219-4958-b008-19ecfc249905 None admin] RBAC: Authorizing `identity:get_user(user_id=2cd044daf056476e8ee8ffa58b8aaffe)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:17.650510 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d064f84-c219-4958-b008-19ecfc249905 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:17.652884 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 118/242] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:17 2022] GET /identity/v3/users/2cd044daf056476e8ee8ffa58b8aaffe => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:17.658468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d064f84-c219-4958-b008-19ecfc249905 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:17.674291 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd74737-779d-4c6f-9683-37532016a622 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:17.737598 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92fc35-e6e8-4571-90c3-6c684ad16360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:17.738338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92fc35-e6e8-4571-90c3-6c684ad16360 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:17.738621 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92fc35-e6e8-4571-90c3-6c684ad16360 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:17.738899 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92fc35-e6e8-4571-90c3-6c684ad16360 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:17.739829 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92fc35-e6e8-4571-90c3-6c684ad16360 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:17.741074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92fc35-e6e8-4571-90c3-6c684ad16360 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:17.750245 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:17.751098 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:17.773926 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 125/243] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:36:17 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:36:17.775187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d064f84-c219-4958-b008-19ecfc249905 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:17.816422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a4980-39e8-47a9-84ea-a4f3bf2220da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:17.817464 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a4980-39e8-47a9-84ea-a4f3bf2220da None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:17.817949 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a4980-39e8-47a9-84ea-a4f3bf2220da None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:17.818461 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a4980-39e8-47a9-84ea-a4f3bf2220da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:17.899941 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a4980-39e8-47a9-84ea-a4f3bf2220da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:17.902211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a4980-39e8-47a9-84ea-a4f3bf2220da None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:17.996189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:18.016172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 119/244] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:18.022673 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92fc35-e6e8-4571-90c3-6c684ad16360 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:18.030993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdac64-4a28-4167-a64a-650d2b5e380a None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:18.031654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdac64-4a28-4167-a64a-650d2b5e380a None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:18.032182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdac64-4a28-4167-a64a-650d2b5e380a None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:18.033938 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 120/245] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:36:18.042300 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e99468-cfe6-4e02-9a3a-504b615fcbd6 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:18.042300 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e99468-cfe6-4e02-9a3a-504b615fcbd6 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:18.042300 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e99468-cfe6-4e02-9a3a-504b615fcbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:18.077163 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-c2e99468-cfe6-4e02-9a3a-504b615fcbd6 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:18.080175 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-c2e99468-cfe6-4e02-9a3a-504b615fcbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:18.170562 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 121/246] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:18.177168 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92fc35-e6e8-4571-90c3-6c684ad16360 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:18.200365 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e99468-cfe6-4e02-9a3a-504b615fcbd6 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:18.224724 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b1684-b295-4b5d-855a-e76f50f98b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:18.225657 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b1684-b295-4b5d-855a-e76f50f98b21 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:18.225935 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b1684-b295-4b5d-855a-e76f50f98b21 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:18.226202 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b1684-b295-4b5d-855a-e76f50f98b21 None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:18.228904 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b1684-b295-4b5d-855a-e76f50f98b21 None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:18.230939 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b1684-b295-4b5d-855a-e76f50f98b21 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:18.233465 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 126/247] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:18 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:18.234645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83210c3-df26-442e-9531-f3a59fed213b None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:18.234922 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83210c3-df26-442e-9531-f3a59fed213b None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:18.235187 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83210c3-df26-442e-9531-f3a59fed213b None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:18.236931 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 127/248] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:18.242724 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a229536-e96c-4cfe-8383-e784557dc5a7 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:18.242999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a229536-e96c-4cfe-8383-e784557dc5a7 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:18.243270 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a229536-e96c-4cfe-8383-e784557dc5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:18.265616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-9a229536-e96c-4cfe-8383-e784557dc5a7 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:18.267013 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-9a229536-e96c-4cfe-8383-e784557dc5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:18.316286 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e99468-cfe6-4e02-9a3a-504b615fcbd6 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:18.330941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:18.338156 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:18.350343 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 128/249] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:18.351819 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a229536-e96c-4cfe-8383-e784557dc5a7 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:18.361663 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306c73f-9ef8-47e5-974b-8293a25d2219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:18.362481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306c73f-9ef8-47e5-974b-8293a25d2219 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:18.362764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306c73f-9ef8-47e5-974b-8293a25d2219 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:18.363161 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306c73f-9ef8-47e5-974b-8293a25d2219 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:18.376237 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9306c73f-9ef8-47e5-974b-8293a25d2219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-9306c73f-9ef8-47e5-974b-8293a25d2219 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:18.384620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:18.385511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:18.386424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:18.386424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:18.386424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:18.386424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:18.386424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:18.386424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:18.386424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:18.386424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:18.386424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:18.386424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:18.386424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:18.386424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:18.386424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 122/250] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:36:18.387046 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306c73f-9ef8-47e5-974b-8293a25d2219 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:18.488509 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a229536-e96c-4cfe-8383-e784557dc5a7 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:18.509620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306c73f-9ef8-47e5-974b-8293a25d2219 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:18.527173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0f132-b358-4ecc-8535-ba7c519fb6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:18.527905 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0f132-b358-4ecc-8535-ba7c519fb6b5 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:18.528275 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0f132-b358-4ecc-8535-ba7c519fb6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:18.528640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0f132-b358-4ecc-8535-ba7c519fb6b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:18.529602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0f132-b358-4ecc-8535-ba7c519fb6b5 None admin] RBAC: Adding query filter params (role.id=6ab02cf5cb1b4d3595ea8ea883ff6f2d, scope.project.id=7c06d53d4c434698958a453f9500feaa, user.id=2cd044daf056476e8ee8ffa58b8aaffe) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:18.530104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0f132-b358-4ecc-8535-ba7c519fb6b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:18.531777 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0f132-b358-4ecc-8535-ba7c519fb6b5 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:18.538831 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 129/251] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:18 2022] GET /identity/v3/role_assignments?user.id=2cd044daf056476e8ee8ffa58b8aaffe&scope.project.id=7c06d53d4c434698958a453f9500feaa&role.id=6ab02cf5cb1b4d3595ea8ea883ff6f2d => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:18.539769 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0f132-b358-4ecc-8535-ba7c519fb6b5 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:18.556187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f36222-d11f-485f-b4fe-275b73a1c338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:18.557259 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f36222-d11f-485f-b4fe-275b73a1c338 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:18.557683 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f36222-d11f-485f-b4fe-275b73a1c338 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:18.558089 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f36222-d11f-485f-b4fe-275b73a1c338 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:18.582984 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f36222-d11f-485f-b4fe-275b73a1c338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-98f36222-d11f-485f-b4fe-275b73a1c338 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:18.597462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:18.598556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:18.599496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:18.599496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:18.599496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:18.599496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:18.599496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:18.599496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:18.599496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:18.599496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:18.599496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:18.599496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:18.599496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:18.599496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:18.603550 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 123/252] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:36:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:18.612033 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f36222-d11f-485f-b4fe-275b73a1c338 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:18.667175 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:18.700035 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:18.780361 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0f132-b358-4ecc-8535-ba7c519fb6b5 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:18.817374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de52f84-8cf2-463b-99f5-690962b575bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:18.818021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de52f84-8cf2-463b-99f5-690962b575bd None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:18.818320 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de52f84-8cf2-463b-99f5-690962b575bd None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:18.818601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de52f84-8cf2-463b-99f5-690962b575bd None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:18.819815 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de52f84-8cf2-463b-99f5-690962b575bd None admin] RBAC: Adding query filter params (name=member) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:18.819815 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de52f84-8cf2-463b-99f5-690962b575bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:18.821174 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de52f84-8cf2-463b-99f5-690962b575bd None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:18.830683 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 130/253] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:18.839129 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de52f84-8cf2-463b-99f5-690962b575bd None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:18.890834 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f36222-d11f-485f-b4fe-275b73a1c338 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:18.936894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053ae32-78de-4804-a5a2-022971b4e4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:18.937672 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053ae32-78de-4804-a5a2-022971b4e4db None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:18.937956 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053ae32-78de-4804-a5a2-022971b4e4db None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:18.938229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053ae32-78de-4804-a5a2-022971b4e4db None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:18.939022 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a053ae32-78de-4804-a5a2-022971b4e4db None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:18.939328 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a053ae32-78de-4804-a5a2-022971b4e4db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:18.940799 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a053ae32-78de-4804-a5a2-022971b4e4db None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:18.949787 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 124/254] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:18.951138 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374bac5-8898-4ab3-bab2-4c946ba96936 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:18.951528 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374bac5-8898-4ab3-bab2-4c946ba96936 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:18.951799 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374bac5-8898-4ab3-bab2-4c946ba96936 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:18.957365 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 125/255] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:18.959140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8063e-e82a-4efc-9f4d-e1a49fdc8e99 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:18.959495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8063e-e82a-4efc-9f4d-e1a49fdc8e99 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:18.959772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8063e-e82a-4efc-9f4d-e1a49fdc8e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:18.979383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de52f84-8cf2-463b-99f5-690962b575bd None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:19.002617 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:19.004278 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-a4b8063e-e82a-4efc-9f4d-e1a49fdc8e99 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:19.005478 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-a4b8063e-e82a-4efc-9f4d-e1a49fdc8e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:19.018304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b84fc5-354b-43f6-8577-1b9a2175387c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:19.018951 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b84fc5-354b-43f6-8577-1b9a2175387c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:19.019275 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b84fc5-354b-43f6-8577-1b9a2175387c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:19.019522 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b84fc5-354b-43f6-8577-1b9a2175387c None admin] PATH_INFO: `/v3/users/3cce36ca0d794853b17a35ca0b7494d8` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:19.022378 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b84fc5-354b-43f6-8577-1b9a2175387c None admin] RBAC: Authorizing `identity:get_user(user_id=3cce36ca0d794853b17a35ca0b7494d8)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:19.022378 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b84fc5-354b-43f6-8577-1b9a2175387c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:19.028809 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 131/256] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:18 2022] GET /identity/v3/users/3cce36ca0d794853b17a35ca0b7494d8 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:19.029725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b84fc5-354b-43f6-8577-1b9a2175387c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:19.186452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b84fc5-354b-43f6-8577-1b9a2175387c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:19.233123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259688a2-2fb1-41eb-84fc-6733c6fb46d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:19.234014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259688a2-2fb1-41eb-84fc-6733c6fb46d4 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:19.234309 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259688a2-2fb1-41eb-84fc-6733c6fb46d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:19.234584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259688a2-2fb1-41eb-84fc-6733c6fb46d4 None admin] PATH_INFO: `/v3/users/b417bc81fb754260ba335d2c48fa86a7` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:19.235779 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259688a2-2fb1-41eb-84fc-6733c6fb46d4 None admin] RBAC: Authorizing `identity:get_user(user_id=b417bc81fb754260ba335d2c48fa86a7)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:19.237257 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259688a2-2fb1-41eb-84fc-6733c6fb46d4 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:19.239560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 132/257] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:19 2022] GET /identity/v3/users/b417bc81fb754260ba335d2c48fa86a7 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:19.244745 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259688a2-2fb1-41eb-84fc-6733c6fb46d4 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:19.257065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 126/258] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:19.257766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8063e-e82a-4efc-9f4d-e1a49fdc8e99 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:19.306783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:19.342755 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:19.370147 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259688a2-2fb1-41eb-84fc-6733c6fb46d4 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:19.404912 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93f11d-c94c-4f82-85c1-546713ab1bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:19.405789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93f11d-c94c-4f82-85c1-546713ab1bf3 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:19.406072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93f11d-c94c-4f82-85c1-546713ab1bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:19.406369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93f11d-c94c-4f82-85c1-546713ab1bf3 None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:19.407561 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be93f11d-c94c-4f82-85c1-546713ab1bf3 None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:19.409041 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be93f11d-c94c-4f82-85c1-546713ab1bf3 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:19.411410 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 133/259] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:19 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:19.412208 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93f11d-c94c-4f82-85c1-546713ab1bf3 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:19.498115 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8063e-e82a-4efc-9f4d-e1a49fdc8e99 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:19.566669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3366a1-eab5-443f-ad11-73d707636978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:19.567339 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3366a1-eab5-443f-ad11-73d707636978 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:19.567623 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3366a1-eab5-443f-ad11-73d707636978 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:19.567903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3366a1-eab5-443f-ad11-73d707636978 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:19.610574 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93f11d-c94c-4f82-85c1-546713ab1bf3 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:19.686489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:19.723202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3366a1-eab5-443f-ad11-73d707636978 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:19.724696 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3366a1-eab5-443f-ad11-73d707636978 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:19.730472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519973b-ba0c-4b4a-80dd-1bebcd9075aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:19.731299 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519973b-ba0c-4b4a-80dd-1bebcd9075aa None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:19.731579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519973b-ba0c-4b4a-80dd-1bebcd9075aa None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:19.731849 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519973b-ba0c-4b4a-80dd-1bebcd9075aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:19.766065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519973b-ba0c-4b4a-80dd-1bebcd9075aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:19.791124 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-c519973b-ba0c-4b4a-80dd-1bebcd9075aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:19.802619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:19.803480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:19.804705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:19.804705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:19.804705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:19.804705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:19.804705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:19.804705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:19.804705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:19.804705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:19.804705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:19.804705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:19.804705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:19.804705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:19.804705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 134/260] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:36:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:19.805335 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519973b-ba0c-4b4a-80dd-1bebcd9075aa None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:19.953712 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 127/261] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:19.954422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3366a1-eab5-443f-ad11-73d707636978 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:20.029890 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519973b-ba0c-4b4a-80dd-1bebcd9075aa None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:20.101426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a001d-a440-47f2-ad13-967af525ee76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:20.102670 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a001d-a440-47f2-ad13-967af525ee76 None admin] REQUEST_METHOD: `PUT` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:20.102948 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a001d-a440-47f2-ad13-967af525ee76 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:20.103205 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a001d-a440-47f2-ad13-967af525ee76 None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa/users/3cce36ca0d794853b17a35ca0b7494d8/roles/fe332203b7e04d6084e26c5be292af35` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:20.106480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a001d-a440-47f2-ad13-967af525ee76 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c06d53d4c434698958a453f9500feaa, user_id=3cce36ca0d794853b17a35ca0b7494d8, role_id=fe332203b7e04d6084e26c5be292af35)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:20.107959 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a001d-a440-47f2-ad13-967af525ee76 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:20.132270 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 135/262] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:36:19 2022] PUT /identity/v3/projects/7c06d53d4c434698958a453f9500feaa/users/3cce36ca0d794853b17a35ca0b7494d8/roles/fe332203b7e04d6084e26c5be292af35 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:36:20.133970 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a001d-a440-47f2-ad13-967af525ee76 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:20.152182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3366a1-eab5-443f-ad11-73d707636978 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:20.205773 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:20.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0782f-aee4-4edc-bf3d-ea9074d9ed8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:20.224582 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0782f-aee4-4edc-bf3d-ea9074d9ed8a None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:20.224702 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:20.224780 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0782f-aee4-4edc-bf3d-ea9074d9ed8a None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:20.224847 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0782f-aee4-4edc-bf3d-ea9074d9ed8a None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:20.225588 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0782f-aee4-4edc-bf3d-ea9074d9ed8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:20.226163 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0782f-aee4-4edc-bf3d-ea9074d9ed8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:20.229536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0782f-aee4-4edc-bf3d-ea9074d9ed8a None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:20.245953 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 128/263] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:20.254801 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0782f-aee4-4edc-bf3d-ea9074d9ed8a None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:20.330024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a001d-a440-47f2-ad13-967af525ee76 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:20.393463 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4129543-6b14-41a6-aaef-2b2710ffc0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:20.394447 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4129543-6b14-41a6-aaef-2b2710ffc0cf None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:20.399826 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4129543-6b14-41a6-aaef-2b2710ffc0cf None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:20.400214 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4129543-6b14-41a6-aaef-2b2710ffc0cf None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:20.403251 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4129543-6b14-41a6-aaef-2b2710ffc0cf None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:20.404752 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4129543-6b14-41a6-aaef-2b2710ffc0cf None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:20.407219 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 136/264] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:20 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:20.412570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4129543-6b14-41a6-aaef-2b2710ffc0cf None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:20.453782 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0782f-aee4-4edc-bf3d-ea9074d9ed8a None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:20.535112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca87d8-d309-4f4a-9675-89bcc6f0b4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:20.536023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca87d8-d309-4f4a-9675-89bcc6f0b4b6 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:20.536303 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca87d8-d309-4f4a-9675-89bcc6f0b4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:20.536577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca87d8-d309-4f4a-9675-89bcc6f0b4b6 None admin] PATH_INFO: `/v3/users/b417bc81fb754260ba335d2c48fa86a7` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:20.537858 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca87d8-d309-4f4a-9675-89bcc6f0b4b6 None admin] RBAC: Authorizing `identity:get_user(user_id=b417bc81fb754260ba335d2c48fa86a7)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:20.539358 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca87d8-d309-4f4a-9675-89bcc6f0b4b6 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:20.541266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 129/265] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:20 2022] GET /identity/v3/users/b417bc81fb754260ba335d2c48fa86a7 => generated 283 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:20.547052 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca87d8-d309-4f4a-9675-89bcc6f0b4b6 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:20.573057 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4129543-6b14-41a6-aaef-2b2710ffc0cf None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:20.589729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:20.617111 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a74f6-0481-41ef-a192-3f104d14a4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:20.617841 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a74f6-0481-41ef-a192-3f104d14a4bc None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:20.618133 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a74f6-0481-41ef-a192-3f104d14a4bc None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:20.618462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a74f6-0481-41ef-a192-3f104d14a4bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:20.619323 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a74f6-0481-41ef-a192-3f104d14a4bc None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=7c06d53d4c434698958a453f9500feaa, user.id=b417bc81fb754260ba335d2c48fa86a7) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:20.619641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a74f6-0481-41ef-a192-3f104d14a4bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:20.621369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a74f6-0481-41ef-a192-3f104d14a4bc None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:20.629953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 137/266] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:20 2022] GET /identity/v3/role_assignments?user.id=b417bc81fb754260ba335d2c48fa86a7&scope.project.id=7c06d53d4c434698958a453f9500feaa&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:20.679054 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:20.727524 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca87d8-d309-4f4a-9675-89bcc6f0b4b6 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:20.771048 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479f386-9560-4214-8b60-33686ccda401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:20.771724 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479f386-9560-4214-8b60-33686ccda401 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:20.772004 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479f386-9560-4214-8b60-33686ccda401 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:20.772279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479f386-9560-4214-8b60-33686ccda401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:20.864360 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcea06d-a637-4b03-b639-d0c589fab7a7 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:20.864680 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcea06d-a637-4b03-b639-d0c589fab7a7 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:20.864841 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcea06d-a637-4b03-b639-d0c589fab7a7 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:20.865091 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479f386-9560-4214-8b60-33686ccda401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:20.866248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 138/267] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:20.866620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479f386-9560-4214-8b60-33686ccda401 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:20.876098 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0688778-1ba2-4b58-99a3-75a5f2a1e996 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:20.876399 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0688778-1ba2-4b58-99a3-75a5f2a1e996 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:20.876657 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0688778-1ba2-4b58-99a3-75a5f2a1e996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:20.919322 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-f0688778-1ba2-4b58-99a3-75a5f2a1e996 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:20.920529 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-f0688778-1ba2-4b58-99a3-75a5f2a1e996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:20.936308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:20.983088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 130/268] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:20.991312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479f386-9560-4214-8b60-33686ccda401 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:21.053023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 139/269] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:21.088433 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0688778-1ba2-4b58-99a3-75a5f2a1e996 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:21.108624 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479f386-9560-4214-8b60-33686ccda401 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:21.149160 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b882ee-71d6-41e1-9b69-d98829eb02c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:21.149965 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b882ee-71d6-41e1-9b69-d98829eb02c4 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:21.150249 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b882ee-71d6-41e1-9b69-d98829eb02c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:21.150583 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b882ee-71d6-41e1-9b69-d98829eb02c4 None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:21.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b882ee-71d6-41e1-9b69-d98829eb02c4 None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:21.153392 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b882ee-71d6-41e1-9b69-d98829eb02c4 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:21.155552 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 131/270] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:20 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:21.161493 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b882ee-71d6-41e1-9b69-d98829eb02c4 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:21.234619 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:21.312233 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0688778-1ba2-4b58-99a3-75a5f2a1e996 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:21.315281 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:21.336151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b882ee-71d6-41e1-9b69-d98829eb02c4 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:21.358543 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a053791-426e-4b20-80d7-22f444c988ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:21.360227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a053791-426e-4b20-80d7-22f444c988ea None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:21.360700 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a053791-426e-4b20-80d7-22f444c988ea None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:21.361419 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a053791-426e-4b20-80d7-22f444c988ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:21.373933 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366698ed-a684-4446-8174-78d45225f58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:21.374619 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366698ed-a684-4446-8174-78d45225f58d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:21.374894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366698ed-a684-4446-8174-78d45225f58d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:21.375158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366698ed-a684-4446-8174-78d45225f58d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:21.375568 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a053791-426e-4b20-80d7-22f444c988ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:21.375974 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366698ed-a684-4446-8174-78d45225f58d None admin] RBAC: Adding query filter params (role.id=4a241925b6cc4218b4f2af580b151877, scope.project.id=7c06d53d4c434698958a453f9500feaa, user.id=b417bc81fb754260ba335d2c48fa86a7) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:21.376276 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366698ed-a684-4446-8174-78d45225f58d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:21.377724 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366698ed-a684-4446-8174-78d45225f58d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:21.384645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 132/271] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:21 2022] GET /identity/v3/role_assignments?user.id=b417bc81fb754260ba335d2c48fa86a7&scope.project.id=7c06d53d4c434698958a453f9500feaa&role.id=4a241925b6cc4218b4f2af580b151877 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:21.388198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86b4bc-1099-445e-952a-9db15a115aca None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:21.388198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86b4bc-1099-445e-952a-9db15a115aca None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:21.388198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86b4bc-1099-445e-952a-9db15a115aca None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:21.388198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 133/272] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-7a053791-426e-4b20-80d7-22f444c988ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:21.389358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:21.390199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:21.391024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:21.391024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:21.391024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:21.391024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:21.391024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:21.391024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:21.391024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:21.391024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:21.391024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:21.391024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:21.391024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:21.391024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:21.396653 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 140/273] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:21.403911 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86b4bc-1099-445e-952a-9db15a115aca None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:21.404692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc4085-9f63-4937-8862-ef8961d292e2 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:21.405172 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc4085-9f63-4937-8862-ef8961d292e2 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:21.405636 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc4085-9f63-4937-8862-ef8961d292e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:21.425773 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-cbfc4085-9f63-4937-8862-ef8961d292e2 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:21.427441 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-cbfc4085-9f63-4937-8862-ef8961d292e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:21.511210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 141/274] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:21.540659 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc4085-9f63-4937-8862-ef8961d292e2 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:21.609176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:21.643204 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86b4bc-1099-445e-952a-9db15a115aca None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:21.652861 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:21.661218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc4085-9f63-4937-8862-ef8961d292e2 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:21.704053 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bd9b4-4c98-464c-9629-c57b9cf4d63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:21.705577 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bd9b4-4c98-464c-9629-c57b9cf4d63a None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:21.705867 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bd9b4-4c98-464c-9629-c57b9cf4d63a None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:21.706146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bd9b4-4c98-464c-9629-c57b9cf4d63a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:21.708800 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08569b-3557-4f6c-a3a9-c25031e5b86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:21.710224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08569b-3557-4f6c-a3a9-c25031e5b86d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:21.710724 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08569b-3557-4f6c-a3a9-c25031e5b86d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:21.711166 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08569b-3557-4f6c-a3a9-c25031e5b86d None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:21.712112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08569b-3557-4f6c-a3a9-c25031e5b86d None admin] RBAC: Adding query filter params (name=member) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:21.712805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08569b-3557-4f6c-a3a9-c25031e5b86d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:21.714547 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08569b-3557-4f6c-a3a9-c25031e5b86d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:21.719112 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bd9b4-4c98-464c-9629-c57b9cf4d63a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:21.725580 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 134/275] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:21.727752 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f5c8f-c177-48bc-a71b-1e532c797c4f None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:21.728243 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f5c8f-c177-48bc-a71b-1e532c797c4f None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:21.728685 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f5c8f-c177-48bc-a71b-1e532c797c4f None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-501bd9b4-4c98-464c-9629-c57b9cf4d63a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:21.728785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:21.729570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:21.734797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:21.734797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:21.734797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:21.734797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:21.734797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:21.734797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:21.734797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:21.734797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:21.734797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:21.734797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:21.734797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:21.734797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:21.734797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 142/276] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:21.734797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bd9b4-4c98-464c-9629-c57b9cf4d63a None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:21.739949 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 135/277] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:21 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:21.741147 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f5c8f-c177-48bc-a71b-1e532c797c4f None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:21.859929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bd9b4-4c98-464c-9629-c57b9cf4d63a None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:21.878115 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f5c8f-c177-48bc-a71b-1e532c797c4f None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:21.897964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ff006-3371-438b-8a7d-45211574b097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:21.898882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ff006-3371-438b-8a7d-45211574b097 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:21.899429 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ff006-3371-438b-8a7d-45211574b097 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:21.899429 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ff006-3371-438b-8a7d-45211574b097 None admin] PATH_INFO: `/v3/users/e52b070e4a29400cb7e030d1bb9229d6` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:21.913623 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b2e2e-8413-4a9a-8d66-b60f1b2276bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:21.914217 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b2e2e-8413-4a9a-8d66-b60f1b2276bd None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:21.914506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ff006-3371-438b-8a7d-45211574b097 None admin] RBAC: Authorizing `identity:get_user(user_id=e52b070e4a29400cb7e030d1bb9229d6)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:21.914607 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b2e2e-8413-4a9a-8d66-b60f1b2276bd None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:21.914784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b2e2e-8413-4a9a-8d66-b60f1b2276bd None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:21.915530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b2e2e-8413-4a9a-8d66-b60f1b2276bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:21.915835 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b2e2e-8413-4a9a-8d66-b60f1b2276bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:21.916025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ff006-3371-438b-8a7d-45211574b097 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:21.917231 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b2e2e-8413-4a9a-8d66-b60f1b2276bd None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:21.917962 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 143/278] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:21 2022] GET /identity/v3/users/e52b070e4a29400cb7e030d1bb9229d6 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:21.922383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4b29a-ec17-422d-8011-7b4ef8b9ffda None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:21.922383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4b29a-ec17-422d-8011-7b4ef8b9ffda None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:21.922383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4b29a-ec17-422d-8011-7b4ef8b9ffda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:21.934376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 136/279] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:21.934376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b2e2e-8413-4a9a-8d66-b60f1b2276bd None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:21.940823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-89a4b29a-ec17-422d-8011-7b4ef8b9ffda None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:21.942276 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-89a4b29a-ec17-422d-8011-7b4ef8b9ffda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:21.950000 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:21.951641 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:22.018312 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 144/280] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:22.018312 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4b29a-ec17-422d-8011-7b4ef8b9ffda None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:22.044577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b2e2e-8413-4a9a-8d66-b60f1b2276bd None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:22.079079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcd329-1aca-488a-b1d3-a1d2c8eea97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:22.079735 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcd329-1aca-488a-b1d3-a1d2c8eea97b None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:22.080016 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcd329-1aca-488a-b1d3-a1d2c8eea97b None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:22.080289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcd329-1aca-488a-b1d3-a1d2c8eea97b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:22.081108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fcd329-1aca-488a-b1d3-a1d2c8eea97b None admin] RBAC: Adding query filter params (role.id=fe332203b7e04d6084e26c5be292af35, user.id=e52b070e4a29400cb7e030d1bb9229d6, scope.system=all) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:22.081422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fcd329-1aca-488a-b1d3-a1d2c8eea97b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:22.083117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fcd329-1aca-488a-b1d3-a1d2c8eea97b None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:22.088305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 137/281] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 19:36:21 2022] GET /identity/v3/role_assignments?user.id=e52b070e4a29400cb7e030d1bb9229d6&scope.system=all&role.id=fe332203b7e04d6084e26c5be292af35 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:22.091181 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcd329-1aca-488a-b1d3-a1d2c8eea97b None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:22.145361 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4b29a-ec17-422d-8011-7b4ef8b9ffda None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:22.193057 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c3d9a-9e35-458d-b434-223f88a15285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:22.194777 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c3d9a-9e35-458d-b434-223f88a15285 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:22.195317 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c3d9a-9e35-458d-b434-223f88a15285 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:22.195687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c3d9a-9e35-458d-b434-223f88a15285 None admin] PATH_INFO: `/v3/users/2cd044daf056476e8ee8ffa58b8aaffe` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:22.197067 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c3d9a-9e35-458d-b434-223f88a15285 None admin] RBAC: Authorizing `identity:get_user(user_id=2cd044daf056476e8ee8ffa58b8aaffe)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:22.199108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c3d9a-9e35-458d-b434-223f88a15285 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:22.199273 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:22.202308 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 145/282] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:22 2022] GET /identity/v3/users/2cd044daf056476e8ee8ffa58b8aaffe => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:22.210157 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c3d9a-9e35-458d-b434-223f88a15285 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:22.227014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:22.233638 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcd329-1aca-488a-b1d3-a1d2c8eea97b None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:22.279917 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14f103-abce-4dc7-b556-5bfbf01fee2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:22.280731 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14f103-abce-4dc7-b556-5bfbf01fee2d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:22.281012 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14f103-abce-4dc7-b556-5bfbf01fee2d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:22.281288 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14f103-abce-4dc7-b556-5bfbf01fee2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:22.282619 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14f103-abce-4dc7-b556-5bfbf01fee2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:22.284104 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14f103-abce-4dc7-b556-5bfbf01fee2d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:22.290345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 138/283] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:22.292181 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14f103-abce-4dc7-b556-5bfbf01fee2d None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:22.350564 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c3d9a-9e35-458d-b434-223f88a15285 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:22.393161 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50b165-7cc4-488d-957b-5a93cf37dc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:22.394741 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50b165-7cc4-488d-957b-5a93cf37dc05 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:22.395523 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50b165-7cc4-488d-957b-5a93cf37dc05 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:22.396003 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50b165-7cc4-488d-957b-5a93cf37dc05 None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:22.397768 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50b165-7cc4-488d-957b-5a93cf37dc05 None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:22.400024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50b165-7cc4-488d-957b-5a93cf37dc05 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:22.401173 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14f103-abce-4dc7-b556-5bfbf01fee2d None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:22.402548 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 146/284] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:22 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:22.409013 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50b165-7cc4-488d-957b-5a93cf37dc05 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:22.450395 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c33eee-81f3-4484-8894-5b26f401fdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:22.450988 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c33eee-81f3-4484-8894-5b26f401fdb2 None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:22.451269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c33eee-81f3-4484-8894-5b26f401fdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:22.451540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c33eee-81f3-4484-8894-5b26f401fdb2 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:22.452254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c33eee-81f3-4484-8894-5b26f401fdb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:22.454291 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c33eee-81f3-4484-8894-5b26f401fdb2 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:22.482407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 139/285] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:36:22 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:36:22.509482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:22.537373 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50b165-7cc4-488d-957b-5a93cf37dc05 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:22.576882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c566037-5895-43ff-921a-4ae6e2e5bbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:22.577788 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c566037-5895-43ff-921a-4ae6e2e5bbad None admin] REQUEST_METHOD: `PUT` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:22.578060 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c566037-5895-43ff-921a-4ae6e2e5bbad None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:22.578353 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c566037-5895-43ff-921a-4ae6e2e5bbad None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa/users/2cd044daf056476e8ee8ffa58b8aaffe/roles/6ab02cf5cb1b4d3595ea8ea883ff6f2d` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:22.580420 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c566037-5895-43ff-921a-4ae6e2e5bbad None admin] RBAC: Authorizing `identity:create_grant(project_id=7c06d53d4c434698958a453f9500feaa, user_id=2cd044daf056476e8ee8ffa58b8aaffe, role_id=6ab02cf5cb1b4d3595ea8ea883ff6f2d)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:22.582090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c566037-5895-43ff-921a-4ae6e2e5bbad None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:22.606487 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 147/286] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:36:22 2022] PUT /identity/v3/projects/7c06d53d4c434698958a453f9500feaa/users/2cd044daf056476e8ee8ffa58b8aaffe/roles/6ab02cf5cb1b4d3595ea8ea883ff6f2d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:36:23.024952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dc24a-2620-465c-8f25-2011d6e099ab None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:23.025291 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dc24a-2620-465c-8f25-2011d6e099ab None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:23.025510 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dc24a-2620-465c-8f25-2011d6e099ab None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:23.027099 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 140/287] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:23.034161 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f54b1-1f96-4d89-b6a9-bf2ae4f9ce8d None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:23.034506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f54b1-1f96-4d89-b6a9-bf2ae4f9ce8d None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:23.034866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f54b1-1f96-4d89-b6a9-bf2ae4f9ce8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:23.055055 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-364f54b1-1f96-4d89-b6a9-bf2ae4f9ce8d None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:23.056236 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-364f54b1-1f96-4d89-b6a9-bf2ae4f9ce8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:23.152943 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 148/288] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:23.196057 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dc24a-2620-465c-8f25-2011d6e099ab None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:23.199138 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:23.351230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dc24a-2620-465c-8f25-2011d6e099ab None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:23.387762 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfcc7a1-ca39-4ef3-9bdc-711a1c38a53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:23.388571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcc7a1-ca39-4ef3-9bdc-711a1c38a53a None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:23.388849 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcc7a1-ca39-4ef3-9bdc-711a1c38a53a None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:23.389119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcc7a1-ca39-4ef3-9bdc-711a1c38a53a None admin] PATH_INFO: `/v3/roles/member` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:23.400369 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfcc7a1-ca39-4ef3-9bdc-711a1c38a53a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-ebfcc7a1-ca39-4ef3-9bdc-711a1c38a53a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:23.408150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:23.408992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:23.411014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:23.411014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:23.411014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:23.411014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:23.411014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:23.411014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:23.411014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:23.411014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:23.411014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:23.411014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:23.411014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:23.411014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:23.411014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 141/289] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:23.415374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f54b1-1f96-4d89-b6a9-bf2ae4f9ce8d None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:23.483285 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:23.530012 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f54b1-1f96-4d89-b6a9-bf2ae4f9ce8d None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:23.583035 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eec230-e96d-4b4a-9354-c8210b8274cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:23.583710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eec230-e96d-4b4a-9354-c8210b8274cd None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:23.583987 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eec230-e96d-4b4a-9354-c8210b8274cd None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:23.584259 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eec230-e96d-4b4a-9354-c8210b8274cd None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:23.585012 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eec230-e96d-4b4a-9354-c8210b8274cd None admin] RBAC: Adding query filter params (name=member) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:23.585313 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eec230-e96d-4b4a-9354-c8210b8274cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:23.586743 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eec230-e96d-4b4a-9354-c8210b8274cd None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:23.595782 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 149/290] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:23.601367 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfcc7a1-ca39-4ef3-9bdc-711a1c38a53a None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:23.701095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:23.742149 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfcc7a1-ca39-4ef3-9bdc-711a1c38a53a None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:23.776064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e023f-f277-48a6-89f2-361eb6af712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:23.776949 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e023f-f277-48a6-89f2-361eb6af712a None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:23.777228 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e023f-f277-48a6-89f2-361eb6af712a None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:23.777501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e023f-f277-48a6-89f2-361eb6af712a None admin] PATH_INFO: `/v3/users/3cce36ca0d794853b17a35ca0b7494d8` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:23.778734 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e023f-f277-48a6-89f2-361eb6af712a None admin] RBAC: Authorizing `identity:get_user(user_id=3cce36ca0d794853b17a35ca0b7494d8)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:23.780194 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e023f-f277-48a6-89f2-361eb6af712a None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:23.782121 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 142/291] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:23 2022] GET /identity/v3/users/3cce36ca0d794853b17a35ca0b7494d8 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:23.788062 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eec230-e96d-4b4a-9354-c8210b8274cd None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:23.898512 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eec230-e96d-4b4a-9354-c8210b8274cd None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:23.934789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:23.942984 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8751631-d06f-409f-9c0d-01e024d16a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:23.943650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8751631-d06f-409f-9c0d-01e024d16a51 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:23.943961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8751631-d06f-409f-9c0d-01e024d16a51 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:23.944275 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8751631-d06f-409f-9c0d-01e024d16a51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:24.030965 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8751631-d06f-409f-9c0d-01e024d16a51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:24.032452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8751631-d06f-409f-9c0d-01e024d16a51 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:24.158656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 150/292] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:24.164106 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e023f-f277-48a6-89f2-361eb6af712a None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:24.293121 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e023f-f277-48a6-89f2-361eb6af712a None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:24.321164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:24.326938 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe94f75-6143-4370-b924-1b076ab4b748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:24.327806 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe94f75-6143-4370-b924-1b076ab4b748 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:24.328084 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe94f75-6143-4370-b924-1b076ab4b748 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:24.328349 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe94f75-6143-4370-b924-1b076ab4b748 None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:24.329509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe94f75-6143-4370-b924-1b076ab4b748 None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:24.331077 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe94f75-6143-4370-b924-1b076ab4b748 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:24.333655 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 143/293] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:24 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:24.339924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8751631-d06f-409f-9c0d-01e024d16a51 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:24.401490 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:24.496256 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8751631-d06f-409f-9c0d-01e024d16a51 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:24.537470 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b87e9-c487-478c-8539-776c6d479fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:24.538124 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b87e9-c487-478c-8539-776c6d479fed None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:24.538426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b87e9-c487-478c-8539-776c6d479fed None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:24.538690 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b87e9-c487-478c-8539-776c6d479fed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:24.539523 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b87e9-c487-478c-8539-776c6d479fed None admin] RBAC: Adding query filter params (role.id=fe332203b7e04d6084e26c5be292af35, scope.project.id=7c06d53d4c434698958a453f9500feaa, user.id=3cce36ca0d794853b17a35ca0b7494d8) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:24.539823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b87e9-c487-478c-8539-776c6d479fed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:24.541370 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b87e9-c487-478c-8539-776c6d479fed None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:24.547563 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 151/294] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:24 2022] GET /identity/v3/role_assignments?user.id=3cce36ca0d794853b17a35ca0b7494d8&scope.project.id=7c06d53d4c434698958a453f9500feaa&role.id=fe332203b7e04d6084e26c5be292af35 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:24.640734 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415193b2-d900-40ac-a2d3-a79486d662cc None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:24.641049 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415193b2-d900-40ac-a2d3-a79486d662cc None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:24.641288 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415193b2-d900-40ac-a2d3-a79486d662cc None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:24.643074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 144/295] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:24.653312 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfadd4-1698-4bbf-908a-320877a74377 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:24.653619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfadd4-1698-4bbf-908a-320877a74377 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:24.653851 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfadd4-1698-4bbf-908a-320877a74377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:24.678809 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-c5cfadd4-1698-4bbf-908a-320877a74377 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:24.680027 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-c5cfadd4-1698-4bbf-908a-320877a74377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:24.777159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 152/296] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:24.812158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415193b2-d900-40ac-a2d3-a79486d662cc None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:24.955818 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415193b2-d900-40ac-a2d3-a79486d662cc None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:24.995996 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372aa43-5758-496b-9269-959825d684ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:24.996846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372aa43-5758-496b-9269-959825d684ea None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:24.997129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372aa43-5758-496b-9269-959825d684ea None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:24.997400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372aa43-5758-496b-9269-959825d684ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:25.008486 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372aa43-5758-496b-9269-959825d684ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-f372aa43-5758-496b-9269-959825d684ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:25.016344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:25.017187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:25.017966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:25.017966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:25.017966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:25.017966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:25.017966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:25.017966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:25.017966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:25.017966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:25.017966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:25.017966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:25.017966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:25.017966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:25.017966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 145/297] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:25.027141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfadd4-1698-4bbf-908a-320877a74377 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:25.051789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:25.145126 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfadd4-1698-4bbf-908a-320877a74377 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:25.180670 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974205f-10f4-40e4-8540-5c52465c8fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:25.181420 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974205f-10f4-40e4-8540-5c52465c8fba None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:25.181615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974205f-10f4-40e4-8540-5c52465c8fba None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:25.181885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974205f-10f4-40e4-8540-5c52465c8fba None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:25.182674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974205f-10f4-40e4-8540-5c52465c8fba None admin] RBAC: Adding query filter params (name=member) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:25.182973 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974205f-10f4-40e4-8540-5c52465c8fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:25.184365 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974205f-10f4-40e4-8540-5c52465c8fba None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:25.193464 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 153/298] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:25.199401 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372aa43-5758-496b-9269-959825d684ea None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:25.264184 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:25.299448 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67751900-ed52-4c6a-8bf8-dc422cc67486 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:25.300083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67751900-ed52-4c6a-8bf8-dc422cc67486 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:25.300629 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67751900-ed52-4c6a-8bf8-dc422cc67486 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:25.302467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 154/299] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:36:25.303919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5d175-ebd9-43f5-9368-3c310335b88f None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:25.304388 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5d175-ebd9-43f5-9368-3c310335b88f None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:25.304859 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5d175-ebd9-43f5-9368-3c310335b88f None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:25.306443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 155/300] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:25.307780 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b8a36-a5be-4294-a70f-24765fa8a3bf None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:25.307883 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372aa43-5758-496b-9269-959825d684ea None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:25.308673 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b8a36-a5be-4294-a70f-24765fa8a3bf None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:25.309154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b8a36-a5be-4294-a70f-24765fa8a3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:25.325983 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-375b8a36-a5be-4294-a70f-24765fa8a3bf None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:25.327179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-375b8a36-a5be-4294-a70f-24765fa8a3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:25.344734 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab18b04-15d3-4c50-aa46-4960f0982e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:25.345620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab18b04-15d3-4c50-aa46-4960f0982e6c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:25.345896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab18b04-15d3-4c50-aa46-4960f0982e6c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:25.346164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab18b04-15d3-4c50-aa46-4960f0982e6c None admin] PATH_INFO: `/v3/users/e52b070e4a29400cb7e030d1bb9229d6` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:25.347536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab18b04-15d3-4c50-aa46-4960f0982e6c None admin] RBAC: Authorizing `identity:get_user(user_id=e52b070e4a29400cb7e030d1bb9229d6)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:25.348997 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab18b04-15d3-4c50-aa46-4960f0982e6c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:25.350839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 146/301] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:25 2022] GET /identity/v3/users/e52b070e4a29400cb7e030d1bb9229d6 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:25.356672 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22a6cb-85f1-4804-9121-3be6dcb0d424 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:25.356919 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22a6cb-85f1-4804-9121-3be6dcb0d424 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:25.357195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22a6cb-85f1-4804-9121-3be6dcb0d424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:25.374081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-0d22a6cb-85f1-4804-9121-3be6dcb0d424 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:25.375285 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-0d22a6cb-85f1-4804-9121-3be6dcb0d424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:25.398008 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 156/302] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:25.399128 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b8a36-a5be-4294-a70f-24765fa8a3bf None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:25.421372 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:25.448994 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 147/303] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:25.450184 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22a6cb-85f1-4804-9121-3be6dcb0d424 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:25.512113 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b8a36-a5be-4294-a70f-24765fa8a3bf None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:25.514975 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:25.550222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c54da-7614-4b29-9164-d1a4fc14f131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:25.551268 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c54da-7614-4b29-9164-d1a4fc14f131 None admin] REQUEST_METHOD: `PUT` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:25.551455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c54da-7614-4b29-9164-d1a4fc14f131 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:25.551726 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c54da-7614-4b29-9164-d1a4fc14f131 None admin] PATH_INFO: `/v3/system/users/e52b070e4a29400cb7e030d1bb9229d6/roles/fe332203b7e04d6084e26c5be292af35` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:25.553315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c54da-7614-4b29-9164-d1a4fc14f131 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e52b070e4a29400cb7e030d1bb9229d6, role_id=fe332203b7e04d6084e26c5be292af35)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:25.554700 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c54da-7614-4b29-9164-d1a4fc14f131 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:25.562533 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22a6cb-85f1-4804-9121-3be6dcb0d424 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:25.564382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 157/304] 10.222.0.84 () {58 vars in 1334 bytes} [Thu May 19 19:36:25 2022] PUT /identity/v3/system/users/e52b070e4a29400cb7e030d1bb9229d6/roles/fe332203b7e04d6084e26c5be292af35 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:36:25.566045 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c54da-7614-4b29-9164-d1a4fc14f131 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:25.603379 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735174c-a503-4025-a0ef-0e8c7e2661e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:25.604343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735174c-a503-4025-a0ef-0e8c7e2661e9 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:25.604738 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735174c-a503-4025-a0ef-0e8c7e2661e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:25.605080 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735174c-a503-4025-a0ef-0e8c7e2661e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:25.620866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735174c-a503-4025-a0ef-0e8c7e2661e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-2735174c-a503-4025-a0ef-0e8c7e2661e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:25.629518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:25.630481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:25.631269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:25.631269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:25.631269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:25.631269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:25.631269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:25.631269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:25.631269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:25.631269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:25.631269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:25.631269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:25.631269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:25.631269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:25.631269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 148/305] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:25.638245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735174c-a503-4025-a0ef-0e8c7e2661e9 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:25.707210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:25.738533 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c54da-7614-4b29-9164-d1a4fc14f131 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:25.781886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b23c9a-6983-4e13-85b3-b54bd14cf3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:25.782719 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b23c9a-6983-4e13-85b3-b54bd14cf3ee None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:25.783072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b23c9a-6983-4e13-85b3-b54bd14cf3ee None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:25.783342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b23c9a-6983-4e13-85b3-b54bd14cf3ee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:25.795438 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735174c-a503-4025-a0ef-0e8c7e2661e9 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:25.797857 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b23c9a-6983-4e13-85b3-b54bd14cf3ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-99b23c9a-6983-4e13-85b3-b54bd14cf3ee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:25.804796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:25.805746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:25.806535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:25.806535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:25.806535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:25.806535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:25.806535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:25.806535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:25.806535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:25.806535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:25.806535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:25.806535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:25.806535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:25.806535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:25.806535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 158/306] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:25.807643 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c91e23-78ed-407a-948e-a032287b4557 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:25.807643 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c91e23-78ed-407a-948e-a032287b4557 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:25.807785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c91e23-78ed-407a-948e-a032287b4557 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:25.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 159/307] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:25.813453 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c91e23-78ed-407a-948e-a032287b4557 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:25.816668 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:25.836644 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26144ac1-c997-41c8-819a-1f874502b22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:25.837316 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26144ac1-c997-41c8-819a-1f874502b22e None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:25.837590 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26144ac1-c997-41c8-819a-1f874502b22e None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:25.837906 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26144ac1-c997-41c8-819a-1f874502b22e None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:25.838683 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26144ac1-c997-41c8-819a-1f874502b22e None admin] RBAC: Adding query filter params (name=reader) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:25.838988 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26144ac1-c997-41c8-819a-1f874502b22e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:25.840422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26144ac1-c997-41c8-819a-1f874502b22e None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:25.848887 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 149/308] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:25.850094 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac808539-e8a2-4254-a868-69e4a72dcfaa None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:25.850410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac808539-e8a2-4254-a868-69e4a72dcfaa None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:25.850687 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac808539-e8a2-4254-a868-69e4a72dcfaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:25.867370 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-ac808539-e8a2-4254-a868-69e4a72dcfaa None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:25.868531 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-ac808539-e8a2-4254-a868-69e4a72dcfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:25.934571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c91e23-78ed-407a-948e-a032287b4557 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:25.944339 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 150/309] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:25.945289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac808539-e8a2-4254-a868-69e4a72dcfaa None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:25.982390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cca895-b967-4012-b674-ba2ce87f1df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:25.983556 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cca895-b967-4012-b674-ba2ce87f1df6 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:25.984332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cca895-b967-4012-b674-ba2ce87f1df6 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:25.984826 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cca895-b967-4012-b674-ba2ce87f1df6 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:25.986016 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cca895-b967-4012-b674-ba2ce87f1df6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:25.986541 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cca895-b967-4012-b674-ba2ce87f1df6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:25.988246 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cca895-b967-4012-b674-ba2ce87f1df6 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:26.004396 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 160/310] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:26.004396 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cca895-b967-4012-b674-ba2ce87f1df6 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:26.004396 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:26.073982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac808539-e8a2-4254-a868-69e4a72dcfaa None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:26.083293 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:26.108427 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cca895-b967-4012-b674-ba2ce87f1df6 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:26.112822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525c242-e953-4124-9423-c1c0aefbbb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:26.113704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525c242-e953-4124-9423-c1c0aefbbb98 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:26.113979 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525c242-e953-4124-9423-c1c0aefbbb98 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:26.114247 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525c242-e953-4124-9423-c1c0aefbbb98 None admin] PATH_INFO: `/v3/users/2cd044daf056476e8ee8ffa58b8aaffe` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:26.115466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525c242-e953-4124-9423-c1c0aefbbb98 None admin] RBAC: Authorizing `identity:get_user(user_id=2cd044daf056476e8ee8ffa58b8aaffe)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:26.116982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525c242-e953-4124-9423-c1c0aefbbb98 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:26.123219 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 151/311] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:25 2022] GET /identity/v3/users/2cd044daf056476e8ee8ffa58b8aaffe => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:26.123219 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525c242-e953-4124-9423-c1c0aefbbb98 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:26.144720 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fdd60-93c9-4d3b-9224-0ba03b50ab51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:26.145610 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fdd60-93c9-4d3b-9224-0ba03b50ab51 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:26.145955 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fdd60-93c9-4d3b-9224-0ba03b50ab51 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:26.146328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fdd60-93c9-4d3b-9224-0ba03b50ab51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:26.147559 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fdd60-93c9-4d3b-9224-0ba03b50ab51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:26.149122 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fdd60-93c9-4d3b-9224-0ba03b50ab51 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:26.151000 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 161/312] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:26.151686 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fdd60-93c9-4d3b-9224-0ba03b50ab51 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:26.225152 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:26.230420 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525c242-e953-4124-9423-c1c0aefbbb98 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:26.257866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fdd60-93c9-4d3b-9224-0ba03b50ab51 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:26.269830 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cec549-d6a8-486f-b0a8-8ef6d7b5a4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:26.271153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cec549-d6a8-486f-b0a8-8ef6d7b5a4d5 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:26.271775 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cec549-d6a8-486f-b0a8-8ef6d7b5a4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:26.272133 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cec549-d6a8-486f-b0a8-8ef6d7b5a4d5 None admin] PATH_INFO: `/v3/users/87b1be9a8a774c239984d2b2e37ac80e` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:26.286661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cec549-d6a8-486f-b0a8-8ef6d7b5a4d5 None admin] RBAC: Authorizing `identity:get_user(user_id=87b1be9a8a774c239984d2b2e37ac80e)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:26.288234 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cec549-d6a8-486f-b0a8-8ef6d7b5a4d5 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:26.294779 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 152/313] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:26 2022] GET /identity/v3/users/87b1be9a8a774c239984d2b2e37ac80e => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:26.294779 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cec549-d6a8-486f-b0a8-8ef6d7b5a4d5 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:26.296567 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f14f85-91d7-49a9-9716-044cd1ffb3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:26.297227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f14f85-91d7-49a9-9716-044cd1ffb3fd None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:26.297503 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f14f85-91d7-49a9-9716-044cd1ffb3fd None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:26.297804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f14f85-91d7-49a9-9716-044cd1ffb3fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:26.330481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:26.370821 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f14f85-91d7-49a9-9716-044cd1ffb3fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:26.372547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f14f85-91d7-49a9-9716-044cd1ffb3fd None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:26.398046 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cec549-d6a8-486f-b0a8-8ef6d7b5a4d5 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:26.435328 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46bcad-0b63-4f61-b3b6-fa25914d3a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:26.435996 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46bcad-0b63-4f61-b3b6-fa25914d3a94 None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:26.436272 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46bcad-0b63-4f61-b3b6-fa25914d3a94 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:26.436540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46bcad-0b63-4f61-b3b6-fa25914d3a94 None admin] PATH_INFO: `/v3/groups` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:26.437215 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46bcad-0b63-4f61-b3b6-fa25914d3a94 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:26.438680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46bcad-0b63-4f61-b3b6-fa25914d3a94 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:26.443709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:26.455899 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 153/314] 10.222.0.84 () {60 vars in 1134 bytes} [Thu May 19 19:36:26 2022] POST /identity/v3/groups => generated 229 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:36:26.459159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46bcad-0b63-4f61-b3b6-fa25914d3a94 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:26.486943 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 162/315] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:26.488856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f14f85-91d7-49a9-9716-044cd1ffb3fd None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:26.594668 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:26.607247 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46bcad-0b63-4f61-b3b6-fa25914d3a94 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:26.617768 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f14f85-91d7-49a9-9716-044cd1ffb3fd None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:26.656449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8de3d8-5d6a-4cd9-8232-bf4c9376cce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:26.656656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8de3d8-5d6a-4cd9-8232-bf4c9376cce0 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:26.656908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8de3d8-5d6a-4cd9-8232-bf4c9376cce0 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:26.657279 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8de3d8-5d6a-4cd9-8232-bf4c9376cce0 None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:26.659348 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8de3d8-5d6a-4cd9-8232-bf4c9376cce0 None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:26.660263 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8de3d8-5d6a-4cd9-8232-bf4c9376cce0 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:26.662478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 163/316] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:26 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:26.663315 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ce224-4ef1-47cd-a96f-a729c222907e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:26.663980 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ce224-4ef1-47cd-a96f-a729c222907e None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:26.664258 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ce224-4ef1-47cd-a96f-a729c222907e None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:26.664529 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ce224-4ef1-47cd-a96f-a729c222907e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:26.665364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ce224-4ef1-47cd-a96f-a729c222907e None admin] RBAC: Adding query filter params (role.id=6ab02cf5cb1b4d3595ea8ea883ff6f2d, user.id=87b1be9a8a774c239984d2b2e37ac80e, scope.system=all) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:26.665674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ce224-4ef1-47cd-a96f-a729c222907e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:26.667189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ce224-4ef1-47cd-a96f-a729c222907e None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:26.669668 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8de3d8-5d6a-4cd9-8232-bf4c9376cce0 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:26.676211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 154/317] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 19:36:26 2022] GET /identity/v3/role_assignments?user.id=87b1be9a8a774c239984d2b2e37ac80e&scope.system=all&role.id=6ab02cf5cb1b4d3595ea8ea883ff6f2d => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:26.721265 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:26.791688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8de3d8-5d6a-4cd9-8232-bf4c9376cce0 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:26.829163 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3be35-8e52-492c-9d13-c13832f2c176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:26.830218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3be35-8e52-492c-9d13-c13832f2c176 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:26.830717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3be35-8e52-492c-9d13-c13832f2c176 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:26.831177 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3be35-8e52-492c-9d13-c13832f2c176 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:26.832333 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3be35-8e52-492c-9d13-c13832f2c176 None admin] RBAC: Adding query filter params (role.id=6ab02cf5cb1b4d3595ea8ea883ff6f2d, scope.project.id=7c06d53d4c434698958a453f9500feaa, user.id=2cd044daf056476e8ee8ffa58b8aaffe) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:26.832889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3be35-8e52-492c-9d13-c13832f2c176 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:26.834623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3be35-8e52-492c-9d13-c13832f2c176 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:26.842189 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 164/318] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:26 2022] GET /identity/v3/role_assignments?user.id=2cd044daf056476e8ee8ffa58b8aaffe&scope.project.id=7c06d53d4c434698958a453f9500feaa&role.id=6ab02cf5cb1b4d3595ea8ea883ff6f2d => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:27.871710 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0f593-f12b-442f-bbe7-0b70c866fdea None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:27.872228 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0f593-f12b-442f-bbe7-0b70c866fdea None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:27.872228 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0f593-f12b-442f-bbe7-0b70c866fdea None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:27.873691 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 155/319] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:27.878993 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39650a7a-5ce7-49a4-a669-06c7acaf120b None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:27.879404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39650a7a-5ce7-49a4-a669-06c7acaf120b None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:27.879497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39650a7a-5ce7-49a4-a669-06c7acaf120b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:27.896703 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-39650a7a-5ce7-49a4-a669-06c7acaf120b None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:27.897865 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-39650a7a-5ce7-49a4-a669-06c7acaf120b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:27.965221 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 165/320] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:27.988622 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0f593-f12b-442f-bbe7-0b70c866fdea None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:28.097274 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0f593-f12b-442f-bbe7-0b70c866fdea None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:28.131799 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f43960-54b3-45fd-ba7e-3e7622b23526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:28.132524 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f43960-54b3-45fd-ba7e-3e7622b23526 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:28.132803 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f43960-54b3-45fd-ba7e-3e7622b23526 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:28.133072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f43960-54b3-45fd-ba7e-3e7622b23526 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:28.143232 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f43960-54b3-45fd-ba7e-3e7622b23526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-43f43960-54b3-45fd-ba7e-3e7622b23526 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:28.151045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:28.151903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:28.152776 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:28.152776 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:28.152776 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:28.152776 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:28.152776 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:28.152776 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:28.152776 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:28.152776 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:28.152776 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:28.152776 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:28.152776 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:28.152776 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:28.152776 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 156/321] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:28.159418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39650a7a-5ce7-49a4-a669-06c7acaf120b None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:28.190718 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:28.261646 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39650a7a-5ce7-49a4-a669-06c7acaf120b None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:28.295188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6bf7c-6d59-4f9f-8111-e058aa95772b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:28.295859 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6bf7c-6d59-4f9f-8111-e058aa95772b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:28.296136 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6bf7c-6d59-4f9f-8111-e058aa95772b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:28.296403 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6bf7c-6d59-4f9f-8111-e058aa95772b None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:28.297156 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6bf7c-6d59-4f9f-8111-e058aa95772b None admin] RBAC: Adding query filter params (name=member) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:28.297457 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6bf7c-6d59-4f9f-8111-e058aa95772b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:28.298875 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6bf7c-6d59-4f9f-8111-e058aa95772b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:28.307670 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 166/322] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:28.313206 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f43960-54b3-45fd-ba7e-3e7622b23526 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:28.333821 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:28.417710 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f43960-54b3-45fd-ba7e-3e7622b23526 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:28.453929 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed71046-a0c8-4c9c-8f49-e99400691717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:28.454843 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed71046-a0c8-4c9c-8f49-e99400691717 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:28.455120 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed71046-a0c8-4c9c-8f49-e99400691717 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:28.455388 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed71046-a0c8-4c9c-8f49-e99400691717 None admin] PATH_INFO: `/v3/users/e52b070e4a29400cb7e030d1bb9229d6` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:28.456625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed71046-a0c8-4c9c-8f49-e99400691717 None admin] RBAC: Authorizing `identity:get_user(user_id=e52b070e4a29400cb7e030d1bb9229d6)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:28.458242 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed71046-a0c8-4c9c-8f49-e99400691717 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:28.460050 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 157/323] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:28 2022] GET /identity/v3/users/e52b070e4a29400cb7e030d1bb9229d6 => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:28.465740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6bf7c-6d59-4f9f-8111-e058aa95772b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:28.559520 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:28.567334 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6bf7c-6d59-4f9f-8111-e058aa95772b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:28.600551 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ab5e6-7838-4bc8-bd5f-e427b99856d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:28.601208 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ab5e6-7838-4bc8-bd5f-e427b99856d9 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:28.601484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ab5e6-7838-4bc8-bd5f-e427b99856d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:28.601751 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ab5e6-7838-4bc8-bd5f-e427b99856d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:28.602977 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ab5e6-7838-4bc8-bd5f-e427b99856d9 None admin] RBAC: Adding query filter params (role.id=fe332203b7e04d6084e26c5be292af35, user.id=e52b070e4a29400cb7e030d1bb9229d6, scope.system=all) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:28.603283 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ab5e6-7838-4bc8-bd5f-e427b99856d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:28.604767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ab5e6-7838-4bc8-bd5f-e427b99856d9 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:28.621526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 167/324] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 19:36:28 2022] GET /identity/v3/role_assignments?user.id=e52b070e4a29400cb7e030d1bb9229d6&scope.system=all&role.id=fe332203b7e04d6084e26c5be292af35 => generated 510 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:28.824511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c27c8-4c48-4f94-a243-29139965b9e6 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:28.824915 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c27c8-4c48-4f94-a243-29139965b9e6 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:28.825089 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c27c8-4c48-4f94-a243-29139965b9e6 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:28.826644 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 158/325] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:28.832202 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da4c4d-672a-435e-8a9e-04e24da755ca None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:28.832639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da4c4d-672a-435e-8a9e-04e24da755ca None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:28.832900 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da4c4d-672a-435e-8a9e-04e24da755ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:28.850866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-a5da4c4d-672a-435e-8a9e-04e24da755ca None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:28.852052 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-a5da4c4d-672a-435e-8a9e-04e24da755ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:28.882663 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1872c-355e-45d7-bc45-50ba3e6e86c5 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:28.883172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1872c-355e-45d7-bc45-50ba3e6e86c5 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:28.883256 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1872c-355e-45d7-bc45-50ba3e6e86c5 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:28.884574 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 159/326] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:28.889364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a2557-f630-4d57-82fb-3c5bb3cf5226 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:28.889645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a2557-f630-4d57-82fb-3c5bb3cf5226 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:28.889912 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a2557-f630-4d57-82fb-3c5bb3cf5226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:28.906187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-148a2557-f630-4d57-82fb-3c5bb3cf5226 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:28.907409 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-148a2557-f630-4d57-82fb-3c5bb3cf5226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:28.921425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 168/327] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:28.944240 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da4c4d-672a-435e-8a9e-04e24da755ca None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:28.973170 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 160/328] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:28.995619 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a2557-f630-4d57-82fb-3c5bb3cf5226 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:29.012669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:29.024913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:29.044578 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da4c4d-672a-435e-8a9e-04e24da755ca None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:29.080958 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a2544-b6de-402c-a425-8689984a1627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:29.082015 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a2544-b6de-402c-a425-8689984a1627 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:29.082321 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a2544-b6de-402c-a425-8689984a1627 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:29.082594 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a2544-b6de-402c-a425-8689984a1627 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:29.083748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a2544-b6de-402c-a425-8689984a1627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:29.085203 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a2544-b6de-402c-a425-8689984a1627 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:29.086859 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 169/329] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:29.092511 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a2544-b6de-402c-a425-8689984a1627 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:29.097808 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a2557-f630-4d57-82fb-3c5bb3cf5226 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:29.133223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8347ab-f56b-4014-a5eb-696518eac883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:29.134066 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8347ab-f56b-4014-a5eb-696518eac883 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:29.134373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8347ab-f56b-4014-a5eb-696518eac883 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:29.134648 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8347ab-f56b-4014-a5eb-696518eac883 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:29.145695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8347ab-f56b-4014-a5eb-696518eac883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-5b8347ab-f56b-4014-a5eb-696518eac883 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:29.153387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:29.154312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:29.155079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:29.155079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:29.155079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:29.155079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:29.155079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:29.155079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:29.155079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:29.155079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:29.155079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:29.155079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:29.155079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:29.155079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:29.155079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 161/330] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:36:29.160898 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8347ab-f56b-4014-a5eb-696518eac883 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:29.197303 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a2544-b6de-402c-a425-8689984a1627 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:29.230592 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff45b12-ed85-457f-b701-4b308c317d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:29.231236 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff45b12-ed85-457f-b701-4b308c317d1c None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:29.231511 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff45b12-ed85-457f-b701-4b308c317d1c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:29.231779 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff45b12-ed85-457f-b701-4b308c317d1c None admin] PATH_INFO: `/v3/groups` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:29.232481 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff45b12-ed85-457f-b701-4b308c317d1c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:29.233933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff45b12-ed85-457f-b701-4b308c317d1c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:29.238516 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:29.250911 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 170/331] 10.222.0.84 () {60 vars in 1134 bytes} [Thu May 19 19:36:29 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:36:29.263669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8347ab-f56b-4014-a5eb-696518eac883 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:29.302478 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163974bc-fd55-4441-89e1-b8b638326613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:29.303153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163974bc-fd55-4441-89e1-b8b638326613 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:29.303437 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163974bc-fd55-4441-89e1-b8b638326613 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:29.303698 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163974bc-fd55-4441-89e1-b8b638326613 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:29.304514 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163974bc-fd55-4441-89e1-b8b638326613 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:29.304826 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163974bc-fd55-4441-89e1-b8b638326613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:29.306411 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163974bc-fd55-4441-89e1-b8b638326613 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:29.316652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 162/332] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:29.322385 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff45b12-ed85-457f-b701-4b308c317d1c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:29.337549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:29.445533 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff45b12-ed85-457f-b701-4b308c317d1c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:29.479583 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41372f49-11f6-4883-bcfb-466816ca39c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:29.480478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41372f49-11f6-4883-bcfb-466816ca39c0 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:29.480754 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41372f49-11f6-4883-bcfb-466816ca39c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:29.481019 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41372f49-11f6-4883-bcfb-466816ca39c0 None admin] PATH_INFO: `/v3/users/87b1be9a8a774c239984d2b2e37ac80e` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:29.482226 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41372f49-11f6-4883-bcfb-466816ca39c0 None admin] RBAC: Authorizing `identity:get_user(user_id=87b1be9a8a774c239984d2b2e37ac80e)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:29.483747 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41372f49-11f6-4883-bcfb-466816ca39c0 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:29.485535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 171/333] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:29 2022] GET /identity/v3/users/87b1be9a8a774c239984d2b2e37ac80e => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:29.491433 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163974bc-fd55-4441-89e1-b8b638326613 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:29.599056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163974bc-fd55-4441-89e1-b8b638326613 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:29.641802 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495fc8b-59f8-4d60-a738-46d96d88ca43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:29.642736 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495fc8b-59f8-4d60-a738-46d96d88ca43 None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:29.642990 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495fc8b-59f8-4d60-a738-46d96d88ca43 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:29.643259 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495fc8b-59f8-4d60-a738-46d96d88ca43 None admin] PATH_INFO: `/v3/system/users/87b1be9a8a774c239984d2b2e37ac80e/roles/6ab02cf5cb1b4d3595ea8ea883ff6f2d` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:29.644937 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495fc8b-59f8-4d60-a738-46d96d88ca43 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=87b1be9a8a774c239984d2b2e37ac80e, role_id=6ab02cf5cb1b4d3595ea8ea883ff6f2d)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:29.646376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495fc8b-59f8-4d60-a738-46d96d88ca43 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:29.655996 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 163/334] 10.222.0.84 () {58 vars in 1334 bytes} [Thu May 19 19:36:29 2022] PUT /identity/v3/system/users/87b1be9a8a774c239984d2b2e37ac80e/roles/6ab02cf5cb1b4d3595ea8ea883ff6f2d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:36:31.924775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e205374-46f8-4406-9616-84a2303e3569 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:31.925458 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e205374-46f8-4406-9616-84a2303e3569 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:31.925458 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e205374-46f8-4406-9616-84a2303e3569 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:31.927888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 172/335] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:31.936824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e854e-39e9-4ea6-ad07-ad5586a71d48 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:31.937793 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e854e-39e9-4ea6-ad07-ad5586a71d48 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:31.938348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e854e-39e9-4ea6-ad07-ad5586a71d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:31.949815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:31.967381 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-d78e854e-39e9-4ea6-ad07-ad5586a71d48 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:31.969813 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-d78e854e-39e9-4ea6-ad07-ad5586a71d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:32.048022 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 164/336] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:32.088177 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e205374-46f8-4406-9616-84a2303e3569 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:32.142488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501cc5e-c05d-4304-9a00-143665e5bfeb None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:32.142832 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501cc5e-c05d-4304-9a00-143665e5bfeb None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:32.143046 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501cc5e-c05d-4304-9a00-143665e5bfeb None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:32.144910 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 165/337] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:36:32.151677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c0b71-8356-4ee2-b5a1-5b5268705160 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:32.151995 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c0b71-8356-4ee2-b5a1-5b5268705160 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:32.152223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c0b71-8356-4ee2-b5a1-5b5268705160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:32.187366 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-fb0c0b71-8356-4ee2-b5a1-5b5268705160 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:32.188975 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-fb0c0b71-8356-4ee2-b5a1-5b5268705160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:32.204911 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:32.233862 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e205374-46f8-4406-9616-84a2303e3569 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:32.272497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85d114-9dd2-44d4-b5e8-e5a4bf358862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:32.273855 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85d114-9dd2-44d4-b5e8-e5a4bf358862 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:32.274312 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85d114-9dd2-44d4-b5e8-e5a4bf358862 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:32.274773 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85d114-9dd2-44d4-b5e8-e5a4bf358862 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:32.286789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85d114-9dd2-44d4-b5e8-e5a4bf358862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-3f85d114-9dd2-44d4-b5e8-e5a4bf358862 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:32.296144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:32.297242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:32.298037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:32.298037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:32.298037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:32.298037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:32.298037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:32.298037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:32.298037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:32.298037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:32.298037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:32.298037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:32.298037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:32.298037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:32.300144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 173/338] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:36:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:32.306566 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85d114-9dd2-44d4-b5e8-e5a4bf358862 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:32.312644 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 166/339] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:32.384840 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c0b71-8356-4ee2-b5a1-5b5268705160 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:32.424409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85d114-9dd2-44d4-b5e8-e5a4bf358862 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:32.461136 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6586b7d-9582-4ed2-ba21-ad0ccb8854fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:32.462188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6586b7d-9582-4ed2-ba21-ad0ccb8854fb None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:32.462674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6586b7d-9582-4ed2-ba21-ad0ccb8854fb None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:32.463208 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6586b7d-9582-4ed2-ba21-ad0ccb8854fb None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:32.464174 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6586b7d-9582-4ed2-ba21-ad0ccb8854fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:32.464685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6586b7d-9582-4ed2-ba21-ad0ccb8854fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:32.466403 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6586b7d-9582-4ed2-ba21-ad0ccb8854fb None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:32.472374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:32.476204 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 174/340] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:32.481399 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:32.483046 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6586b7d-9582-4ed2-ba21-ad0ccb8854fb None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:32.540367 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c0b71-8356-4ee2-b5a1-5b5268705160 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:32.591259 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061d0fe-ebd0-41b8-b637-c92c880f0875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:32.592157 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061d0fe-ebd0-41b8-b637-c92c880f0875 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:32.592474 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6586b7d-9582-4ed2-ba21-ad0ccb8854fb None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:32.592579 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061d0fe-ebd0-41b8-b637-c92c880f0875 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:32.592818 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061d0fe-ebd0-41b8-b637-c92c880f0875 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:32.610863 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061d0fe-ebd0-41b8-b637-c92c880f0875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:32.632442 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9b8fc-9709-43a8-aed0-96b3b06744e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-8061d0fe-ebd0-41b8-b637-c92c880f0875 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:32.634036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:32.634952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:32.635714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:32.635714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:32.635714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:32.635714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:32.635714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:32.635714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:32.635714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:32.635714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:32.635714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:32.635714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:32.635714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:32.635714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:32.636290 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9b8fc-9709-43a8-aed0-96b3b06744e4 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:32.636290 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9b8fc-9709-43a8-aed0-96b3b06744e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:32.636290 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9b8fc-9709-43a8-aed0-96b3b06744e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:32.638475 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 167/341] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:36:32.644714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061d0fe-ebd0-41b8-b637-c92c880f0875 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:32.721239 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:32.725837 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9b8fc-9709-43a8-aed0-96b3b06744e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:32.727956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9b8fc-9709-43a8-aed0-96b3b06744e4 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:32.766254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061d0fe-ebd0-41b8-b637-c92c880f0875 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:32.804650 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:32.830434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531328a-3ce6-49cc-a65a-94fa9562a05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:32.831694 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531328a-3ce6-49cc-a65a-94fa9562a05b None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:32.832160 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531328a-3ce6-49cc-a65a-94fa9562a05b None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:32.832602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531328a-3ce6-49cc-a65a-94fa9562a05b None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:32.833701 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531328a-3ce6-49cc-a65a-94fa9562a05b None admin] RBAC: Adding query filter params (name=member) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:32.834196 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531328a-3ce6-49cc-a65a-94fa9562a05b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:32.835795 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531328a-3ce6-49cc-a65a-94fa9562a05b None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:32.846897 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 175/342] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:32.848686 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 168/343] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:32.855299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531328a-3ce6-49cc-a65a-94fa9562a05b None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:32.855445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9b8fc-9709-43a8-aed0-96b3b06744e4 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:32.991542 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531328a-3ce6-49cc-a65a-94fa9562a05b None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:33.010646 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:33.013233 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9b8fc-9709-43a8-aed0-96b3b06744e4 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:33.036945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa7aa2-3bfe-4164-832a-792a0835a158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:33.037887 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa7aa2-3bfe-4164-832a-792a0835a158 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:33.038322 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa7aa2-3bfe-4164-832a-792a0835a158 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:33.038743 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa7aa2-3bfe-4164-832a-792a0835a158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:33.067363 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094f284-7751-479f-a6f0-f397febe56c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:33.068232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094f284-7751-479f-a6f0-f397febe56c5 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:33.068510 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094f284-7751-479f-a6f0-f397febe56c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:33.068781 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094f284-7751-479f-a6f0-f397febe56c5 None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:33.071821 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094f284-7751-479f-a6f0-f397febe56c5 None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:33.074194 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094f284-7751-479f-a6f0-f397febe56c5 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:33.075952 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 176/344] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:32 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:33.079531 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3881da7-d198-4329-b801-5921891aa90c None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:33.079809 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3881da7-d198-4329-b801-5921891aa90c None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:33.080079 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3881da7-d198-4329-b801-5921891aa90c None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:33.087041 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 177/345] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:33 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:33.087822 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3881da7-d198-4329-b801-5921891aa90c None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:33.140135 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:33.156807 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa7aa2-3bfe-4164-832a-792a0835a158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:33.158645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa7aa2-3bfe-4164-832a-792a0835a158 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:33.219027 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3881da7-d198-4329-b801-5921891aa90c None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:33.262734 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 169/346] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:33.264152 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c6390-887f-474e-aa1f-8e09c3aa6807 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:33.264510 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c6390-887f-474e-aa1f-8e09c3aa6807 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:33.264782 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c6390-887f-474e-aa1f-8e09c3aa6807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:33.271198 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc62c4d-6797-4aaf-9ebe-297c40478f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:33.272084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc62c4d-6797-4aaf-9ebe-297c40478f2e None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:33.272364 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc62c4d-6797-4aaf-9ebe-297c40478f2e None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:33.272634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc62c4d-6797-4aaf-9ebe-297c40478f2e None admin] PATH_INFO: `/v3/groups/7c33f3b0c75c4857b3281ec028418591` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:33.281800 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-de1c6390-887f-474e-aa1f-8e09c3aa6807 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:33.283007 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-de1c6390-887f-474e-aa1f-8e09c3aa6807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:33.283116 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc62c4d-6797-4aaf-9ebe-297c40478f2e None admin] RBAC: Authorizing `identity:get_group(group_id=7c33f3b0c75c4857b3281ec028418591)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:33.284615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc62c4d-6797-4aaf-9ebe-297c40478f2e None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:33.287894 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 178/347] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:36:33 2022] GET /identity/v3/groups/7c33f3b0c75c4857b3281ec028418591 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:33.289219 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8bf4c-eddd-44fb-85b5-b9d2cd406612 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:33.289503 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8bf4c-eddd-44fb-85b5-b9d2cd406612 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:33.289775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8bf4c-eddd-44fb-85b5-b9d2cd406612 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:33.293241 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 179/348] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:33.293984 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8bf4c-eddd-44fb-85b5-b9d2cd406612 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:33.341237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:33.357230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 170/349] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:33.358548 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845a6fa-71ef-4ec5-9de0-a4eba5381449 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:33.358828 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845a6fa-71ef-4ec5-9de0-a4eba5381449 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:33.359101 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845a6fa-71ef-4ec5-9de0-a4eba5381449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:33.378294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-b845a6fa-71ef-4ec5-9de0-a4eba5381449 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:33.379451 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-b845a6fa-71ef-4ec5-9de0-a4eba5381449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:33.389558 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:33.429989 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8bf4c-eddd-44fb-85b5-b9d2cd406612 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:33.464732 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 171/350] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:33.465915 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845a6fa-71ef-4ec5-9de0-a4eba5381449 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:33.470721 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01904809-ade8-4a23-94ce-64db95942d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:33.471908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01904809-ade8-4a23-94ce-64db95942d0d None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:33.472397 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01904809-ade8-4a23-94ce-64db95942d0d None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:33.472858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01904809-ade8-4a23-94ce-64db95942d0d None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:33.474287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01904809-ade8-4a23-94ce-64db95942d0d None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:33.475958 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01904809-ade8-4a23-94ce-64db95942d0d None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:33.478530 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 180/351] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:33 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:33.479494 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01904809-ade8-4a23-94ce-64db95942d0d None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:33.575146 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845a6fa-71ef-4ec5-9de0-a4eba5381449 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:33.581210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:33.585066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01904809-ade8-4a23-94ce-64db95942d0d None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:33.618644 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb17b28-2a6b-423d-82e4-594e286fe250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:33.619237 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb17b28-2a6b-423d-82e4-594e286fe250 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:33.619516 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb17b28-2a6b-423d-82e4-594e286fe250 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:33.619781 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb17b28-2a6b-423d-82e4-594e286fe250 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:33.620816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb17b28-2a6b-423d-82e4-594e286fe250 None admin] RBAC: Adding query filter params (group.id=7c33f3b0c75c4857b3281ec028418591, role.id=4a241925b6cc4218b4f2af580b151877, scope.project.id=644a8176f5874355bc660ed98897307f) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:33.621163 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb17b28-2a6b-423d-82e4-594e286fe250 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:33.622638 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb17b28-2a6b-423d-82e4-594e286fe250 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:33.623052 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd166d34-b978-4f99-b6e8-527e9d19d803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:33.624073 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd166d34-b978-4f99-b6e8-527e9d19d803 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:33.624537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd166d34-b978-4f99-b6e8-527e9d19d803 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:33.625004 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd166d34-b978-4f99-b6e8-527e9d19d803 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:33.636040 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 172/352] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:36:33 2022] GET /identity/v3/role_assignments?group.id=7c33f3b0c75c4857b3281ec028418591&scope.project.id=644a8176f5874355bc660ed98897307f&role.id=4a241925b6cc4218b4f2af580b151877 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:33.636945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb17b28-2a6b-423d-82e4-594e286fe250 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:33.638491 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd166d34-b978-4f99-b6e8-527e9d19d803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-cd166d34-b978-4f99-b6e8-527e9d19d803 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:33.647022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:33.648018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:33.648824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:33.648824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:33.648824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:33.648824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:33.648824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:33.648824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:33.648824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:33.648824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:33.648824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:33.648824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:33.648824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:33.648824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:33.649910 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:33.651942 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 181/353] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:33.653113 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd166d34-b978-4f99-b6e8-527e9d19d803 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:33.762422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb17b28-2a6b-423d-82e4-594e286fe250 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:33.792378 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd166d34-b978-4f99-b6e8-527e9d19d803 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:33.801768 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39208aa5-dd21-4b5c-be59-a06351758bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:33.802679 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39208aa5-dd21-4b5c-be59-a06351758bfc None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:33.802956 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39208aa5-dd21-4b5c-be59-a06351758bfc None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:33.803224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39208aa5-dd21-4b5c-be59-a06351758bfc None admin] PATH_INFO: `/v3/groups/7c33f3b0c75c4857b3281ec028418591` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:33.804393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39208aa5-dd21-4b5c-be59-a06351758bfc None admin] RBAC: Authorizing `identity:get_group(group_id=7c33f3b0c75c4857b3281ec028418591)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:33.809298 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39208aa5-dd21-4b5c-be59-a06351758bfc None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:33.809298 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 173/354] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:36:33 2022] GET /identity/v3/groups/7c33f3b0c75c4857b3281ec028418591 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:33.809298 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39208aa5-dd21-4b5c-be59-a06351758bfc None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:33.878553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280430ae-dfb1-426b-bcd6-523d041f4a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:33.879737 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280430ae-dfb1-426b-bcd6-523d041f4a20 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:33.880222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280430ae-dfb1-426b-bcd6-523d041f4a20 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:33.880693 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280430ae-dfb1-426b-bcd6-523d041f4a20 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:33.902677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:33.907714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280430ae-dfb1-426b-bcd6-523d041f4a20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-280430ae-dfb1-426b-bcd6-523d041f4a20 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:33.922450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:33.923601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:33.924444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:33.924444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:33.924444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:33.924444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:33.924444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:33.924444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:33.924444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:33.924444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:33.924444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:33.924444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:33.924444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:36:33.924444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:33.925507 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 182/355] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:33.926368 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280430ae-dfb1-426b-bcd6-523d041f4a20 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:33.931602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39208aa5-dd21-4b5c-be59-a06351758bfc None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:33.964744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:33.989213 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65e8c4-f708-4a38-9dd4-15cd4180cb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:33.990762 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65e8c4-f708-4a38-9dd4-15cd4180cb69 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:33.991644 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65e8c4-f708-4a38-9dd4-15cd4180cb69 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:33.992784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65e8c4-f708-4a38-9dd4-15cd4180cb69 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:33.993703 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad65e8c4-f708-4a38-9dd4-15cd4180cb69 None admin] RBAC: Adding query filter params (name=member) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:33.994423 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad65e8c4-f708-4a38-9dd4-15cd4180cb69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:33.996939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad65e8c4-f708-4a38-9dd4-15cd4180cb69 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:34.007950 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 174/356] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:34.009300 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65e8c4-f708-4a38-9dd4-15cd4180cb69 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:34.065095 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280430ae-dfb1-426b-bcd6-523d041f4a20 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:34.107257 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955a28b-6b28-4550-a2fe-366621f9326b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:34.107969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955a28b-6b28-4550-a2fe-366621f9326b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:34.108256 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955a28b-6b28-4550-a2fe-366621f9326b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:34.108528 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955a28b-6b28-4550-a2fe-366621f9326b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:34.109365 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955a28b-6b28-4550-a2fe-366621f9326b None admin] RBAC: Adding query filter params (group.id=7c33f3b0c75c4857b3281ec028418591, role.id=fe332203b7e04d6084e26c5be292af35, scope.project.id=644a8176f5874355bc660ed98897307f) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:34.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955a28b-6b28-4550-a2fe-366621f9326b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:34.111858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955a28b-6b28-4550-a2fe-366621f9326b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:34.122975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 183/357] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:36:33 2022] GET /identity/v3/role_assignments?group.id=7c33f3b0c75c4857b3281ec028418591&scope.project.id=644a8176f5874355bc660ed98897307f&role.id=fe332203b7e04d6084e26c5be292af35 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:34.124595 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb4aa4-d9fc-4c07-8183-f3a9ef1391cf None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:34.125087 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb4aa4-d9fc-4c07-8183-f3a9ef1391cf None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:34.125570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb4aa4-d9fc-4c07-8183-f3a9ef1391cf None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:34.127135 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 184/358] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:34.128202 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb4aa4-d9fc-4c07-8183-f3a9ef1391cf None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:34.163994 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65e8c4-f708-4a38-9dd4-15cd4180cb69 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:34.214715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f035482-58fd-4b9f-956e-0eca30248f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:34.215466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f035482-58fd-4b9f-956e-0eca30248f3a None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:34.215747 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f035482-58fd-4b9f-956e-0eca30248f3a None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:34.216015 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f035482-58fd-4b9f-956e-0eca30248f3a None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:34.216832 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f035482-58fd-4b9f-956e-0eca30248f3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:34.217139 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f035482-58fd-4b9f-956e-0eca30248f3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:34.218573 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f035482-58fd-4b9f-956e-0eca30248f3a None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:34.225755 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:34.230434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 175/359] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:34.231846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26258f96-4a55-4b58-ad2b-ebb9d44d3741 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:34.232135 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26258f96-4a55-4b58-ad2b-ebb9d44d3741 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:34.232410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26258f96-4a55-4b58-ad2b-ebb9d44d3741 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:34.234005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 176/360] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:34.235397 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f80db3-c438-4a90-b68f-913af0f29c28 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:34.235612 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f80db3-c438-4a90-b68f-913af0f29c28 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:34.236197 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f80db3-c438-4a90-b68f-913af0f29c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:34.240769 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:34.255820 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-d2f80db3-c438-4a90-b68f-913af0f29c28 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:34.257222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-d2f80db3-c438-4a90-b68f-913af0f29c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:34.262824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb4aa4-d9fc-4c07-8183-f3a9ef1391cf None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:34.306599 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba015a9-3d02-4577-8836-d5d5d47760b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:34.307269 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba015a9-3d02-4577-8836-d5d5d47760b4 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:34.307550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba015a9-3d02-4577-8836-d5d5d47760b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:34.307821 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba015a9-3d02-4577-8836-d5d5d47760b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:34.361273 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 177/361] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:34.362940 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655402ce-747f-49ab-9c4f-aae385055c65 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:34.363344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655402ce-747f-49ab-9c4f-aae385055c65 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:34.363667 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655402ce-747f-49ab-9c4f-aae385055c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:34.385044 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-655402ce-747f-49ab-9c4f-aae385055c65 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:34.386514 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-655402ce-747f-49ab-9c4f-aae385055c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:34.389885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba015a9-3d02-4577-8836-d5d5d47760b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:34.391416 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba015a9-3d02-4577-8836-d5d5d47760b4 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:34.473216 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 178/362] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:34.474245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655402ce-747f-49ab-9c4f-aae385055c65 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:34.502597 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:34.523576 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 185/363] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:34.527949 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba015a9-3d02-4577-8836-d5d5d47760b4 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:34.545733 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:34.589577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655402ce-747f-49ab-9c4f-aae385055c65 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:34.635519 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e86f1-b85a-4c47-bf89-cd5aca744d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:34.636652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e86f1-b85a-4c47-bf89-cd5aca744d9e None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:34.637061 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e86f1-b85a-4c47-bf89-cd5aca744d9e None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:34.637489 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e86f1-b85a-4c47-bf89-cd5aca744d9e None admin] PATH_INFO: `/v3/users/87b1be9a8a774c239984d2b2e37ac80e` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:34.638853 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e86f1-b85a-4c47-bf89-cd5aca744d9e None admin] RBAC: Authorizing `identity:get_user(user_id=87b1be9a8a774c239984d2b2e37ac80e)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:34.640434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e86f1-b85a-4c47-bf89-cd5aca744d9e None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:34.642199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 179/364] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:36:34 2022] GET /identity/v3/users/87b1be9a8a774c239984d2b2e37ac80e => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:34.643118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e86f1-b85a-4c47-bf89-cd5aca744d9e None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:34.645086 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba015a9-3d02-4577-8836-d5d5d47760b4 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:34.684246 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d419c82-a5b3-418c-a6ed-2ce86470d292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:34.685116 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d419c82-a5b3-418c-a6ed-2ce86470d292 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:34.685397 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d419c82-a5b3-418c-a6ed-2ce86470d292 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:34.685668 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d419c82-a5b3-418c-a6ed-2ce86470d292 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:34.698770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d419c82-a5b3-418c-a6ed-2ce86470d292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-1d419c82-a5b3-418c-a6ed-2ce86470d292 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:34.707882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:34.708936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:34.709874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:34.709874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:34.709874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:34.709874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:34.709874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:34.709874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:34.709874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:34.709874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:34.709874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:34.709874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:34.709874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:34.709874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:34.710536 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 186/365] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:36:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:34.710832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d419c82-a5b3-418c-a6ed-2ce86470d292 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:34.756144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e86f1-b85a-4c47-bf89-cd5aca744d9e None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:34.763731 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:34.792384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f58080-8a9f-4650-8641-97c0713906b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:34.793435 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f58080-8a9f-4650-8641-97c0713906b4 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:34.793869 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f58080-8a9f-4650-8641-97c0713906b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:34.794309 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f58080-8a9f-4650-8641-97c0713906b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:34.805479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f58080-8a9f-4650-8641-97c0713906b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-34f58080-8a9f-4650-8641-97c0713906b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:34.814673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:34.815982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:34.817261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:34.817261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:34.817261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:34.817261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:34.817261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:34.817261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:34.817261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:34.817261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:34.817261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:34.817261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:34.817261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:34.817261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:34.819203 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 180/366] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:36:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:34.820154 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f58080-8a9f-4650-8641-97c0713906b4 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:34.834748 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:34.837296 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d419c82-a5b3-418c-a6ed-2ce86470d292 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:34.874738 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c11403-6ca0-47f9-8a83-0d7848e65fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:34.875800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c11403-6ca0-47f9-8a83-0d7848e65fbd None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:34.875923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c11403-6ca0-47f9-8a83-0d7848e65fbd None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:34.876188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c11403-6ca0-47f9-8a83-0d7848e65fbd None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:34.877385 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c11403-6ca0-47f9-8a83-0d7848e65fbd None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:34.879150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c11403-6ca0-47f9-8a83-0d7848e65fbd None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:34.885521 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 187/367] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:34 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:34.886499 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c11403-6ca0-47f9-8a83-0d7848e65fbd None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:34.928335 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f58080-8a9f-4650-8641-97c0713906b4 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:34.963109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b587a2-ff4a-4ad3-9370-5845f7f86e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:34.964112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b587a2-ff4a-4ad3-9370-5845f7f86e06 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:34.964764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b587a2-ff4a-4ad3-9370-5845f7f86e06 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:34.965238 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b587a2-ff4a-4ad3-9370-5845f7f86e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:34.966251 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b587a2-ff4a-4ad3-9370-5845f7f86e06 None admin] RBAC: Adding query filter params (role.id=6ab02cf5cb1b4d3595ea8ea883ff6f2d, user.id=87b1be9a8a774c239984d2b2e37ac80e, scope.system=all) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:34.967091 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b587a2-ff4a-4ad3-9370-5845f7f86e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:34.968718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b587a2-ff4a-4ad3-9370-5845f7f86e06 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:34.981391 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 181/368] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 19:36:34 2022] GET /identity/v3/role_assignments?user.id=87b1be9a8a774c239984d2b2e37ac80e&scope.system=all&role.id=6ab02cf5cb1b4d3595ea8ea883ff6f2d => generated 510 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:34.982628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b587a2-ff4a-4ad3-9370-5845f7f86e06 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:34.997334 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c11403-6ca0-47f9-8a83-0d7848e65fbd None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:35.012555 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:35.042218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917475a7-37f4-43cd-ad62-5bac82e06f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:35.043180 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917475a7-37f4-43cd-ad62-5bac82e06f16 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:35.043305 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917475a7-37f4-43cd-ad62-5bac82e06f16 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:35.043537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917475a7-37f4-43cd-ad62-5bac82e06f16 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:35.044304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917475a7-37f4-43cd-ad62-5bac82e06f16 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:35.044612 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917475a7-37f4-43cd-ad62-5bac82e06f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:35.046128 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917475a7-37f4-43cd-ad62-5bac82e06f16 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:35.047930 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:35.057664 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 188/369] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:35.057664 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917475a7-37f4-43cd-ad62-5bac82e06f16 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:35.109282 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b587a2-ff4a-4ad3-9370-5845f7f86e06 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:35.146913 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7564577-9dfb-4875-85af-339dd3f45570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:35.147864 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7564577-9dfb-4875-85af-339dd3f45570 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:35.148279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7564577-9dfb-4875-85af-339dd3f45570 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:35.148684 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7564577-9dfb-4875-85af-339dd3f45570 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:35.149593 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7564577-9dfb-4875-85af-339dd3f45570 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:35.150049 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7564577-9dfb-4875-85af-339dd3f45570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:35.152705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7564577-9dfb-4875-85af-339dd3f45570 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:35.163611 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 182/370] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:35.164834 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7564577-9dfb-4875-85af-339dd3f45570 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:35.185253 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917475a7-37f4-43cd-ad62-5bac82e06f16 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:35.243920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cc106-c576-4184-a2e8-1479472b9bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:35.245182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cc106-c576-4184-a2e8-1479472b9bf2 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:35.245652 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cc106-c576-4184-a2e8-1479472b9bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:35.246110 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cc106-c576-4184-a2e8-1479472b9bf2 None admin] PATH_INFO: `/v3/groups/7c33f3b0c75c4857b3281ec028418591` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:35.247556 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cc106-c576-4184-a2e8-1479472b9bf2 None admin] RBAC: Authorizing `identity:get_group(group_id=7c33f3b0c75c4857b3281ec028418591)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:35.249242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cc106-c576-4184-a2e8-1479472b9bf2 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:35.251479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 189/371] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:36:35 2022] GET /identity/v3/groups/7c33f3b0c75c4857b3281ec028418591 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:35.252551 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cc106-c576-4184-a2e8-1479472b9bf2 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:35.302092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:35.303673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7564577-9dfb-4875-85af-339dd3f45570 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:35.313742 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:35.342540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18120fd6-a113-45aa-b2e4-374947abc701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:35.343391 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18120fd6-a113-45aa-b2e4-374947abc701 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:35.343771 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18120fd6-a113-45aa-b2e4-374947abc701 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:35.344141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18120fd6-a113-45aa-b2e4-374947abc701 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:35.367105 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cc106-c576-4184-a2e8-1479472b9bf2 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:35.405703 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d4b26-96e3-4451-9d38-49044276d559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:35.406656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d4b26-96e3-4451-9d38-49044276d559 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:35.407124 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d4b26-96e3-4451-9d38-49044276d559 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:35.407557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d4b26-96e3-4451-9d38-49044276d559 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:35.423883 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18120fd6-a113-45aa-b2e4-374947abc701 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:35.426057 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18120fd6-a113-45aa-b2e4-374947abc701 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:35.482353 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d4b26-96e3-4451-9d38-49044276d559 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:35.484135 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d4b26-96e3-4451-9d38-49044276d559 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:35.526448 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:35.538817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 183/372] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:35.539723 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18120fd6-a113-45aa-b2e4-374947abc701 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:35.558702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:35.593264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 190/373] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:35.594447 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d4b26-96e3-4451-9d38-49044276d559 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:35.655233 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18120fd6-a113-45aa-b2e4-374947abc701 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:35.695081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f130f-3626-4391-9426-5d618d14abe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:35.696189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f130f-3626-4391-9426-5d618d14abe0 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:35.696756 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f130f-3626-4391-9426-5d618d14abe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:35.697238 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f130f-3626-4391-9426-5d618d14abe0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:35.698317 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f130f-3626-4391-9426-5d618d14abe0 None admin] RBAC: Adding query filter params (group.id=7c33f3b0c75c4857b3281ec028418591, role.id=fe332203b7e04d6084e26c5be292af35, scope.project.id=7c06d53d4c434698958a453f9500feaa) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:35.698877 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f130f-3626-4391-9426-5d618d14abe0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:35.700631 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f130f-3626-4391-9426-5d618d14abe0 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:35.707690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 184/374] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:36:35 2022] GET /identity/v3/role_assignments?group.id=7c33f3b0c75c4857b3281ec028418591&scope.project.id=7c06d53d4c434698958a453f9500feaa&role.id=fe332203b7e04d6084e26c5be292af35 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:35.708085 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d4b26-96e3-4451-9d38-49044276d559 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:35.709136 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f130f-3626-4391-9426-5d618d14abe0 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:35.746302 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236036e-33ea-4e5c-8cd2-91b0aac572db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:35.747120 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236036e-33ea-4e5c-8cd2-91b0aac572db None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:35.747393 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236036e-33ea-4e5c-8cd2-91b0aac572db None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:35.747657 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236036e-33ea-4e5c-8cd2-91b0aac572db None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:35.748838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236036e-33ea-4e5c-8cd2-91b0aac572db None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:35.750360 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236036e-33ea-4e5c-8cd2-91b0aac572db None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:35.752255 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 191/375] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:35 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:35.757829 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236036e-33ea-4e5c-8cd2-91b0aac572db None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:35.771942 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:35.802670 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:35.838790 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f130f-3626-4391-9426-5d618d14abe0 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:35.872869 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236036e-33ea-4e5c-8cd2-91b0aac572db None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:35.882183 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198768f0-9e98-40e8-b8d6-5b47bf9495f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:35.883468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198768f0-9e98-40e8-b8d6-5b47bf9495f7 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:35.883960 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198768f0-9e98-40e8-b8d6-5b47bf9495f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:35.884528 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198768f0-9e98-40e8-b8d6-5b47bf9495f7 None admin] PATH_INFO: `/v3/projects/526171dca7b844d9b1cd612ef681909d` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:35.886106 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198768f0-9e98-40e8-b8d6-5b47bf9495f7 None admin] RBAC: Authorizing `identity:get_project(project_id=526171dca7b844d9b1cd612ef681909d)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:35.888001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198768f0-9e98-40e8-b8d6-5b47bf9495f7 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:35.890785 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 185/376] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:35 2022] GET /identity/v3/projects/526171dca7b844d9b1cd612ef681909d => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:35.895847 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198768f0-9e98-40e8-b8d6-5b47bf9495f7 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:35.917403 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950c175-fa76-4ef1-bb4d-4cb07fd6401b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:35.917905 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950c175-fa76-4ef1-bb4d-4cb07fd6401b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:35.918178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950c175-fa76-4ef1-bb4d-4cb07fd6401b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:35.918487 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950c175-fa76-4ef1-bb4d-4cb07fd6401b None admin] PATH_INFO: `/v3/groups/7c33f3b0c75c4857b3281ec028418591` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:35.919692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1950c175-fa76-4ef1-bb4d-4cb07fd6401b None admin] RBAC: Authorizing `identity:get_group(group_id=7c33f3b0c75c4857b3281ec028418591)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:35.921164 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1950c175-fa76-4ef1-bb4d-4cb07fd6401b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:35.923182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 192/377] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:36:35 2022] GET /identity/v3/groups/7c33f3b0c75c4857b3281ec028418591 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:35.928944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950c175-fa76-4ef1-bb4d-4cb07fd6401b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:36.030965 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:36.046674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198768f0-9e98-40e8-b8d6-5b47bf9495f7 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:36.077727 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950c175-fa76-4ef1-bb4d-4cb07fd6401b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:36.114290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b4a76-3670-4fea-a862-87adc5c44173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:36.115086 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b4a76-3670-4fea-a862-87adc5c44173 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:36.115363 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b4a76-3670-4fea-a862-87adc5c44173 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:36.115630 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b4a76-3670-4fea-a862-87adc5c44173 None admin] PATH_INFO: `/v3/groups/d03b473e776e426182f51460f54c8478` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:36.131261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b4a76-3670-4fea-a862-87adc5c44173 None admin] RBAC: Authorizing `identity:get_group(group_id=d03b473e776e426182f51460f54c8478)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:36.132747 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b4a76-3670-4fea-a862-87adc5c44173 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:36.134718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 186/378] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:36:35 2022] GET /identity/v3/groups/d03b473e776e426182f51460f54c8478 => generated 229 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:36.140728 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b4a76-3670-4fea-a862-87adc5c44173 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:36.145680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:36.149450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c536b-2e1a-43f9-9b3d-c877f813f680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:36.150149 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c536b-2e1a-43f9-9b3d-c877f813f680 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:36.150450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c536b-2e1a-43f9-9b3d-c877f813f680 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:36.150792 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c536b-2e1a-43f9-9b3d-c877f813f680 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:36.151690 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c536b-2e1a-43f9-9b3d-c877f813f680 None admin] RBAC: Adding query filter params (group.id=7c33f3b0c75c4857b3281ec028418591, role.id=4a241925b6cc4218b4f2af580b151877, scope.project.id=7c06d53d4c434698958a453f9500feaa) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:36.152005 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c536b-2e1a-43f9-9b3d-c877f813f680 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:36.155036 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c536b-2e1a-43f9-9b3d-c877f813f680 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:36.166734 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 193/379] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:36:35 2022] GET /identity/v3/role_assignments?group.id=7c33f3b0c75c4857b3281ec028418591&scope.project.id=7c06d53d4c434698958a453f9500feaa&role.id=4a241925b6cc4218b4f2af580b151877 => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:36.294639 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b4a76-3670-4fea-a862-87adc5c44173 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:36.338410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bde5e2-7f07-439b-a538-5e9818513c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:36.339072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bde5e2-7f07-439b-a538-5e9818513c78 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:36.339389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bde5e2-7f07-439b-a538-5e9818513c78 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:36.339667 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bde5e2-7f07-439b-a538-5e9818513c78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:36.340501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bde5e2-7f07-439b-a538-5e9818513c78 None admin] RBAC: Adding query filter params (group.id=d03b473e776e426182f51460f54c8478, role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=526171dca7b844d9b1cd612ef681909d) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:36.340803 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bde5e2-7f07-439b-a538-5e9818513c78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:36.346400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bde5e2-7f07-439b-a538-5e9818513c78 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:36.354014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 187/380] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:36:36 2022] GET /identity/v3/role_assignments?group.id=d03b473e776e426182f51460f54c8478&scope.project.id=526171dca7b844d9b1cd612ef681909d&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:36.382984 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873fd84-d3a6-4cd4-bf19-4aa11186199c None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:36.383491 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873fd84-d3a6-4cd4-bf19-4aa11186199c None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:36.383491 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873fd84-d3a6-4cd4-bf19-4aa11186199c None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:36.385062 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 194/381] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:36.390975 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b66e9-3c43-4bb6-83d0-7ab224c6fba8 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:36.391299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b66e9-3c43-4bb6-83d0-7ab224c6fba8 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:36.391521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b66e9-3c43-4bb6-83d0-7ab224c6fba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:36.413348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-b65b66e9-3c43-4bb6-83d0-7ab224c6fba8 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:36.414602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-b65b66e9-3c43-4bb6-83d0-7ab224c6fba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:36.489715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:36.495773 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 188/382] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:36.522306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873fd84-d3a6-4cd4-bf19-4aa11186199c None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:36.654552 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873fd84-d3a6-4cd4-bf19-4aa11186199c None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:36.690857 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b697f9-fb8e-4f87-b7a5-ceea7b67c355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:36.691678 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b697f9-fb8e-4f87-b7a5-ceea7b67c355 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:36.691955 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b697f9-fb8e-4f87-b7a5-ceea7b67c355 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:36.692224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b697f9-fb8e-4f87-b7a5-ceea7b67c355 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:36.702660 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b697f9-fb8e-4f87-b7a5-ceea7b67c355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-a1b697f9-fb8e-4f87-b7a5-ceea7b67c355 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:36.711228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:36.712173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:36.712923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:36.712923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:36.712923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:36.712923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:36.712923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:36.712923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:36.712923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:36.712923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:36.712923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:36.712923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:36.712923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:36.712923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:36.712923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 195/383] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:36:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:36.718607 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b66e9-3c43-4bb6-83d0-7ab224c6fba8 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:36.859634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b66e9-3c43-4bb6-83d0-7ab224c6fba8 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:36.901694 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefe7a6-cc15-4f2f-97ab-6c78d0d2609e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:36.902765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefe7a6-cc15-4f2f-97ab-6c78d0d2609e None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:36.903263 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefe7a6-cc15-4f2f-97ab-6c78d0d2609e None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:36.903764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefe7a6-cc15-4f2f-97ab-6c78d0d2609e None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:36.904747 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefe7a6-cc15-4f2f-97ab-6c78d0d2609e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:36.905239 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefe7a6-cc15-4f2f-97ab-6c78d0d2609e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:36.906833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefe7a6-cc15-4f2f-97ab-6c78d0d2609e None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:36.917302 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 189/384] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:36.927471 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b697f9-fb8e-4f87-b7a5-ceea7b67c355 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:36.930450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:37.098972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b697f9-fb8e-4f87-b7a5-ceea7b67c355 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:37.144689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1deec5-1c79-4fda-ad02-57331fc6ea71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:37.145567 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1deec5-1c79-4fda-ad02-57331fc6ea71 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:37.145850 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1deec5-1c79-4fda-ad02-57331fc6ea71 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:37.146130 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1deec5-1c79-4fda-ad02-57331fc6ea71 None admin] PATH_INFO: `/v3/groups/7c33f3b0c75c4857b3281ec028418591` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:37.147358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1deec5-1c79-4fda-ad02-57331fc6ea71 None admin] RBAC: Authorizing `identity:get_group(group_id=7c33f3b0c75c4857b3281ec028418591)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:37.148878 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1deec5-1c79-4fda-ad02-57331fc6ea71 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:37.151530 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 196/385] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:36:36 2022] GET /identity/v3/groups/7c33f3b0c75c4857b3281ec028418591 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:37.156572 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefe7a6-cc15-4f2f-97ab-6c78d0d2609e None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:37.235677 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157b287-2a04-4480-b5d0-0d4f743094e9 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:37.235999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157b287-2a04-4480-b5d0-0d4f743094e9 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:37.236229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157b287-2a04-4480-b5d0-0d4f743094e9 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:37.238829 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 197/386] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:37.245678 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc751c-2013-4399-b2ce-5aceb9719c74 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:37.245949 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc751c-2013-4399-b2ce-5aceb9719c74 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:37.246229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc751c-2013-4399-b2ce-5aceb9719c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:37.246997 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:37.266300 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-02bc751c-2013-4399-b2ce-5aceb9719c74 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:37.267535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-02bc751c-2013-4399-b2ce-5aceb9719c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:37.295559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefe7a6-cc15-4f2f-97ab-6c78d0d2609e None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:37.343245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb36d1-e27e-46ce-9a5d-7fcba97c10f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:37.344582 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb36d1-e27e-46ce-9a5d-7fcba97c10f7 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:37.345128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb36d1-e27e-46ce-9a5d-7fcba97c10f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:37.345405 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb36d1-e27e-46ce-9a5d-7fcba97c10f7 None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:37.347560 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb36d1-e27e-46ce-9a5d-7fcba97c10f7 None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:37.349295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb36d1-e27e-46ce-9a5d-7fcba97c10f7 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:37.352384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 190/387] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:37 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:37.354103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:37.358741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb36d1-e27e-46ce-9a5d-7fcba97c10f7 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:37.359813 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 198/388] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:37.395515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc751c-2013-4399-b2ce-5aceb9719c74 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:37.470999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb36d1-e27e-46ce-9a5d-7fcba97c10f7 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:37.504687 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:37.515841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd173ede-ff90-4814-8f04-3ff4e25245ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:37.517211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd173ede-ff90-4814-8f04-3ff4e25245ed None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:37.517796 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd173ede-ff90-4814-8f04-3ff4e25245ed None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:37.518256 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd173ede-ff90-4814-8f04-3ff4e25245ed None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f/groups/7c33f3b0c75c4857b3281ec028418591/roles/4a241925b6cc4218b4f2af580b151877` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:37.521171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd173ede-ff90-4814-8f04-3ff4e25245ed None admin] RBAC: Authorizing `identity:create_grant(project_id=644a8176f5874355bc660ed98897307f, group_id=7c33f3b0c75c4857b3281ec028418591, role_id=4a241925b6cc4218b4f2af580b151877)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:37.523377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd173ede-ff90-4814-8f04-3ff4e25245ed None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:37.547057 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 191/389] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 19:36:37 2022] PUT /identity/v3/projects/644a8176f5874355bc660ed98897307f/groups/7c33f3b0c75c4857b3281ec028418591/roles/4a241925b6cc4218b4f2af580b151877 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:36:37.576718 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc751c-2013-4399-b2ce-5aceb9719c74 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:37.620472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828cf0c-9c67-4207-aa9d-7a189edbe719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:37.621306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828cf0c-9c67-4207-aa9d-7a189edbe719 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:37.621583 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828cf0c-9c67-4207-aa9d-7a189edbe719 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:37.621851 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828cf0c-9c67-4207-aa9d-7a189edbe719 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:37.653144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828cf0c-9c67-4207-aa9d-7a189edbe719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-b828cf0c-9c67-4207-aa9d-7a189edbe719 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:37.661682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:37.662687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:37.663449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:37.663449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:37.663449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:37.663449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:37.663449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:37.663449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:37.663449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:37.663449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:37.663449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:37.663449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:37.663449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:37.663449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:37.663449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 199/390] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:37.668997 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd173ede-ff90-4814-8f04-3ff4e25245ed None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:37.814997 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd173ede-ff90-4814-8f04-3ff4e25245ed None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:37.863036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9351c-991d-4fda-826b-f9c2e708926f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:37.864094 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9351c-991d-4fda-826b-f9c2e708926f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:37.864566 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9351c-991d-4fda-826b-f9c2e708926f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:37.865054 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9351c-991d-4fda-826b-f9c2e708926f None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:37.866009 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9351c-991d-4fda-826b-f9c2e708926f None admin] RBAC: Adding query filter params (name=member) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:37.866528 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9351c-991d-4fda-826b-f9c2e708926f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:37.868125 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9351c-991d-4fda-826b-f9c2e708926f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:37.877410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 192/391] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:37.883345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828cf0c-9c67-4207-aa9d-7a189edbe719 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:37.985335 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:38.032051 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828cf0c-9c67-4207-aa9d-7a189edbe719 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:38.070785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ec356-25b0-4e9d-ab8c-71b36d060473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:38.071977 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ec356-25b0-4e9d-ab8c-71b36d060473 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:38.072460 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ec356-25b0-4e9d-ab8c-71b36d060473 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:38.072916 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ec356-25b0-4e9d-ab8c-71b36d060473 None admin] PATH_INFO: `/v3/groups/7c33f3b0c75c4857b3281ec028418591` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:38.074319 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ec356-25b0-4e9d-ab8c-71b36d060473 None admin] RBAC: Authorizing `identity:get_group(group_id=7c33f3b0c75c4857b3281ec028418591)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:38.075963 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ec356-25b0-4e9d-ab8c-71b36d060473 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:38.078070 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 200/392] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:36:37 2022] GET /identity/v3/groups/7c33f3b0c75c4857b3281ec028418591 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:38.083494 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9351c-991d-4fda-826b-f9c2e708926f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:38.153773 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:38.237426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9351c-991d-4fda-826b-f9c2e708926f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:38.283944 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f5392-45d2-487d-ab33-b6ba24decc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:38.285091 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f5392-45d2-487d-ab33-b6ba24decc10 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:38.285370 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f5392-45d2-487d-ab33-b6ba24decc10 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:38.285639 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f5392-45d2-487d-ab33-b6ba24decc10 None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:38.290997 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f5392-45d2-487d-ab33-b6ba24decc10 None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:38.292473 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f5392-45d2-487d-ab33-b6ba24decc10 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:38.294520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 193/393] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:38 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:38.300079 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ec356-25b0-4e9d-ab8c-71b36d060473 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:38.415955 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ec356-25b0-4e9d-ab8c-71b36d060473 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:38.443014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:38.459932 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddba1e9-7367-42a9-856a-df14da504398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:38.460847 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddba1e9-7367-42a9-856a-df14da504398 None admin] REQUEST_METHOD: `PUT` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:38.461122 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddba1e9-7367-42a9-856a-df14da504398 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:38.461389 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddba1e9-7367-42a9-856a-df14da504398 None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f/groups/7c33f3b0c75c4857b3281ec028418591/roles/fe332203b7e04d6084e26c5be292af35` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:38.463353 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddba1e9-7367-42a9-856a-df14da504398 None admin] RBAC: Authorizing `identity:create_grant(project_id=644a8176f5874355bc660ed98897307f, group_id=7c33f3b0c75c4857b3281ec028418591, role_id=fe332203b7e04d6084e26c5be292af35)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:38.464824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddba1e9-7367-42a9-856a-df14da504398 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:38.488106 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 201/394] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 19:36:38 2022] PUT /identity/v3/projects/644a8176f5874355bc660ed98897307f/groups/7c33f3b0c75c4857b3281ec028418591/roles/fe332203b7e04d6084e26c5be292af35 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:36:38.810957 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5a5b6-96ae-4f57-9feb-564eee3eaff0 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:38.811336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5a5b6-96ae-4f57-9feb-564eee3eaff0 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:38.811526 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5a5b6-96ae-4f57-9feb-564eee3eaff0 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:38.812890 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 194/395] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:38.819984 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd3aba-26d6-4e2c-b5e2-55a6128bfbad None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:38.820299 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd3aba-26d6-4e2c-b5e2-55a6128bfbad None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:38.820558 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd3aba-26d6-4e2c-b5e2-55a6128bfbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:38.847367 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-f9fd3aba-26d6-4e2c-b5e2-55a6128bfbad None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:38.847367 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-f9fd3aba-26d6-4e2c-b5e2-55a6128bfbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:38.927492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 202/396] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:38.952554 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5a5b6-96ae-4f57-9feb-564eee3eaff0 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:39.095084 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5a5b6-96ae-4f57-9feb-564eee3eaff0 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:39.117259 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:39.138957 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90694734-9d4c-4fc3-b119-27bd04be8ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:39.139791 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90694734-9d4c-4fc3-b119-27bd04be8ed4 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:39.140068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90694734-9d4c-4fc3-b119-27bd04be8ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:39.140344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90694734-9d4c-4fc3-b119-27bd04be8ed4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:39.162323 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90694734-9d4c-4fc3-b119-27bd04be8ed4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-90694734-9d4c-4fc3-b119-27bd04be8ed4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:39.170344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:39.171245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:39.172448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:39.172448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:39.172448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:39.172448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:39.172448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:39.172448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:39.172448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:39.172448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:39.172448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:39.172448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:39.172448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:39.172448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:39.172448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 195/397] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:39.177789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd3aba-26d6-4e2c-b5e2-55a6128bfbad None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:39.313185 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd3aba-26d6-4e2c-b5e2-55a6128bfbad None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:39.339471 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:39.362073 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a3b22-2eba-4748-93ab-0f2598f7f955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:39.363492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a3b22-2eba-4748-93ab-0f2598f7f955 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:39.363631 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a3b22-2eba-4748-93ab-0f2598f7f955 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:39.363871 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a3b22-2eba-4748-93ab-0f2598f7f955 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:39.364658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a3b22-2eba-4748-93ab-0f2598f7f955 None admin] RBAC: Adding query filter params (name=member) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:39.365452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a3b22-2eba-4748-93ab-0f2598f7f955 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:39.366913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a3b22-2eba-4748-93ab-0f2598f7f955 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:39.377553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 203/398] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:39.383696 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90694734-9d4c-4fc3-b119-27bd04be8ed4 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:39.461936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582513b-db10-4922-aa75-42329429aca2 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:39.462459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582513b-db10-4922-aa75-42329429aca2 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:39.462533 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582513b-db10-4922-aa75-42329429aca2 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:39.464125 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 204/399] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:39.478556 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7e4f6-112a-47b2-92bd-a380be9b4c1e None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:39.479168 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7e4f6-112a-47b2-92bd-a380be9b4c1e None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:39.479636 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7e4f6-112a-47b2-92bd-a380be9b4c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:39.508566 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-4de7e4f6-112a-47b2-92bd-a380be9b4c1e None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:39.510129 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-4de7e4f6-112a-47b2-92bd-a380be9b4c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:39.520339 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90694734-9d4c-4fc3-b119-27bd04be8ed4 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:39.562374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fe534-e523-4982-a79a-9e41901914ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:39.563241 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fe534-e523-4982-a79a-9e41901914ee None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:39.563517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fe534-e523-4982-a79a-9e41901914ee None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:39.563802 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fe534-e523-4982-a79a-9e41901914ee None admin] PATH_INFO: `/v3/groups/7c33f3b0c75c4857b3281ec028418591` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:39.564982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5fe534-e523-4982-a79a-9e41901914ee None admin] RBAC: Authorizing `identity:get_group(group_id=7c33f3b0c75c4857b3281ec028418591)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:39.567502 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5fe534-e523-4982-a79a-9e41901914ee None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:39.569615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 196/400] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:36:39 2022] GET /identity/v3/groups/7c33f3b0c75c4857b3281ec028418591 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:39.570999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5fe42-7f01-488d-9c33-e6b850549a8b None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:39.571267 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5fe42-7f01-488d-9c33-e6b850549a8b None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:39.571532 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5fe42-7f01-488d-9c33-e6b850549a8b None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:39.572992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 197/401] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:39.574568 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5fe42-7f01-488d-9c33-e6b850549a8b None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:39.603606 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 205/402] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:39.605470 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec894823-5f10-4a2c-9c2c-5294b663340d None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:39.605972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec894823-5f10-4a2c-9c2c-5294b663340d None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:39.606499 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec894823-5f10-4a2c-9c2c-5294b663340d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:39.624457 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:39.635353 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-ec894823-5f10-4a2c-9c2c-5294b663340d None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:39.636980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-ec894823-5f10-4a2c-9c2c-5294b663340d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:39.685647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5fe42-7f01-488d-9c33-e6b850549a8b None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:39.713243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 206/403] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:39.714548 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec894823-5f10-4a2c-9c2c-5294b663340d None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:39.722654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173cf71-7a42-4e60-9e5e-818a63b02e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:39.723527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173cf71-7a42-4e60-9e5e-818a63b02e20 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:39.723806 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173cf71-7a42-4e60-9e5e-818a63b02e20 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:39.724085 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173cf71-7a42-4e60-9e5e-818a63b02e20 None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:39.725414 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e173cf71-7a42-4e60-9e5e-818a63b02e20 None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:39.727412 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e173cf71-7a42-4e60-9e5e-818a63b02e20 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:39.729588 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 198/404] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:39 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:39.735281 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173cf71-7a42-4e60-9e5e-818a63b02e20 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:39.768156 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:39.832681 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec894823-5f10-4a2c-9c2c-5294b663340d None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:39.862185 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173cf71-7a42-4e60-9e5e-818a63b02e20 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:39.873228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98e682-77c7-4854-b8bd-2aad26cff798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:39.874109 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98e682-77c7-4854-b8bd-2aad26cff798 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:39.874217 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98e682-77c7-4854-b8bd-2aad26cff798 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:39.874613 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98e682-77c7-4854-b8bd-2aad26cff798 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:39.881435 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:39.886026 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98e682-77c7-4854-b8bd-2aad26cff798 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-ee98e682-77c7-4854-b8bd-2aad26cff798 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:39.894997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:39.896121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:39.896964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:39.896964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:39.896964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:39.896964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:39.896964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:39.896964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:39.896964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:39.896964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:39.896964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:39.896964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:39.896964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:39.896964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:39.899123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 207/405] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:36:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:39.900075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98e682-77c7-4854-b8bd-2aad26cff798 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:39.907002 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de91aa4-cf03-42fd-9f9e-02ad941b9ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:39.908161 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de91aa4-cf03-42fd-9f9e-02ad941b9ad1 None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:39.908582 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de91aa4-cf03-42fd-9f9e-02ad941b9ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:39.908990 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de91aa4-cf03-42fd-9f9e-02ad941b9ad1 None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa/groups/7c33f3b0c75c4857b3281ec028418591/roles/fe332203b7e04d6084e26c5be292af35` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:39.911227 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de91aa4-cf03-42fd-9f9e-02ad941b9ad1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c06d53d4c434698958a453f9500feaa, group_id=7c33f3b0c75c4857b3281ec028418591, role_id=fe332203b7e04d6084e26c5be292af35)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:39.913481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de91aa4-cf03-42fd-9f9e-02ad941b9ad1 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:39.935273 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 199/406] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 19:36:39 2022] PUT /identity/v3/projects/7c06d53d4c434698958a453f9500feaa/groups/7c33f3b0c75c4857b3281ec028418591/roles/fe332203b7e04d6084e26c5be292af35 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:36:39.936902 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de91aa4-cf03-42fd-9f9e-02ad941b9ad1 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:40.034956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:40.039646 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98e682-77c7-4854-b8bd-2aad26cff798 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:40.074076 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de91aa4-cf03-42fd-9f9e-02ad941b9ad1 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:40.089124 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761905b-2444-40ee-85a9-aff6e53228e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:40.090150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761905b-2444-40ee-85a9-aff6e53228e2 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:40.090562 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761905b-2444-40ee-85a9-aff6e53228e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:40.090926 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761905b-2444-40ee-85a9-aff6e53228e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:40.104508 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9761905b-2444-40ee-85a9-aff6e53228e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-9761905b-2444-40ee-85a9-aff6e53228e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:40.118315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:40.119266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:40.120032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:40.120032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:40.120032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:40.120032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:40.120032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:40.120032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:40.120032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:40.120032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:40.120032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:40.120032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:40.120032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:40.120032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:40.120032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 208/407] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:36:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:40.121762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761905b-2444-40ee-85a9-aff6e53228e2 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:40.125379 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd209416-7118-474a-8aa0-2318045777dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:40.126448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd209416-7118-474a-8aa0-2318045777dc None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:40.126992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd209416-7118-474a-8aa0-2318045777dc None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:40.127460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd209416-7118-474a-8aa0-2318045777dc None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:40.128563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd209416-7118-474a-8aa0-2318045777dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:40.129144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd209416-7118-474a-8aa0-2318045777dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:40.130910 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd209416-7118-474a-8aa0-2318045777dc None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:40.141143 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 200/408] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:40.146628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd209416-7118-474a-8aa0-2318045777dc None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:40.204304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:40.305007 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761905b-2444-40ee-85a9-aff6e53228e2 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:40.309425 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd209416-7118-474a-8aa0-2318045777dc None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:40.349779 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026e78b-45ab-4756-a66f-e4a801436ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:40.350872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026e78b-45ab-4756-a66f-e4a801436ce1 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:40.351276 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026e78b-45ab-4756-a66f-e4a801436ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:40.351848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026e78b-45ab-4756-a66f-e4a801436ce1 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:40.353286 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026e78b-45ab-4756-a66f-e4a801436ce1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:40.353779 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026e78b-45ab-4756-a66f-e4a801436ce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:40.355436 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026e78b-45ab-4756-a66f-e4a801436ce1 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:40.356591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b3697-7a76-4025-ac2c-0cf37377a323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:40.357577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b3697-7a76-4025-ac2c-0cf37377a323 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:40.357947 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b3697-7a76-4025-ac2c-0cf37377a323 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:40.358358 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b3697-7a76-4025-ac2c-0cf37377a323 None admin] PATH_INFO: `/v3/groups/7c33f3b0c75c4857b3281ec028418591` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:40.359646 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b3697-7a76-4025-ac2c-0cf37377a323 None admin] RBAC: Authorizing `identity:get_group(group_id=7c33f3b0c75c4857b3281ec028418591)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:40.361178 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b3697-7a76-4025-ac2c-0cf37377a323 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:40.363899 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 201/409] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:36:40 2022] GET /identity/v3/groups/7c33f3b0c75c4857b3281ec028418591 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:40.366101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 209/410] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:40.371336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b3697-7a76-4025-ac2c-0cf37377a323 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:40.373097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026e78b-45ab-4756-a66f-e4a801436ce1 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:40.402774 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:40.485802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026e78b-45ab-4756-a66f-e4a801436ce1 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:40.551838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4405d61-56e5-452a-94e2-af9cc3949e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:40.552708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4405d61-56e5-452a-94e2-af9cc3949e4c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:40.552987 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4405d61-56e5-452a-94e2-af9cc3949e4c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:40.553257 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4405d61-56e5-452a-94e2-af9cc3949e4c None admin] PATH_INFO: `/v3/groups/d03b473e776e426182f51460f54c8478` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:40.554943 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4405d61-56e5-452a-94e2-af9cc3949e4c None admin] RBAC: Authorizing `identity:get_group(group_id=d03b473e776e426182f51460f54c8478)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:40.556410 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4405d61-56e5-452a-94e2-af9cc3949e4c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:40.567879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 210/411] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:36:40 2022] GET /identity/v3/groups/d03b473e776e426182f51460f54c8478 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:40.568843 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4405d61-56e5-452a-94e2-af9cc3949e4c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:40.636990 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b3697-7a76-4025-ac2c-0cf37377a323 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:40.651710 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:40.675526 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3aa3c3-a4b9-4d21-87f2-5cf24242e31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:40.676316 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3aa3c3-a4b9-4d21-87f2-5cf24242e31f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:40.676600 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3aa3c3-a4b9-4d21-87f2-5cf24242e31f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:40.676873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3aa3c3-a4b9-4d21-87f2-5cf24242e31f None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:40.678063 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3aa3c3-a4b9-4d21-87f2-5cf24242e31f None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:40.679694 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3aa3c3-a4b9-4d21-87f2-5cf24242e31f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:40.681465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 202/412] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:40 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:40.687327 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3aa3c3-a4b9-4d21-87f2-5cf24242e31f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:40.787966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:40.805646 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3aa3c3-a4b9-4d21-87f2-5cf24242e31f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:40.820219 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4405d61-56e5-452a-94e2-af9cc3949e4c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:40.849754 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7aeb5-049e-4ec4-b4e9-f67b86a3f2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:40.850916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7aeb5-049e-4ec4-b4e9-f67b86a3f2a0 None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:40.851195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7aeb5-049e-4ec4-b4e9-f67b86a3f2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:40.851491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7aeb5-049e-4ec4-b4e9-f67b86a3f2a0 None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa/groups/7c33f3b0c75c4857b3281ec028418591/roles/4a241925b6cc4218b4f2af580b151877` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:40.854410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7aeb5-049e-4ec4-b4e9-f67b86a3f2a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c06d53d4c434698958a453f9500feaa, group_id=7c33f3b0c75c4857b3281ec028418591, role_id=4a241925b6cc4218b4f2af580b151877)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:40.856448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7aeb5-049e-4ec4-b4e9-f67b86a3f2a0 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:40.876543 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 203/413] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 19:36:40 2022] PUT /identity/v3/projects/7c06d53d4c434698958a453f9500feaa/groups/7c33f3b0c75c4857b3281ec028418591/roles/4a241925b6cc4218b4f2af580b151877 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:36:40.878035 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520409eb-c18d-43c0-931a-7bb6709ef878 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:40.878426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520409eb-c18d-43c0-931a-7bb6709ef878 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:40.878680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520409eb-c18d-43c0-931a-7bb6709ef878 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:40.880267 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 204/414] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:40.888037 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6df2ed-063c-4701-ae80-2e7f2e238676 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:40.888402 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6df2ed-063c-4701-ae80-2e7f2e238676 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:40.888740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6df2ed-063c-4701-ae80-2e7f2e238676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:40.896845 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0fb00-bfef-40ab-8760-e61e14e8a3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:40.897739 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0fb00-bfef-40ab-8760-e61e14e8a3db None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:40.898019 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0fb00-bfef-40ab-8760-e61e14e8a3db None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:40.898376 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0fb00-bfef-40ab-8760-e61e14e8a3db None admin] PATH_INFO: `/v3/projects/526171dca7b844d9b1cd612ef681909d` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:40.903208 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0fb00-bfef-40ab-8760-e61e14e8a3db None admin] RBAC: Authorizing `identity:get_project(project_id=526171dca7b844d9b1cd612ef681909d)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:40.904780 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0fb00-bfef-40ab-8760-e61e14e8a3db None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:40.907052 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 211/415] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:40 2022] GET /identity/v3/projects/526171dca7b844d9b1cd612ef681909d => generated 344 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:40.907181 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-cb6df2ed-063c-4701-ae80-2e7f2e238676 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:40.908354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-cb6df2ed-063c-4701-ae80-2e7f2e238676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:40.913468 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0fb00-bfef-40ab-8760-e61e14e8a3db None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:40.942745 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:40.993064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 205/416] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:41.019828 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6df2ed-063c-4701-ae80-2e7f2e238676 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:41.135392 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6df2ed-063c-4701-ae80-2e7f2e238676 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:41.174133 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03315e57-c97b-4fab-92c2-e1d1ce65a52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:41.175060 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03315e57-c97b-4fab-92c2-e1d1ce65a52b None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:41.175331 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03315e57-c97b-4fab-92c2-e1d1ce65a52b None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:41.175650 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03315e57-c97b-4fab-92c2-e1d1ce65a52b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:41.188350 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03315e57-c97b-4fab-92c2-e1d1ce65a52b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-03315e57-c97b-4fab-92c2-e1d1ce65a52b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:41.198151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:41.199074 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:41.199831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:41.199831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:41.199831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:41.199831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:41.199831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:41.199831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:41.199831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:41.199831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:41.199831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:41.199831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:41.199831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:41.199831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:41.200404 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 206/417] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:36:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:36:41.200971 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0fb00-bfef-40ab-8760-e61e14e8a3db None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:41.206416 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03315e57-c97b-4fab-92c2-e1d1ce65a52b None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:41.238691 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:41.248672 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de218cf-4562-480e-b512-2c6143f7a750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:41.249635 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de218cf-4562-480e-b512-2c6143f7a750 None admin] REQUEST_METHOD: `PUT` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:41.249963 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de218cf-4562-480e-b512-2c6143f7a750 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:41.250243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de218cf-4562-480e-b512-2c6143f7a750 None admin] PATH_INFO: `/v3/projects/526171dca7b844d9b1cd612ef681909d/groups/d03b473e776e426182f51460f54c8478/roles/a33caf2301fb44db90304eb3d9afc3a5` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:41.253110 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de218cf-4562-480e-b512-2c6143f7a750 None admin] RBAC: Authorizing `identity:create_grant(project_id=526171dca7b844d9b1cd612ef681909d, group_id=d03b473e776e426182f51460f54c8478, role_id=a33caf2301fb44db90304eb3d9afc3a5)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:41.254691 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de218cf-4562-480e-b512-2c6143f7a750 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:41.278485 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 212/418] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 19:36:40 2022] PUT /identity/v3/projects/526171dca7b844d9b1cd612ef681909d/groups/d03b473e776e426182f51460f54c8478/roles/a33caf2301fb44db90304eb3d9afc3a5 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:36:41.338142 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03315e57-c97b-4fab-92c2-e1d1ce65a52b None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:41.378624 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9008efe3-e324-45a9-b803-382c0872275e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:41.379333 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008efe3-e324-45a9-b803-382c0872275e None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:41.379648 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008efe3-e324-45a9-b803-382c0872275e None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:41.379925 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008efe3-e324-45a9-b803-382c0872275e None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:41.380783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9008efe3-e324-45a9-b803-382c0872275e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:41.381089 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9008efe3-e324-45a9-b803-382c0872275e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:41.382709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9008efe3-e324-45a9-b803-382c0872275e None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:41.392908 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 207/419] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:41.399192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de218cf-4562-480e-b512-2c6143f7a750 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:41.420184 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:41.532569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de218cf-4562-480e-b512-2c6143f7a750 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:41.569293 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285222e7-1c15-4726-badb-2ab3cc233541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:41.569949 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285222e7-1c15-4726-badb-2ab3cc233541 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:41.570287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285222e7-1c15-4726-badb-2ab3cc233541 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:41.570560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285222e7-1c15-4726-badb-2ab3cc233541 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:41.607070 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b30cd-06cc-4663-834e-8991e08bba17 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:41.607378 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b30cd-06cc-4663-834e-8991e08bba17 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:41.607598 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b30cd-06cc-4663-834e-8991e08bba17 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:41.608970 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 208/420] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:36:41.624094 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635d8f1-8bea-46d6-8467-f6ab7bc66743 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:41.624396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635d8f1-8bea-46d6-8467-f6ab7bc66743 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:41.624656 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635d8f1-8bea-46d6-8467-f6ab7bc66743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:41.654475 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-c635d8f1-8bea-46d6-8467-f6ab7bc66743 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:41.655674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-c635d8f1-8bea-46d6-8467-f6ab7bc66743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:41.665316 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285222e7-1c15-4726-badb-2ab3cc233541 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:41.667263 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285222e7-1c15-4726-badb-2ab3cc233541 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:41.694496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:41.742128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 209/421] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 19:36:41.767321 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635d8f1-8bea-46d6-8467-f6ab7bc66743 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:41.783885 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:41.792374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 213/422] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:41.798335 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285222e7-1c15-4726-badb-2ab3cc233541 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:41.878996 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635d8f1-8bea-46d6-8467-f6ab7bc66743 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:41.914510 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3bc98-2b02-46ac-8f2e-accff34fd5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:41.915391 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3bc98-2b02-46ac-8f2e-accff34fd5aa None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:41.915727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3bc98-2b02-46ac-8f2e-accff34fd5aa None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:41.915995 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3bc98-2b02-46ac-8f2e-accff34fd5aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:41.930352 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3bc98-2b02-46ac-8f2e-accff34fd5aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:41.933356 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285222e7-1c15-4726-badb-2ab3cc233541 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-d6b3bc98-2b02-46ac-8f2e-accff34fd5aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:41.942307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:41.943189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:41.943983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:41.943983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:41.943983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:41.943983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:41.943983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:41.943983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:41.943983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:41.943983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:41.943983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:41.943983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:41.943983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:41.943983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:41.943983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 210/423] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:36:41.949304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3bc98-2b02-46ac-8f2e-accff34fd5aa None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:41.981249 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:42.019820 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a85d26-9aeb-472a-9246-d425d48f099c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:42.020820 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a85d26-9aeb-472a-9246-d425d48f099c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:42.021093 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a85d26-9aeb-472a-9246-d425d48f099c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:42.021405 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a85d26-9aeb-472a-9246-d425d48f099c None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:42.024468 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a85d26-9aeb-472a-9246-d425d48f099c None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:42.025941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a85d26-9aeb-472a-9246-d425d48f099c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:42.035975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 214/424] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:41 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:42.039214 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a85d26-9aeb-472a-9246-d425d48f099c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:42.097826 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:42.145985 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3bc98-2b02-46ac-8f2e-accff34fd5aa None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:42.202145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a85d26-9aeb-472a-9246-d425d48f099c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:42.207061 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba73d2f8-f9e4-4dd0-b0b1-e1146fca618b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:42.207749 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73d2f8-f9e4-4dd0-b0b1-e1146fca618b None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:42.208036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73d2f8-f9e4-4dd0-b0b1-e1146fca618b None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:42.208310 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73d2f8-f9e4-4dd0-b0b1-e1146fca618b None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:42.209070 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba73d2f8-f9e4-4dd0-b0b1-e1146fca618b None admin] RBAC: Adding query filter params (name=member) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:42.209374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba73d2f8-f9e4-4dd0-b0b1-e1146fca618b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:42.210799 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba73d2f8-f9e4-4dd0-b0b1-e1146fca618b None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:42.221738 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 211/425] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:42.227814 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba73d2f8-f9e4-4dd0-b0b1-e1146fca618b None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:42.249040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e9026-df5f-4e6a-af9c-9b6bedeec971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:42.249907 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e9026-df5f-4e6a-af9c-9b6bedeec971 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:42.250182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e9026-df5f-4e6a-af9c-9b6bedeec971 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:42.250477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e9026-df5f-4e6a-af9c-9b6bedeec971 None admin] PATH_INFO: `/v3/groups/7c33f3b0c75c4857b3281ec028418591` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:42.253277 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e9026-df5f-4e6a-af9c-9b6bedeec971 None admin] RBAC: Authorizing `identity:get_group(group_id=7c33f3b0c75c4857b3281ec028418591)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:42.254937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e9026-df5f-4e6a-af9c-9b6bedeec971 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:42.262212 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 215/426] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:36:42 2022] GET /identity/v3/groups/7c33f3b0c75c4857b3281ec028418591 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:42.263072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e9026-df5f-4e6a-af9c-9b6bedeec971 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:42.370517 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:42.421429 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e9026-df5f-4e6a-af9c-9b6bedeec971 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:42.447164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba73d2f8-f9e4-4dd0-b0b1-e1146fca618b None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:42.462142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f872c4c-5c42-4c7a-b3a3-818bccdbadbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:42.462877 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f872c4c-5c42-4c7a-b3a3-818bccdbadbe None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:42.463157 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f872c4c-5c42-4c7a-b3a3-818bccdbadbe None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:42.463473 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f872c4c-5c42-4c7a-b3a3-818bccdbadbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:42.465047 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f872c4c-5c42-4c7a-b3a3-818bccdbadbe None admin] RBAC: Adding query filter params (group.id=7c33f3b0c75c4857b3281ec028418591, role.id=4a241925b6cc4218b4f2af580b151877, scope.project.id=644a8176f5874355bc660ed98897307f) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:42.465357 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f872c4c-5c42-4c7a-b3a3-818bccdbadbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:42.467325 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f872c4c-5c42-4c7a-b3a3-818bccdbadbe None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:42.474348 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 216/427] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:36:42 2022] GET /identity/v3/role_assignments?group.id=7c33f3b0c75c4857b3281ec028418591&scope.project.id=644a8176f5874355bc660ed98897307f&role.id=4a241925b6cc4218b4f2af580b151877 => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:42.502132 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9101d09-ad01-4780-bd15-55b02b995f96 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:42.502479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9101d09-ad01-4780-bd15-55b02b995f96 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:42.502780 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9101d09-ad01-4780-bd15-55b02b995f96 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:42.505377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 217/428] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:42.512389 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa7292-8284-40da-8fdc-2c1c87bf63b8 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:42.512676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa7292-8284-40da-8fdc-2c1c87bf63b8 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:42.512943 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa7292-8284-40da-8fdc-2c1c87bf63b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:42.533071 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-82fa7292-8284-40da-8fdc-2c1c87bf63b8 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:42.534372 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-82fa7292-8284-40da-8fdc-2c1c87bf63b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:42.555010 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70242d4-2a48-4e9a-8575-fdd8f9c20138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:42.555681 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70242d4-2a48-4e9a-8575-fdd8f9c20138 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:42.555958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70242d4-2a48-4e9a-8575-fdd8f9c20138 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:42.556231 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70242d4-2a48-4e9a-8575-fdd8f9c20138 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:42.564655 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:42.636253 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 218/429] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:42.656597 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa7292-8284-40da-8fdc-2c1c87bf63b8 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:42.718669 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:42.759843 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70242d4-2a48-4e9a-8575-fdd8f9c20138 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:42.761699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70242d4-2a48-4e9a-8575-fdd8f9c20138 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:42.794832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa7292-8284-40da-8fdc-2c1c87bf63b8 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:42.838181 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d162cf-8fcb-4d49-9f3e-c81e43833b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:42.839198 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d162cf-8fcb-4d49-9f3e-c81e43833b95 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:42.839484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d162cf-8fcb-4d49-9f3e-c81e43833b95 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:42.839744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d162cf-8fcb-4d49-9f3e-c81e43833b95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:42.854053 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d162cf-8fcb-4d49-9f3e-c81e43833b95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-35d162cf-8fcb-4d49-9f3e-c81e43833b95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:42.862934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:42.863816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:42.864570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:42.864570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:42.864570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:42.864570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:42.864570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:42.864570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:42.864570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:42.864570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:42.864570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:42.864570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:42.864570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:36:42.864570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:42.865144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 219/430] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:36:42.872327 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d162cf-8fcb-4d49-9f3e-c81e43833b95 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:42.908554 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 212/431] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:42.915865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70242d4-2a48-4e9a-8575-fdd8f9c20138 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:42.964454 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:43.008912 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d162cf-8fcb-4d49-9f3e-c81e43833b95 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:43.020896 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:43.051383 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70242d4-2a48-4e9a-8575-fdd8f9c20138 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:43.053540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22890f-f051-4fa5-933f-59cc679a4a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:43.054999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22890f-f051-4fa5-933f-59cc679a4a39 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:43.055275 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22890f-f051-4fa5-933f-59cc679a4a39 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:43.055544 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22890f-f051-4fa5-933f-59cc679a4a39 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:43.056300 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22890f-f051-4fa5-933f-59cc679a4a39 None admin] RBAC: Adding query filter params (name=member) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:43.056604 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22890f-f051-4fa5-933f-59cc679a4a39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:43.057995 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22890f-f051-4fa5-933f-59cc679a4a39 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:43.067507 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 220/432] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:36:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:43.072728 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22890f-f051-4fa5-933f-59cc679a4a39 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:43.095526 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d7ffd-1e05-4775-8bbd-3872f37e0c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:43.096817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d7ffd-1e05-4775-8bbd-3872f37e0c76 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:43.097305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d7ffd-1e05-4775-8bbd-3872f37e0c76 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:43.097821 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d7ffd-1e05-4775-8bbd-3872f37e0c76 None admin] PATH_INFO: `/v3/projects/644a8176f5874355bc660ed98897307f` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:43.099244 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d7ffd-1e05-4775-8bbd-3872f37e0c76 None admin] RBAC: Authorizing `identity:get_project(project_id=644a8176f5874355bc660ed98897307f)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:43.100931 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d7ffd-1e05-4775-8bbd-3872f37e0c76 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:43.103498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 213/433] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:42 2022] GET /identity/v3/projects/644a8176f5874355bc660ed98897307f => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:43.108904 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d7ffd-1e05-4775-8bbd-3872f37e0c76 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:43.207896 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22890f-f051-4fa5-933f-59cc679a4a39 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:43.219389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d7ffd-1e05-4775-8bbd-3872f37e0c76 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:43.225165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:43.250294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146e803-743e-4ac7-a882-6946875d5efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:43.251062 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146e803-743e-4ac7-a882-6946875d5efd None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:43.251341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146e803-743e-4ac7-a882-6946875d5efd None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:43.251615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146e803-743e-4ac7-a882-6946875d5efd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:43.260452 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e73d7-12bc-4199-8867-7d739ff04c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:43.261605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e73d7-12bc-4199-8867-7d739ff04c60 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:43.262084 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e73d7-12bc-4199-8867-7d739ff04c60 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:43.262588 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e73d7-12bc-4199-8867-7d739ff04c60 None admin] PATH_INFO: `/v3/groups/7c33f3b0c75c4857b3281ec028418591` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:43.264081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e73d7-12bc-4199-8867-7d739ff04c60 None admin] RBAC: Authorizing `identity:get_group(group_id=7c33f3b0c75c4857b3281ec028418591)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:43.265734 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e73d7-12bc-4199-8867-7d739ff04c60 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:43.269659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 214/434] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:36:43 2022] GET /identity/v3/groups/7c33f3b0c75c4857b3281ec028418591 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:43.275717 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e73d7-12bc-4199-8867-7d739ff04c60 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:43.301803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:43.333303 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f146e803-743e-4ac7-a882-6946875d5efd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:43.334897 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f146e803-743e-4ac7-a882-6946875d5efd None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:43.382108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e73d7-12bc-4199-8867-7d739ff04c60 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:43.418386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774ae2d-0340-4ec5-b63d-2cdbf3640041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:43.419519 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774ae2d-0340-4ec5-b63d-2cdbf3640041 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:43.420036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774ae2d-0340-4ec5-b63d-2cdbf3640041 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:43.420678 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774ae2d-0340-4ec5-b63d-2cdbf3640041 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:43.421787 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774ae2d-0340-4ec5-b63d-2cdbf3640041 None admin] RBAC: Adding query filter params (group.id=7c33f3b0c75c4857b3281ec028418591, role.id=fe332203b7e04d6084e26c5be292af35, scope.project.id=644a8176f5874355bc660ed98897307f) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:43.422345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774ae2d-0340-4ec5-b63d-2cdbf3640041 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:43.424102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774ae2d-0340-4ec5-b63d-2cdbf3640041 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:43.432051 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 215/435] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:36:43 2022] GET /identity/v3/role_assignments?group.id=7c33f3b0c75c4857b3281ec028418591&scope.project.id=644a8176f5874355bc660ed98897307f&role.id=fe332203b7e04d6084e26c5be292af35 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:43.458820 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 221/436] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:36:43.464277 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774ae2d-0340-4ec5-b63d-2cdbf3640041 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:43.506734 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:43.583227 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774ae2d-0340-4ec5-b63d-2cdbf3640041 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:43.609897 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c900ba-c725-42e2-aa2b-c47e6e55038f None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:43.610452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c900ba-c725-42e2-aa2b-c47e6e55038f None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:43.610563 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c900ba-c725-42e2-aa2b-c47e6e55038f None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:43.612449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 222/437] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:36:43.618706 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07367fc8-177d-49ff-b134-b181bd994940 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:43.622297 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07367fc8-177d-49ff-b134-b181bd994940 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:43.622297 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07367fc8-177d-49ff-b134-b181bd994940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:43.626578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67934e-3d4b-4588-824d-14ef0529653e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:43.627555 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67934e-3d4b-4588-824d-14ef0529653e None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:43.627943 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67934e-3d4b-4588-824d-14ef0529653e None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:43.628219 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67934e-3d4b-4588-824d-14ef0529653e None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:43.629511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67934e-3d4b-4588-824d-14ef0529653e None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:43.631032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67934e-3d4b-4588-824d-14ef0529653e None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:43.633088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 216/438] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:43 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:43.639703 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67934e-3d4b-4588-824d-14ef0529653e None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:43.643568 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-07367fc8-177d-49ff-b134-b181bd994940 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:43.645885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-07367fc8-177d-49ff-b134-b181bd994940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:43.732585 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:43.737598 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 223/439] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:43.760839 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07367fc8-177d-49ff-b134-b181bd994940 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:43.774189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67934e-3d4b-4588-824d-14ef0529653e None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:43.778069 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:43.815416 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d85f5-3014-4d64-ad05-65e63eb7286d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:43.816876 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d85f5-3014-4d64-ad05-65e63eb7286d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:43.817148 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d85f5-3014-4d64-ad05-65e63eb7286d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:43.817426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d85f5-3014-4d64-ad05-65e63eb7286d None admin] PATH_INFO: `/v3/groups/7c33f3b0c75c4857b3281ec028418591` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:43.819131 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d85f5-3014-4d64-ad05-65e63eb7286d None admin] RBAC: Authorizing `identity:get_group(group_id=7c33f3b0c75c4857b3281ec028418591)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:43.821347 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d85f5-3014-4d64-ad05-65e63eb7286d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:43.823510 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 217/440] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:36:43 2022] GET /identity/v3/groups/7c33f3b0c75c4857b3281ec028418591 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:43.831588 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d85f5-3014-4d64-ad05-65e63eb7286d None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:43.887365 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07367fc8-177d-49ff-b134-b181bd994940 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:43.934308 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52a909-a05c-482d-bc89-01d8c4ab0f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:43.935539 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52a909-a05c-482d-bc89-01d8c4ab0f38 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:43.936013 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52a909-a05c-482d-bc89-01d8c4ab0f38 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:43.936472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52a909-a05c-482d-bc89-01d8c4ab0f38 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:43.953631 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52a909-a05c-482d-bc89-01d8c4ab0f38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-aa52a909-a05c-482d-bc89-01d8c4ab0f38 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:43.963207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:43.964108 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:43.964879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:43.964879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:43.964879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:43.964879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:43.964879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:43.964879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:43.964879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:43.964879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:43.964879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:43.964879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:43.964879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:43.964879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:43.964879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 224/441] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:36:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:43.968069 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d85f5-3014-4d64-ad05-65e63eb7286d None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:43.972364 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52a909-a05c-482d-bc89-01d8c4ab0f38 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:44.021673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc6070-add4-49af-a19e-e27bc792fe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:44.022635 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc6070-add4-49af-a19e-e27bc792fe95 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:44.023064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc6070-add4-49af-a19e-e27bc792fe95 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:44.023480 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc6070-add4-49af-a19e-e27bc792fe95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:44.025554 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc6070-add4-49af-a19e-e27bc792fe95 None admin] RBAC: Adding query filter params (group.id=7c33f3b0c75c4857b3281ec028418591, role.id=fe332203b7e04d6084e26c5be292af35, scope.project.id=7c06d53d4c434698958a453f9500feaa) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:44.026454 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc6070-add4-49af-a19e-e27bc792fe95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:44.029380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc6070-add4-49af-a19e-e27bc792fe95 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:44.042284 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 218/442] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:36:43 2022] GET /identity/v3/role_assignments?group.id=7c33f3b0c75c4857b3281ec028418591&scope.project.id=7c06d53d4c434698958a453f9500feaa&role.id=fe332203b7e04d6084e26c5be292af35 => generated 611 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:44.050576 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:44.091516 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317ac9b-2471-4e67-b037-a3e1089096c5 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:44.092139 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317ac9b-2471-4e67-b037-a3e1089096c5 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:44.092425 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317ac9b-2471-4e67-b037-a3e1089096c5 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:44.093982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 219/443] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:44.098429 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52a909-a05c-482d-bc89-01d8c4ab0f38 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:44.099817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2687f4-033e-4efe-9db3-2d851613767d None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:44.100276 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2687f4-033e-4efe-9db3-2d851613767d None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:44.100409 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2687f4-033e-4efe-9db3-2d851613767d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:44.122558 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-2c2687f4-033e-4efe-9db3-2d851613767d None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:44.124260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-2c2687f4-033e-4efe-9db3-2d851613767d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:44.140177 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae771fb-473a-461d-a4ef-f932ab423718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:44.141111 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae771fb-473a-461d-a4ef-f932ab423718 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:44.141636 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae771fb-473a-461d-a4ef-f932ab423718 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:44.142044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae771fb-473a-461d-a4ef-f932ab423718 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:44.142987 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae771fb-473a-461d-a4ef-f932ab423718 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:44.143541 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae771fb-473a-461d-a4ef-f932ab423718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:44.145110 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae771fb-473a-461d-a4ef-f932ab423718 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:44.155073 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 225/444] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:44.159613 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:44.163378 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae771fb-473a-461d-a4ef-f932ab423718 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:44.206332 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 220/445] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:44.229682 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2687f4-033e-4efe-9db3-2d851613767d None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:44.291567 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae771fb-473a-461d-a4ef-f932ab423718 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:44.319081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:44.335039 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f56d1-5ac3-4d13-8954-a5c15b6a7a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:44.335694 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f56d1-5ac3-4d13-8954-a5c15b6a7a60 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:44.335978 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f56d1-5ac3-4d13-8954-a5c15b6a7a60 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:44.336249 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f56d1-5ac3-4d13-8954-a5c15b6a7a60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:44.347447 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2687f4-033e-4efe-9db3-2d851613767d None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:44.388017 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b459404-8dbd-47a8-a1ba-ca37c44d0238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:44.388818 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b459404-8dbd-47a8-a1ba-ca37c44d0238 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:44.389088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b459404-8dbd-47a8-a1ba-ca37c44d0238 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:44.389355 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b459404-8dbd-47a8-a1ba-ca37c44d0238 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:44.401804 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b459404-8dbd-47a8-a1ba-ca37c44d0238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-9b459404-8dbd-47a8-a1ba-ca37c44d0238 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:44.410605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:44.411503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:44.412279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:44.412279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:44.412279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:44.412279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:44.412279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:44.412279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:44.412279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:44.412279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:44.412279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:44.412279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:44.412279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:36:44.412279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:44.412279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 221/446] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:36:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:44.419696 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b459404-8dbd-47a8-a1ba-ca37c44d0238 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:44.420707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01f56d1-5ac3-4d13-8954-a5c15b6a7a60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:44.422195 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01f56d1-5ac3-4d13-8954-a5c15b6a7a60 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:44.451171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:44.527265 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b459404-8dbd-47a8-a1ba-ca37c44d0238 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:44.532227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 226/447] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:36:44.533282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f56d1-5ac3-4d13-8954-a5c15b6a7a60 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:44.556752 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:44.565094 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa9371-3fbc-4618-a7b6-13ec7a22fb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:44.565907 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa9371-3fbc-4618-a7b6-13ec7a22fb3b None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:44.566185 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa9371-3fbc-4618-a7b6-13ec7a22fb3b None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:44.566482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa9371-3fbc-4618-a7b6-13ec7a22fb3b None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:44.567225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa9371-3fbc-4618-a7b6-13ec7a22fb3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:44.567514 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa9371-3fbc-4618-a7b6-13ec7a22fb3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:44.568890 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa9371-3fbc-4618-a7b6-13ec7a22fb3b None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:44.577323 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 222/448] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:44.585391 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa9371-3fbc-4618-a7b6-13ec7a22fb3b None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:44.636754 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f56d1-5ac3-4d13-8954-a5c15b6a7a60 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:44.670672 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d7eb5-03a8-4b08-accd-19343326b2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:44.671519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d7eb5-03a8-4b08-accd-19343326b2a8 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:44.671793 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d7eb5-03a8-4b08-accd-19343326b2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:44.672062 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d7eb5-03a8-4b08-accd-19343326b2a8 None admin] PATH_INFO: `/v3/projects/7c06d53d4c434698958a453f9500feaa` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:44.673238 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d7eb5-03a8-4b08-accd-19343326b2a8 None admin] RBAC: Authorizing `identity:get_project(project_id=7c06d53d4c434698958a453f9500feaa)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:44.674750 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d7eb5-03a8-4b08-accd-19343326b2a8 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:44.676489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 227/449] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:44 2022] GET /identity/v3/projects/7c06d53d4c434698958a453f9500feaa => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:44.679475 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:44.682046 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d7eb5-03a8-4b08-accd-19343326b2a8 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:44.693103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa9371-3fbc-4618-a7b6-13ec7a22fb3b None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:44.726646 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577d87f-8c7b-4511-be3c-72629e3d9b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:44.727616 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577d87f-8c7b-4511-be3c-72629e3d9b12 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:44.727894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577d87f-8c7b-4511-be3c-72629e3d9b12 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:44.728165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577d87f-8c7b-4511-be3c-72629e3d9b12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:44.769916 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:44.780874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d7eb5-03a8-4b08-accd-19343326b2a8 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:44.795416 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a577d87f-8c7b-4511-be3c-72629e3d9b12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:44.796883 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a577d87f-8c7b-4511-be3c-72629e3d9b12 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:44.814545 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688f63c-4c55-45ea-a5db-583747e675bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:44.815473 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688f63c-4c55-45ea-a5db-583747e675bd None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:44.815961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688f63c-4c55-45ea-a5db-583747e675bd None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:44.816235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688f63c-4c55-45ea-a5db-583747e675bd None admin] PATH_INFO: `/v3/groups/7c33f3b0c75c4857b3281ec028418591` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:44.817429 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688f63c-4c55-45ea-a5db-583747e675bd None admin] RBAC: Authorizing `identity:get_group(group_id=7c33f3b0c75c4857b3281ec028418591)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:44.818899 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688f63c-4c55-45ea-a5db-583747e675bd None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:44.820539 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 228/450] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:36:44 2022] GET /identity/v3/groups/7c33f3b0c75c4857b3281ec028418591 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:44.826224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688f63c-4c55-45ea-a5db-583747e675bd None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:44.886984 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:44.901944 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 223/451] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:44.907609 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577d87f-8c7b-4511-be3c-72629e3d9b12 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:44.936110 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688f63c-4c55-45ea-a5db-583747e675bd None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:44.972528 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff5fb5-f8b8-4759-a57e-416119701cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:44.973178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff5fb5-f8b8-4759-a57e-416119701cf5 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:44.973454 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff5fb5-f8b8-4759-a57e-416119701cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:44.973718 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff5fb5-f8b8-4759-a57e-416119701cf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:44.974572 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff5fb5-f8b8-4759-a57e-416119701cf5 None admin] RBAC: Adding query filter params (group.id=7c33f3b0c75c4857b3281ec028418591, role.id=4a241925b6cc4218b4f2af580b151877, scope.project.id=7c06d53d4c434698958a453f9500feaa) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:44.974881 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff5fb5-f8b8-4759-a57e-416119701cf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:44.976315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff5fb5-f8b8-4759-a57e-416119701cf5 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:44.982568 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 229/452] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:36:44 2022] GET /identity/v3/role_assignments?group.id=7c33f3b0c75c4857b3281ec028418591&scope.project.id=7c06d53d4c434698958a453f9500feaa&role.id=4a241925b6cc4218b4f2af580b151877 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:36:45.005573 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577d87f-8c7b-4511-be3c-72629e3d9b12 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:45.042027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca4d46-a238-44a8-badf-8b81e9a986ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:45.042945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca4d46-a238-44a8-badf-8b81e9a986ca None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:45.043226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca4d46-a238-44a8-badf-8b81e9a986ca None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:45.043495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca4d46-a238-44a8-badf-8b81e9a986ca None admin] PATH_INFO: `/v3/projects/526171dca7b844d9b1cd612ef681909d` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:45.044782 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca4d46-a238-44a8-badf-8b81e9a986ca None admin] RBAC: Authorizing `identity:get_project(project_id=526171dca7b844d9b1cd612ef681909d)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:45.046321 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca4d46-a238-44a8-badf-8b81e9a986ca None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:45.048109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 224/453] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:36:44 2022] GET /identity/v3/projects/526171dca7b844d9b1cd612ef681909d => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:45.054056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff5fb5-f8b8-4759-a57e-416119701cf5 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:45.082882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:45.169882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff5fb5-f8b8-4759-a57e-416119701cf5 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:45.205392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f1ed0-9a7b-4a0f-9099-93ad2fda63b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:45.206625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f1ed0-9a7b-4a0f-9099-93ad2fda63b3 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:45.206957 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f1ed0-9a7b-4a0f-9099-93ad2fda63b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:45.207232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f1ed0-9a7b-4a0f-9099-93ad2fda63b3 None admin] PATH_INFO: `/v3/groups/d03b473e776e426182f51460f54c8478` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:45.208424 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f1ed0-9a7b-4a0f-9099-93ad2fda63b3 None admin] RBAC: Authorizing `identity:get_group(group_id=d03b473e776e426182f51460f54c8478)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:45.209871 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f1ed0-9a7b-4a0f-9099-93ad2fda63b3 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:45.211568 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 230/454] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:36:45 2022] GET /identity/v3/groups/d03b473e776e426182f51460f54c8478 => generated 229 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:45.217454 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca4d46-a238-44a8-badf-8b81e9a986ca None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:45.276226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:45.319959 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca4d46-a238-44a8-badf-8b81e9a986ca None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:45.355047 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a93f05-5700-4495-95b0-843b770251b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:45.355659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a93f05-5700-4495-95b0-843b770251b2 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:45.355935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a93f05-5700-4495-95b0-843b770251b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:45.356203 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a93f05-5700-4495-95b0-843b770251b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:45.357030 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a93f05-5700-4495-95b0-843b770251b2 None admin] RBAC: Adding query filter params (group.id=d03b473e776e426182f51460f54c8478, role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=526171dca7b844d9b1cd612ef681909d) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:45.357335 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a93f05-5700-4495-95b0-843b770251b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:45.358794 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a93f05-5700-4495-95b0-843b770251b2 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:45.364609 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 225/455] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:36:45 2022] GET /identity/v3/role_assignments?group.id=d03b473e776e426182f51460f54c8478&scope.project.id=526171dca7b844d9b1cd612ef681909d&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:47.984455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527c849-cb0d-46a7-82fe-6370779e4999 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:47.985297 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527c849-cb0d-46a7-82fe-6370779e4999 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:47.985714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527c849-cb0d-46a7-82fe-6370779e4999 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:47.988485 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 231/456] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:47.994036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deb022-8446-4f94-b8eb-8d7eebc44f3b None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:47.994615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deb022-8446-4f94-b8eb-8d7eebc44f3b None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:47.994845 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deb022-8446-4f94-b8eb-8d7eebc44f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:48.021240 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-61deb022-8446-4f94-b8eb-8d7eebc44f3b None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:48.022468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-61deb022-8446-4f94-b8eb-8d7eebc44f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:48.111942 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 226/457] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:48.160135 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527c849-cb0d-46a7-82fe-6370779e4999 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:48.225240 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be767b4c-2c8a-44c6-bc97-49bd0f75e7e3 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:48.225874 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be767b4c-2c8a-44c6-bc97-49bd0f75e7e3 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:48.226377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be767b4c-2c8a-44c6-bc97-49bd0f75e7e3 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:48.229641 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 227/458] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:36:48.234162 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fc2a3-5b1c-4910-9c79-43ce19d85ea7 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:48.234850 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fc2a3-5b1c-4910-9c79-43ce19d85ea7 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:48.235324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fc2a3-5b1c-4910-9c79-43ce19d85ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:48.260157 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-6d4fc2a3-5b1c-4910-9c79-43ce19d85ea7 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:48.262585 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-6d4fc2a3-5b1c-4910-9c79-43ce19d85ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:48.320009 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:48.345146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527c849-cb0d-46a7-82fe-6370779e4999 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:48.355472 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:48.401257 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 228/459] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:48.405644 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073d28c-cb21-4347-8886-c7c3c88961ba None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:48.406153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073d28c-cb21-4347-8886-c7c3c88961ba None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:48.407139 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073d28c-cb21-4347-8886-c7c3c88961ba None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:48.411279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 229/460] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:48.413733 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813d136-14bd-4d2e-ac7b-f480d518b7ea None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:48.414753 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813d136-14bd-4d2e-ac7b-f480d518b7ea None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:48.416071 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813d136-14bd-4d2e-ac7b-f480d518b7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:48.426066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e352a9-2bb6-48a5-bc05-4f0c850af55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:48.428913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e352a9-2bb6-48a5-bc05-4f0c850af55d None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:48.428913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e352a9-2bb6-48a5-bc05-4f0c850af55d None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:48.429084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e352a9-2bb6-48a5-bc05-4f0c850af55d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:48.439981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e352a9-2bb6-48a5-bc05-4f0c850af55d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:48.441801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e352a9-2bb6-48a5-bc05-4f0c850af55d None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:48.442293 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-c813d136-14bd-4d2e-ac7b-f480d518b7ea None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:48.443959 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-c813d136-14bd-4d2e-ac7b-f480d518b7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:48.462294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-71e352a9-2bb6-48a5-bc05-4f0c850af55d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:48.462294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:48.462294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:48.462294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:48.462294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:48.462294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:48.462294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:48.462294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:48.462294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:48.462294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:48.462294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:48.462294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:48.462294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:36:48.462294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:36:48.462294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:48.463192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:48.464027 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:36:48.464027 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:36:48.464027 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:48.464027 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:48.468570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 232/461] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:48.469468 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e352a9-2bb6-48a5-bc05-4f0c850af55d None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:48.536077 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 230/462] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:48.537358 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813d136-14bd-4d2e-ac7b-f480d518b7ea None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:48.611967 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e352a9-2bb6-48a5-bc05-4f0c850af55d None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:48.661791 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0772f47-db28-452e-a40b-3271ad0ce122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:48.662568 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813d136-14bd-4d2e-ac7b-f480d518b7ea None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:48.663195 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0772f47-db28-452e-a40b-3271ad0ce122 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:48.663669 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0772f47-db28-452e-a40b-3271ad0ce122 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:48.664396 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0772f47-db28-452e-a40b-3271ad0ce122 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:48.675545 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:48.679641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0772f47-db28-452e-a40b-3271ad0ce122 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:48.682619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0772f47-db28-452e-a40b-3271ad0ce122 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:48.684995 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:48.703350 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30973226-6d19-42d5-b650-826dd761d6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:48.704384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30973226-6d19-42d5-b650-826dd761d6a0 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:48.704861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30973226-6d19-42d5-b650-826dd761d6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:48.705326 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30973226-6d19-42d5-b650-826dd761d6a0 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:48.706424 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30973226-6d19-42d5-b650-826dd761d6a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:48.706930 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30973226-6d19-42d5-b650-826dd761d6a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:48.707349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-d0772f47-db28-452e-a40b-3271ad0ce122 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:48.707349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:48.707349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:48.707349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:48.707349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:48.707349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:48.707349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:48.707349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:48.707349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:48.707349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:48.707349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:48.707349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:48.707349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:36:48.707349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:36:48.707349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:48.708142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:48.708994 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:36:48.708994 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:36:48.708994 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:48.708994 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:48.710940 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30973226-6d19-42d5-b650-826dd761d6a0 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:48.713412 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 233/463] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:48.714562 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0772f47-db28-452e-a40b-3271ad0ce122 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:48.729143 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 231/464] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:48.730460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30973226-6d19-42d5-b650-826dd761d6a0 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:48.825901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0772f47-db28-452e-a40b-3271ad0ce122 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:48.845407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30973226-6d19-42d5-b650-826dd761d6a0 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:48.864815 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40953c0d-6fd3-4e18-bda5-fea4a9bb8508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:48.865615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40953c0d-6fd3-4e18-bda5-fea4a9bb8508 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:48.865891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40953c0d-6fd3-4e18-bda5-fea4a9bb8508 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:48.866160 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40953c0d-6fd3-4e18-bda5-fea4a9bb8508 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:48.876621 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40953c0d-6fd3-4e18-bda5-fea4a9bb8508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:48.878428 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40953c0d-6fd3-4e18-bda5-fea4a9bb8508 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:48.882891 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2e57b-e299-42ad-88e7-a1fb8987f51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:48.883559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2e57b-e299-42ad-88e7-a1fb8987f51c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:48.883839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2e57b-e299-42ad-88e7-a1fb8987f51c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:48.884112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2e57b-e299-42ad-88e7-a1fb8987f51c None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:48.884826 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2e57b-e299-42ad-88e7-a1fb8987f51c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:48.885326 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2e57b-e299-42ad-88e7-a1fb8987f51c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:48.886508 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2e57b-e299-42ad-88e7-a1fb8987f51c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:48.897565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-40953c0d-6fd3-4e18-bda5-fea4a9bb8508 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:48.897565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:48.897565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:48.897565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:48.897565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:48.897565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:48.897565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:48.897565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:48.897565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:48.897565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:48.897565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:48.897565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:48.897565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:36:48.897565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:36:48.897565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:48.898444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:48.899268 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:36:48.899268 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:36:48.899268 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:48.899268 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:48.899268 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 234/465] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:48.900029 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40953c0d-6fd3-4e18-bda5-fea4a9bb8508 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:48.903069 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 232/466] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:48.904580 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc0e1d-841c-4611-abf4-cda899ca26dc None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:48.904580 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc0e1d-841c-4611-abf4-cda899ca26dc None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:48.905152 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc0e1d-841c-4611-abf4-cda899ca26dc None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:48.906421 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 233/467] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:48.907096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc0e1d-841c-4611-abf4-cda899ca26dc None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:48.947060 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:48.955417 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:49.015219 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40953c0d-6fd3-4e18-bda5-fea4a9bb8508 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:49.026369 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc0e1d-841c-4611-abf4-cda899ca26dc None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:49.054331 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04294a1c-a669-4b2a-a068-5301863bcc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:49.055015 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04294a1c-a669-4b2a-a068-5301863bcc31 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:49.055291 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04294a1c-a669-4b2a-a068-5301863bcc31 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:49.055566 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04294a1c-a669-4b2a-a068-5301863bcc31 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:49.056269 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04294a1c-a669-4b2a-a068-5301863bcc31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:49.057711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04294a1c-a669-4b2a-a068-5301863bcc31 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:49.072056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8d905-94af-4806-8906-d99946b45558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:49.072734 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8d905-94af-4806-8906-d99946b45558 None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:49.073018 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8d905-94af-4806-8906-d99946b45558 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:49.073337 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8d905-94af-4806-8906-d99946b45558 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:49.074051 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8d905-94af-4806-8906-d99946b45558 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:49.075600 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8d905-94af-4806-8906-d99946b45558 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:49.086704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 235/468] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:36:48 2022] POST /identity/v3/users => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:36:49.086704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04294a1c-a669-4b2a-a068-5301863bcc31 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:49.105420 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 234/469] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:36:48 2022] POST /identity/v3/users => generated 248 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:36:49.107056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4de140-1589-45d8-8c69-441b4a59af15 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:49.107456 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4de140-1589-45d8-8c69-441b4a59af15 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:49.107796 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4de140-1589-45d8-8c69-441b4a59af15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:49.127731 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-3b4de140-1589-45d8-8c69-441b4a59af15 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:49.129227 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-3b4de140-1589-45d8-8c69-441b4a59af15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:49.219463 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 235/470] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:49.245715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4de140-1589-45d8-8c69-441b4a59af15 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:49.267626 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:49.293920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04294a1c-a669-4b2a-a068-5301863bcc31 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:49.300070 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:49.389674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da2cd1-5484-4da2-aaa3-26d0fd865dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:49.390015 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da2cd1-5484-4da2-aaa3-26d0fd865dbf None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:49.390872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da2cd1-5484-4da2-aaa3-26d0fd865dbf None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:49.391158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da2cd1-5484-4da2-aaa3-26d0fd865dbf None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:49.392015 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da2cd1-5484-4da2-aaa3-26d0fd865dbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:49.392326 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da2cd1-5484-4da2-aaa3-26d0fd865dbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:49.394190 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da2cd1-5484-4da2-aaa3-26d0fd865dbf None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:49.405234 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4de140-1589-45d8-8c69-441b4a59af15 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:49.418496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 236/471] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:49.427661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da2cd1-5484-4da2-aaa3-26d0fd865dbf None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:49.476562 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c371d3-b475-422a-b9ae-ce1575236b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:49.477976 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c371d3-b475-422a-b9ae-ce1575236b2f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:49.478435 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c371d3-b475-422a-b9ae-ce1575236b2f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:49.478839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c371d3-b475-422a-b9ae-ce1575236b2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:49.495077 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c371d3-b475-422a-b9ae-ce1575236b2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:49.497055 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c371d3-b475-422a-b9ae-ce1575236b2f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:49.524744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-68c371d3-b475-422a-b9ae-ce1575236b2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:49.524744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:49.524744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:49.524744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:49.524744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:49.524744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:49.524744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:49.524744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:49.524744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:49.524744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:49.524744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:49.524744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:49.524744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:36:49.524744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:36:49.524744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:49.525727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:49.526604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:36:49.526604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:36:49.526604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:49.526604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:49.526604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 236/472] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:36:49.536069 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c371d3-b475-422a-b9ae-ce1575236b2f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:49.608935 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da2cd1-5484-4da2-aaa3-26d0fd865dbf None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:49.638358 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:49.657518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661fe38-2ce5-4582-b8a8-06ec27d8b077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:49.658545 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661fe38-2ce5-4582-b8a8-06ec27d8b077 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:49.659011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661fe38-2ce5-4582-b8a8-06ec27d8b077 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:49.659216 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661fe38-2ce5-4582-b8a8-06ec27d8b077 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:49.659846 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661fe38-2ce5-4582-b8a8-06ec27d8b077 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:49.663001 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661fe38-2ce5-4582-b8a8-06ec27d8b077 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:49.689488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c371d3-b475-422a-b9ae-ce1575236b2f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:49.691259 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 237/473] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:36:49 2022] POST /identity/v3/users => generated 248 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:36:49.693640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a2f3c-a574-4a5f-8b84-18e4c99740ec None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:49.693927 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a2f3c-a574-4a5f-8b84-18e4c99740ec None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:49.694202 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a2f3c-a574-4a5f-8b84-18e4c99740ec None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:49.695529 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 238/474] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:49.701479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bda3b-44b3-43fe-9877-5ad0d97fafab None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:49.701726 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bda3b-44b3-43fe-9877-5ad0d97fafab None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:49.702017 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bda3b-44b3-43fe-9877-5ad0d97fafab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:49.720328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-e53bda3b-44b3-43fe-9877-5ad0d97fafab None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:49.721611 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-e53bda3b-44b3-43fe-9877-5ad0d97fafab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:49.733127 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b1fa5-f387-4880-be46-769ee76d3a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:49.733810 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b1fa5-f387-4880-be46-769ee76d3a5c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:49.734139 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b1fa5-f387-4880-be46-769ee76d3a5c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:49.734448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b1fa5-f387-4880-be46-769ee76d3a5c None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:49.735186 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b1fa5-f387-4880-be46-769ee76d3a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:49.735763 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b1fa5-f387-4880-be46-769ee76d3a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:49.737547 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b1fa5-f387-4880-be46-769ee76d3a5c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:49.738164 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:49.757126 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 237/475] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:49.763348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b1fa5-f387-4880-be46-769ee76d3a5c None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:49.808412 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 239/476] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:49.829061 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bda3b-44b3-43fe-9877-5ad0d97fafab None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:49.910180 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b1fa5-f387-4880-be46-769ee76d3a5c None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:49.944385 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bda3b-44b3-43fe-9877-5ad0d97fafab None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:49.958683 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:49.974004 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63752edc-7265-4dc8-9c60-c7288462d14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:49.974564 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63752edc-7265-4dc8-9c60-c7288462d14f None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:49.974843 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63752edc-7265-4dc8-9c60-c7288462d14f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:49.975117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63752edc-7265-4dc8-9c60-c7288462d14f None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:49.976384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63752edc-7265-4dc8-9c60-c7288462d14f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:49.980179 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63752edc-7265-4dc8-9c60-c7288462d14f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:49.992004 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c159b-1774-4aba-a8da-73722eb46d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:49.992910 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c159b-1774-4aba-a8da-73722eb46d2e None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:49.993242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c159b-1774-4aba-a8da-73722eb46d2e None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:49.993655 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c159b-1774-4aba-a8da-73722eb46d2e None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:49.995131 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c159b-1774-4aba-a8da-73722eb46d2e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:49.997153 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c159b-1774-4aba-a8da-73722eb46d2e None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:50.007936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.sql.core [None req-c31c159b-1774-4aba-a8da-73722eb46d2e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 19:36:50.007936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 19:36:50.007936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [parameters: {'id': 'b0d9e6a5bc0f4ae2a93eefe18c1f57c2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 19:36:50.007936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86981) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 19:36:50.016485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 238/477] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:36:49 2022] POST /identity/v3/users => generated 249 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:36:50.027235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-c31c159b-1774-4aba-a8da-73722eb46d2e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 19:36:50.027235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:50.027235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:36:50.027235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:36:50.027235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:36:50.027235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:36:50.027235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:36:50.027235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:36:50.027235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:36:50.027235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application conn.query(q) May 19 19:36:50.027235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:36:50.027235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:36:50.027235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:36:50.027235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application result.read() May 19 19:36:50.027235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return ref.to_dict() May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:36:50.028650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application next(self.gen) May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application yield resource May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application next(self.gen) May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self.session.rollback() May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self.force_reraise() May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise self.value May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application session.commit() May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 19:36:50.029866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._prepare_impl() May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self.session.flush() May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._flush(objects) May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application compat.raise_( May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application flush_context.execute() May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rec.execute(self) May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 19:36:50.030828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:36:50.032284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application conn.query(q) May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application result.read() May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:36:50.033224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [parameters: {'id': 'b0d9e6a5bc0f4ae2a93eefe18c1f57c2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:50.034345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 19:36:50.035477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:50.038860 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 240/478] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 19:36:49 2022] POST /identity/v3/roles => generated 144 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:50.044911 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63752edc-7265-4dc8-9c60-c7288462d14f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:50.177226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63752edc-7265-4dc8-9c60-c7288462d14f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:50.216754 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6bfe2-00d2-4c8f-8442-5e0d310ce7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:50.217951 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6bfe2-00d2-4c8f-8442-5e0d310ce7b9 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:50.218510 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6bfe2-00d2-4c8f-8442-5e0d310ce7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:50.219001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6bfe2-00d2-4c8f-8442-5e0d310ce7b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:50.232754 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6bfe2-00d2-4c8f-8442-5e0d310ce7b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-b4e6bfe2-00d2-4c8f-8442-5e0d310ce7b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:50.241854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:50.242983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:50.243936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:50.243936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:50.243936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:50.243936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:50.243936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:50.243936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:50.243936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:50.243936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:50.243936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:50.243936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:50.243936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:36:50.243936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:50.245966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 239/479] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:36:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:36:50.251612 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c159b-1774-4aba-a8da-73722eb46d2e None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:50.272633 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:50.358572 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c159b-1774-4aba-a8da-73722eb46d2e None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:50.395249 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1f9df-6c06-4478-b472-484523d05602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:50.396014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1f9df-6c06-4478-b472-484523d05602 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:50.396357 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1f9df-6c06-4478-b472-484523d05602 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:50.396694 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1f9df-6c06-4478-b472-484523d05602 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:50.397524 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1f9df-6c06-4478-b472-484523d05602 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:50.397896 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1f9df-6c06-4478-b472-484523d05602 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:50.399427 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1f9df-6c06-4478-b472-484523d05602 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:50.414125 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 241/480] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:52.092508 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b1857-c06a-4e7e-a475-4fd9620ca5c2 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:52.093075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b1857-c06a-4e7e-a475-4fd9620ca5c2 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:52.093075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b1857-c06a-4e7e-a475-4fd9620ca5c2 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:52.094817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 240/481] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:52.100709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabd488-7e9b-47db-9a9f-d9e4a2e1d0b5 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:52.101243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabd488-7e9b-47db-9a9f-d9e4a2e1d0b5 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:52.101243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabd488-7e9b-47db-9a9f-d9e4a2e1d0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:52.119879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-edabd488-7e9b-47db-9a9f-d9e4a2e1d0b5 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:52.121089 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-edabd488-7e9b-47db-9a9f-d9e4a2e1d0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:52.212689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 242/482] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:52.247775 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b1857-c06a-4e7e-a475-4fd9620ca5c2 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:52.324315 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:52.352694 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6e965-51ed-484b-9a2f-3181c90d2c36 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:52.352879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6e965-51ed-484b-9a2f-3181c90d2c36 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:52.353159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6e965-51ed-484b-9a2f-3181c90d2c36 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:52.355002 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 243/483] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:52.361471 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af8b10-ac7f-459b-a353-2476d5be2811 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:52.362053 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af8b10-ac7f-459b-a353-2476d5be2811 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:52.362549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af8b10-ac7f-459b-a353-2476d5be2811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:52.374648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:52.393544 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-95af8b10-ac7f-459b-a353-2476d5be2811 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:52.396138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-95af8b10-ac7f-459b-a353-2476d5be2811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:52.421225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b1857-c06a-4e7e-a475-4fd9620ca5c2 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:52.487797 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17286d6d-5868-4f1c-9c65-347bf738fc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:52.489559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17286d6d-5868-4f1c-9c65-347bf738fc31 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:52.490203 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17286d6d-5868-4f1c-9c65-347bf738fc31 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:52.491790 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17286d6d-5868-4f1c-9c65-347bf738fc31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:52.499381 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 244/484] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:52.507648 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17286d6d-5868-4f1c-9c65-347bf738fc31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-17286d6d-5868-4f1c-9c65-347bf738fc31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:52.515503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:52.516351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:52.517164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:52.517164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:52.517164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:52.517164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:52.517164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:52.517164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:52.517164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:52.517164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:52.517164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:52.517164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:52.517164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:36:52.517164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:52.519891 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 241/485] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:36:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:52.527371 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af8b10-ac7f-459b-a353-2476d5be2811 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:52.558148 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17286d6d-5868-4f1c-9c65-347bf738fc31 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:52.646712 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af8b10-ac7f-459b-a353-2476d5be2811 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:52.683338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eab213-4d9e-4216-9e49-8e6bce60ea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:52.683663 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17286d6d-5868-4f1c-9c65-347bf738fc31 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:52.684017 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eab213-4d9e-4216-9e49-8e6bce60ea60 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:52.684302 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eab213-4d9e-4216-9e49-8e6bce60ea60 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:52.684577 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eab213-4d9e-4216-9e49-8e6bce60ea60 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:52.685470 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eab213-4d9e-4216-9e49-8e6bce60ea60 None admin] RBAC: Adding query filter params (name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:52.686004 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:52.686414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eab213-4d9e-4216-9e49-8e6bce60ea60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:52.687829 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eab213-4d9e-4216-9e49-8e6bce60ea60 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:52.689366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:52.699232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 245/486] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:36:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:52.700876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd7d36-d0c8-4764-891e-e9f4a1af8c20 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:52.701232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd7d36-d0c8-4764-891e-e9f4a1af8c20 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:52.701577 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd7d36-d0c8-4764-891e-e9f4a1af8c20 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:52.710285 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 246/487] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:52.710285 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd7d36-d0c8-4764-891e-e9f4a1af8c20 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:52.746228 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b6fb9-bddd-46b0-a5f6-7063af847bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:52.747266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b6fb9-bddd-46b0-a5f6-7063af847bb0 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:52.747956 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b6fb9-bddd-46b0-a5f6-7063af847bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:52.748308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b6fb9-bddd-46b0-a5f6-7063af847bb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:52.765348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b6fb9-bddd-46b0-a5f6-7063af847bb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-f16b6fb9-bddd-46b0-a5f6-7063af847bb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:52.776336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:52.777254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:52.778027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:52.778027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:52.778027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:52.778027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:52.778027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:52.778027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:52.778027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:52.778027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:52.778027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:52.778027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:52.778027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:36:52.778027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:52.778636 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 242/488] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:36:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:36:52.779956 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a697806-4870-439b-ace7-3958c3259400 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:52.780241 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a697806-4870-439b-ace7-3958c3259400 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:52.780515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a697806-4870-439b-ace7-3958c3259400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:52.807854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-6a697806-4870-439b-ace7-3958c3259400 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:52.809631 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-6a697806-4870-439b-ace7-3958c3259400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:52.835581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd7d36-d0c8-4764-891e-e9f4a1af8c20 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:52.877699 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc17f9-4340-4d82-ad7f-8ddce5e8b701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:52.878633 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc17f9-4340-4d82-ad7f-8ddce5e8b701 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:52.878977 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc17f9-4340-4d82-ad7f-8ddce5e8b701 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:52.879314 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc17f9-4340-4d82-ad7f-8ddce5e8b701 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:52.893914 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc17f9-4340-4d82-ad7f-8ddce5e8b701 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:52.895530 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc17f9-4340-4d82-ad7f-8ddce5e8b701 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:52.896904 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 243/489] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:52.899086 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a697806-4870-439b-ace7-3958c3259400 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:52.929337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-94dc17f9-4340-4d82-ad7f-8ddce5e8b701 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:52.929337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:52.929337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:52.929337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:52.929337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:52.929337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:52.929337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:52.929337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:52.929337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:52.929337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:52.929337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:52.929337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:52.929337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:36:52.929337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:36:52.929337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:52.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:52.932859 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:36:52.932859 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:36:52.932859 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:52.932859 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:52.932859 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 247/490] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:36:52.932859 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc17f9-4340-4d82-ad7f-8ddce5e8b701 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:52.995030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:53.022445 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:53.030975 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a697806-4870-439b-ace7-3958c3259400 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:53.050181 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc17f9-4340-4d82-ad7f-8ddce5e8b701 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:53.068841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933df13-bfb8-462d-bd4d-dca408ad437b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:53.069515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933df13-bfb8-462d-bd4d-dca408ad437b None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:53.069798 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933df13-bfb8-462d-bd4d-dca408ad437b None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:53.070506 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933df13-bfb8-462d-bd4d-dca408ad437b None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:53.071269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933df13-bfb8-462d-bd4d-dca408ad437b None admin] RBAC: Adding query filter params (name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:53.071578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933df13-bfb8-462d-bd4d-dca408ad437b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:53.072964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933df13-bfb8-462d-bd4d-dca408ad437b None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:53.082688 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 244/491] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:36:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:53.083501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933df13-bfb8-462d-bd4d-dca408ad437b None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:53.089592 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f350d2e-88d0-415b-bf8a-69e820966f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:53.090498 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f350d2e-88d0-415b-bf8a-69e820966f37 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:53.090776 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f350d2e-88d0-415b-bf8a-69e820966f37 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:53.091042 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f350d2e-88d0-415b-bf8a-69e820966f37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:53.102226 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f350d2e-88d0-415b-bf8a-69e820966f37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-1f350d2e-88d0-415b-bf8a-69e820966f37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:53.109941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:53.111014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:53.111838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:53.111838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:53.111838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:53.111838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:53.111838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:53.111838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:53.111838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:53.111838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:53.111838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:53.111838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:53.111838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:36:53.111838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:53.111838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 248/492] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:36:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:53.113369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ee97f-187f-4112-ac35-ff43e0924746 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:53.113369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ee97f-187f-4112-ac35-ff43e0924746 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:53.113369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ee97f-187f-4112-ac35-ff43e0924746 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:53.118389 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 249/493] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:53.118389 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ee97f-187f-4112-ac35-ff43e0924746 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:53.198928 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933df13-bfb8-462d-bd4d-dca408ad437b None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:53.233339 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ee97f-187f-4112-ac35-ff43e0924746 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:53.239875 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68323fe0-8650-4bd0-ad70-f28a75562c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:53.240554 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68323fe0-8650-4bd0-ad70-f28a75562c14 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:53.240985 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68323fe0-8650-4bd0-ad70-f28a75562c14 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:53.241226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68323fe0-8650-4bd0-ad70-f28a75562c14 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:53.241954 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68323fe0-8650-4bd0-ad70-f28a75562c14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:53.242290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68323fe0-8650-4bd0-ad70-f28a75562c14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:53.243401 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:53.244622 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68323fe0-8650-4bd0-ad70-f28a75562c14 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:53.259517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 245/494] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:53.260355 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68323fe0-8650-4bd0-ad70-f28a75562c14 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:53.278061 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb16edc-0ef5-44a1-9dce-0517368e225e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:53.278872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb16edc-0ef5-44a1-9dce-0517368e225e None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:53.279149 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb16edc-0ef5-44a1-9dce-0517368e225e None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:53.279426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb16edc-0ef5-44a1-9dce-0517368e225e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:53.279974 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:53.294118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb16edc-0ef5-44a1-9dce-0517368e225e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:53.295780 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb16edc-0ef5-44a1-9dce-0517368e225e None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:53.314090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-3fb16edc-0ef5-44a1-9dce-0517368e225e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:53.314090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:53.314090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:53.314090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:53.314090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:53.314090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:53.314090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:53.314090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:53.314090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:53.314090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:53.314090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:53.314090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:53.314090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:36:53.314090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:36:53.314090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:53.315025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:53.315830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:36:53.315830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:36:53.315830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:53.315830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:53.316457 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 250/495] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:53.317672 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74805f-06cd-4f28-aaf7-831b53e0664e None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:53.317959 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74805f-06cd-4f28-aaf7-831b53e0664e None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:53.318232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74805f-06cd-4f28-aaf7-831b53e0664e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:53.335632 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-3d74805f-06cd-4f28-aaf7-831b53e0664e None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:53.337063 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-3d74805f-06cd-4f28-aaf7-831b53e0664e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:53.375292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68323fe0-8650-4bd0-ad70-f28a75562c14 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:53.407926 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 251/496] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:53.408855 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74805f-06cd-4f28-aaf7-831b53e0664e None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:53.415567 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de18929-0a37-4ddf-bd63-25a88b72c175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:53.416306 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de18929-0a37-4ddf-bd63-25a88b72c175 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:53.416647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de18929-0a37-4ddf-bd63-25a88b72c175 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:53.416989 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de18929-0a37-4ddf-bd63-25a88b72c175 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:53.417796 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de18929-0a37-4ddf-bd63-25a88b72c175 None admin] RBAC: Adding query filter params (name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:53.418166 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de18929-0a37-4ddf-bd63-25a88b72c175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:53.419634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de18929-0a37-4ddf-bd63-25a88b72c175 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:53.428254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 246/497] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:36:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:53.429052 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de18929-0a37-4ddf-bd63-25a88b72c175 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:53.506645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:53.509650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:53.519451 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74805f-06cd-4f28-aaf7-831b53e0664e None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:53.527666 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de18929-0a37-4ddf-bd63-25a88b72c175 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:53.552329 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3d9b9-5a9d-4c8f-a01f-52fafa010410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:53.553148 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3d9b9-5a9d-4c8f-a01f-52fafa010410 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:53.553435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3d9b9-5a9d-4c8f-a01f-52fafa010410 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:53.553704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3d9b9-5a9d-4c8f-a01f-52fafa010410 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:53.559864 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f03e4-ecd0-4c2a-bfe6-fb98191089c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:53.560520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f03e4-ecd0-4c2a-bfe6-fb98191089c9 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:53.560799 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f03e4-ecd0-4c2a-bfe6-fb98191089c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:53.561032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3d9b9-5a9d-4c8f-a01f-52fafa010410 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:53.561125 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f03e4-ecd0-4c2a-bfe6-fb98191089c9 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:53.561778 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f03e4-ecd0-4c2a-bfe6-fb98191089c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:53.562084 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f03e4-ecd0-4c2a-bfe6-fb98191089c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:53.562531 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3d9b9-5a9d-4c8f-a01f-52fafa010410 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:53.563523 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f03e4-ecd0-4c2a-bfe6-fb98191089c9 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:53.575885 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 247/498] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:53.577128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86819679-59ad-4a00-a0ec-c7a7b1301536 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:53.577413 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86819679-59ad-4a00-a0ec-c7a7b1301536 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:53.577684 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86819679-59ad-4a00-a0ec-c7a7b1301536 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-18a3d9b9-5a9d-4c8f-a01f-52fafa010410 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:36:53.578084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:53.578945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:53.580752 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:53.580752 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:53.580752 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:36:53.580752 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:36:53.580752 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:36:53.580752 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:36:53.580752 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:36:53.580752 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:36:53.580752 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 19:36:53.580752 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:53.581250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 248/499] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:53.581250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86819679-59ad-4a00-a0ec-c7a7b1301536 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:53.581881 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 252/500] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:53.582245 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3d9b9-5a9d-4c8f-a01f-52fafa010410 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:53.682946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3d9b9-5a9d-4c8f-a01f-52fafa010410 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:53.684176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86819679-59ad-4a00-a0ec-c7a7b1301536 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:53.717923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f096ae-a9e3-49e6-a5b1-b4a8ee8ca05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:53.718839 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f096ae-a9e3-49e6-a5b1-b4a8ee8ca05b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:53.719107 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f096ae-a9e3-49e6-a5b1-b4a8ee8ca05b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:53.719366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f096ae-a9e3-49e6-a5b1-b4a8ee8ca05b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:53.719977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9076bf-ef43-49fd-95bc-555a8dc0e053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:53.720768 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9076bf-ef43-49fd-95bc-555a8dc0e053 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:53.721140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9076bf-ef43-49fd-95bc-555a8dc0e053 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:53.721552 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9076bf-ef43-49fd-95bc-555a8dc0e053 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:53.729051 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:53.730603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f096ae-a9e3-49e6-a5b1-b4a8ee8ca05b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:53.732113 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f096ae-a9e3-49e6-a5b1-b4a8ee8ca05b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:53.733500 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9076bf-ef43-49fd-95bc-555a8dc0e053 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-ac9076bf-ef43-49fd-95bc-555a8dc0e053 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:53.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:53.743095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:53.743866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:53.743866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:53.743866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:53.743866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:53.743866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:53.743866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:53.743866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:53.743866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:53.743866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:53.743866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:53.743866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:36:53.743866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:53.743866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 249/501] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:36:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:53.744510 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9076bf-ef43-49fd-95bc-555a8dc0e053 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:53.750118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-61f096ae-a9e3-49e6-a5b1-b4a8ee8ca05b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:53.750118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:53.750118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:53.750118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:53.750118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:53.750118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:53.750118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:53.750118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:53.750118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:53.750118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:53.750118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:53.750118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:53.750118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:36:53.750118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:36:53.750118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:53.750980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:53.751781 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:36:53.751781 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:36:53.751781 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:53.751781 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:53.751781 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 253/502] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:53.752598 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394c689-8005-4461-bb96-d6b715ade305 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:53.752874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394c689-8005-4461-bb96-d6b715ade305 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:53.753151 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394c689-8005-4461-bb96-d6b715ade305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:53.758706 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:53.772096 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-2394c689-8005-4461-bb96-d6b715ade305 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:53.773290 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-2394c689-8005-4461-bb96-d6b715ade305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:53.838210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 254/503] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:36:53.839286 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394c689-8005-4461-bb96-d6b715ade305 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:53.852172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9076bf-ef43-49fd-95bc-555a8dc0e053 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:53.889821 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd1a7a-2faf-4520-abbe-514d448f1972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:53.890648 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd1a7a-2faf-4520-abbe-514d448f1972 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:53.890924 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd1a7a-2faf-4520-abbe-514d448f1972 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:53.891191 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd1a7a-2faf-4520-abbe-514d448f1972 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:53.899049 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd1a7a-2faf-4520-abbe-514d448f1972 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:53.900525 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd1a7a-2faf-4520-abbe-514d448f1972 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-1bcd1a7a-2faf-4520-abbe-514d448f1972 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:36:53.915741 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:53.916632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:53.917498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:53.917498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:53.917498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:36:53.917498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:36:53.917498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:36:53.917498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:36:53.917498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:36:53.917498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:36:53.917498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:36:53.917498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:53.917498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 250/504] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:36:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:53.918276 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd1a7a-2faf-4520-abbe-514d448f1972 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:53.941354 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394c689-8005-4461-bb96-d6b715ade305 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:53.970883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:53.975568 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:53.977087 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045a79c-d902-47ca-842e-6257a2dfdb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:53.977785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045a79c-d902-47ca-842e-6257a2dfdb1b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:53.978086 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045a79c-d902-47ca-842e-6257a2dfdb1b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:53.978378 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045a79c-d902-47ca-842e-6257a2dfdb1b None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:53.979201 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045a79c-d902-47ca-842e-6257a2dfdb1b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:53.979486 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045a79c-d902-47ca-842e-6257a2dfdb1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:53.980948 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045a79c-d902-47ca-842e-6257a2dfdb1b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:54.017765 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 255/505] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 19:36:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:54.018056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd1a7a-2faf-4520-abbe-514d448f1972 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:54.018731 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045a79c-d902-47ca-842e-6257a2dfdb1b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:54.052155 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c17fa-80c2-4b84-8b28-b06f524cd785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:54.052820 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c17fa-80c2-4b84-8b28-b06f524cd785 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:54.053093 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c17fa-80c2-4b84-8b28-b06f524cd785 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:54.053358 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c17fa-80c2-4b84-8b28-b06f524cd785 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:54.054095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c17fa-80c2-4b84-8b28-b06f524cd785 None admin] RBAC: Adding query filter params (name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:54.054430 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c17fa-80c2-4b84-8b28-b06f524cd785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:54.055821 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c17fa-80c2-4b84-8b28-b06f524cd785 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:54.064111 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 251/506] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:36:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:54.065058 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c17fa-80c2-4b84-8b28-b06f524cd785 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:54.124687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045a79c-d902-47ca-842e-6257a2dfdb1b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:54.159049 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b565256-73aa-47b6-9d73-28d7f7906355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:54.159723 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b565256-73aa-47b6-9d73-28d7f7906355 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:54.159979 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b565256-73aa-47b6-9d73-28d7f7906355 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:54.160245 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b565256-73aa-47b6-9d73-28d7f7906355 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:54.160957 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b565256-73aa-47b6-9d73-28d7f7906355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:54.161254 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b565256-73aa-47b6-9d73-28d7f7906355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:54.162664 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b565256-73aa-47b6-9d73-28d7f7906355 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:54.163129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c17fa-80c2-4b84-8b28-b06f524cd785 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:54.173806 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 256/507] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:54.174718 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b565256-73aa-47b6-9d73-28d7f7906355 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:54.187155 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:54.203588 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b27d61-61cf-4439-bcae-79de0851a8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:54.204409 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b27d61-61cf-4439-bcae-79de0851a8cf None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:54.204687 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b27d61-61cf-4439-bcae-79de0851a8cf None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:54.204964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b27d61-61cf-4439-bcae-79de0851a8cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:54.215757 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b27d61-61cf-4439-bcae-79de0851a8cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:54.217305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b27d61-61cf-4439-bcae-79de0851a8cf None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:54.223994 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:54.235067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-52b27d61-61cf-4439-bcae-79de0851a8cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:54.235067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:54.235067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:54.235067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:54.235067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:54.235067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:54.235067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:54.235067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:54.235067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:54.235067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:54.235067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:54.235067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:54.235067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:36:54.235067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:36:54.235067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:54.236075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:54.237321 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:36:54.237321 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:36:54.237321 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:54.237321 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:54.237321 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 252/508] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:54.239543 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b27d61-61cf-4439-bcae-79de0851a8cf None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:54.277921 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b565256-73aa-47b6-9d73-28d7f7906355 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:54.310285 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce488f-b85e-493e-a9b4-b5903cef7dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:54.310973 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce488f-b85e-493e-a9b4-b5903cef7dc5 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:54.311250 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce488f-b85e-493e-a9b4-b5903cef7dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:54.311517 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce488f-b85e-493e-a9b4-b5903cef7dc5 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:54.312343 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce488f-b85e-493e-a9b4-b5903cef7dc5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:54.312650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce488f-b85e-493e-a9b4-b5903cef7dc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:54.314096 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce488f-b85e-493e-a9b4-b5903cef7dc5 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:54.331353 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 257/509] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:36:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:54.332116 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce488f-b85e-493e-a9b4-b5903cef7dc5 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:54.339597 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b27d61-61cf-4439-bcae-79de0851a8cf None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:54.374368 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97f045-814b-4f9a-a1b2-1b1ee748c563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:54.375012 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97f045-814b-4f9a-a1b2-1b1ee748c563 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:54.375290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97f045-814b-4f9a-a1b2-1b1ee748c563 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:54.375557 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97f045-814b-4f9a-a1b2-1b1ee748c563 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:54.416146 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:54.428767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce488f-b85e-493e-a9b4-b5903cef7dc5 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:54.438023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:54.444808 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97f045-814b-4f9a-a1b2-1b1ee748c563 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:54.446293 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97f045-814b-4f9a-a1b2-1b1ee748c563 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:54.464216 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac42855-cd3b-4733-bd4b-c6f9a3703d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:54.465007 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac42855-cd3b-4733-bd4b-c6f9a3703d24 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:54.465285 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac42855-cd3b-4733-bd4b-c6f9a3703d24 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:54.465560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac42855-cd3b-4733-bd4b-c6f9a3703d24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:54.475559 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac42855-cd3b-4733-bd4b-c6f9a3703d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:54.477334 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac42855-cd3b-4733-bd4b-c6f9a3703d24 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:54.494537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-0ac42855-cd3b-4733-bd4b-c6f9a3703d24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:54.494537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:54.494537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:54.494537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:54.494537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:54.494537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:54.494537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:54.494537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:54.494537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:54.494537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:54.494537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:54.494537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:54.494537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:36:54.494537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:36:54.494537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:54.495455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:54.496439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:36:54.496439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:36:54.496439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:54.496439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:54.496439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 258/510] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:54.496826 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac42855-cd3b-4733-bd4b-c6f9a3703d24 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:54.550208 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 253/511] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:54.551110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97f045-814b-4f9a-a1b2-1b1ee748c563 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:54.599617 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac42855-cd3b-4733-bd4b-c6f9a3703d24 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:54.622629 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:54.641850 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5692e-1c2e-4031-b442-36873603a5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:54.642741 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5692e-1c2e-4031-b442-36873603a5fd None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:54.643216 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5692e-1c2e-4031-b442-36873603a5fd None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:54.643750 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5692e-1c2e-4031-b442-36873603a5fd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:54.651553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5692e-1c2e-4031-b442-36873603a5fd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:54.653017 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5692e-1c2e-4031-b442-36873603a5fd None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:54.654718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97f045-814b-4f9a-a1b2-1b1ee748c563 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-47d5692e-1c2e-4031-b442-36873603a5fd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:36:54.667227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:54.668228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:54.669244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:54.669244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:54.669244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:36:54.669244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:36:54.669244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:36:54.669244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:36:54.669244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:36:54.669244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:36:54.669244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:36:54.669244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:54.669244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 259/512] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:36:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:54.669867 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5692e-1c2e-4031-b442-36873603a5fd None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:54.690721 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0e7f3-8869-4eb8-9e98-6bdc599678b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:54.691412 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:54.691530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0e7f3-8869-4eb8-9e98-6bdc599678b9 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:54.691640 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0e7f3-8869-4eb8-9e98-6bdc599678b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:54.691905 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0e7f3-8869-4eb8-9e98-6bdc599678b9 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:54.692687 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0e7f3-8869-4eb8-9e98-6bdc599678b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:54.692922 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0e7f3-8869-4eb8-9e98-6bdc599678b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:54.694354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0e7f3-8869-4eb8-9e98-6bdc599678b9 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:54.706350 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 254/513] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:54.706350 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0e7f3-8869-4eb8-9e98-6bdc599678b9 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:54.767737 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5692e-1c2e-4031-b442-36873603a5fd None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:54.800403 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf901c-42df-4b4f-952e-1096a877b92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:54.800974 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf901c-42df-4b4f-952e-1096a877b92f None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:54.801251 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf901c-42df-4b4f-952e-1096a877b92f None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:54.801521 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf901c-42df-4b4f-952e-1096a877b92f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:54.807047 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0e7f3-8869-4eb8-9e98-6bdc599678b9 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:54.835012 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:54.840217 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb49479-210f-4e99-9379-e9b5ab8f49a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:54.840871 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb49479-210f-4e99-9379-e9b5ab8f49a2 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:54.841147 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb49479-210f-4e99-9379-e9b5ab8f49a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:54.841413 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb49479-210f-4e99-9379-e9b5ab8f49a2 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:54.842127 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb49479-210f-4e99-9379-e9b5ab8f49a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:54.842451 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb49479-210f-4e99-9379-e9b5ab8f49a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:54.843819 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb49479-210f-4e99-9379-e9b5ab8f49a2 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:54.856278 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 255/514] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:54.857014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb49479-210f-4e99-9379-e9b5ab8f49a2 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:54.867309 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf901c-42df-4b4f-952e-1096a877b92f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:54.868783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf901c-42df-4b4f-952e-1096a877b92f None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:54.888560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:54.959006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb49479-210f-4e99-9379-e9b5ab8f49a2 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:54.975564 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 260/515] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:54.975564 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf901c-42df-4b4f-952e-1096a877b92f None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:54.995873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa07c65-7235-4c8f-9611-b89da9c3dcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:54.996668 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa07c65-7235-4c8f-9611-b89da9c3dcfc None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:54.996938 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa07c65-7235-4c8f-9611-b89da9c3dcfc None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:54.997203 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa07c65-7235-4c8f-9611-b89da9c3dcfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:55.007503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa07c65-7235-4c8f-9611-b89da9c3dcfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:55.008973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa07c65-7235-4c8f-9611-b89da9c3dcfc None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:55.026389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-2aa07c65-7235-4c8f-9611-b89da9c3dcfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:55.026389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:55.026389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:55.026389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:55.026389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:55.026389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:55.026389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:55.026389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:55.026389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:55.026389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:55.026389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:55.026389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:55.026389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:36:55.026389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:36:55.026389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:55.027376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:55.028242 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:36:55.028242 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:36:55.028242 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:55.028242 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:55.028242 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 256/516] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:55.028501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa07c65-7235-4c8f-9611-b89da9c3dcfc None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:55.072021 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:55.076242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf901c-42df-4b4f-952e-1096a877b92f None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:55.096243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:55.110465 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cdd2c2-ab1e-4757-a37b-153ab545b53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:55.111128 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdd2c2-ab1e-4757-a37b-153ab545b53d None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:55.111398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdd2c2-ab1e-4757-a37b-153ab545b53d None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:55.111719 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdd2c2-ab1e-4757-a37b-153ab545b53d None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:55.112534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cdd2c2-ab1e-4757-a37b-153ab545b53d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:55.112834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cdd2c2-ab1e-4757-a37b-153ab545b53d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:55.114273 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cdd2c2-ab1e-4757-a37b-153ab545b53d None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:55.124679 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa07c65-7235-4c8f-9611-b89da9c3dcfc None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:55.131068 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 261/517] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 19:36:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:55.131861 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cdd2c2-ab1e-4757-a37b-153ab545b53d None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:55.158392 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e03f1-4792-4b2c-afec-92b53523b9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:55.159137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e03f1-4792-4b2c-afec-92b53523b9a4 None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:55.159415 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e03f1-4792-4b2c-afec-92b53523b9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:55.159681 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e03f1-4792-4b2c-afec-92b53523b9a4 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:55.160374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e03f1-4792-4b2c-afec-92b53523b9a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:55.161865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e03f1-4792-4b2c-afec-92b53523b9a4 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:55.182212 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 257/518] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:36:55 2022] POST /identity/v3/users => generated 247 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:36:55.183052 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e03f1-4792-4b2c-afec-92b53523b9a4 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:55.246590 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cdd2c2-ab1e-4757-a37b-153ab545b53d None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:55.283070 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540ae87-eb47-4350-b07b-dce397289636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:55.283974 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540ae87-eb47-4350-b07b-dce397289636 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:55.284324 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540ae87-eb47-4350-b07b-dce397289636 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:55.284551 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540ae87-eb47-4350-b07b-dce397289636 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:55.292461 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540ae87-eb47-4350-b07b-dce397289636 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:55.293936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540ae87-eb47-4350-b07b-dce397289636 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:55.308371 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-4540ae87-eb47-4350-b07b-dce397289636 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:36:55.309602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:55.310740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:55.316830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:55.316830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:55.316830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:36:55.316830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:36:55.316830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:36:55.316830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:36:55.316830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:36:55.316830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:36:55.316830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:36:55.316830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:55.316830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 262/519] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:36:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:55.316830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540ae87-eb47-4350-b07b-dce397289636 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:55.320465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e03f1-4792-4b2c-afec-92b53523b9a4 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:55.358065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:55.361669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef7654-496a-4b27-a85a-759ccfd74a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:55.362589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef7654-496a-4b27-a85a-759ccfd74a99 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:55.362858 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef7654-496a-4b27-a85a-759ccfd74a99 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:55.363128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef7654-496a-4b27-a85a-759ccfd74a99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:55.375037 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef7654-496a-4b27-a85a-759ccfd74a99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:55.377057 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef7654-496a-4b27-a85a-759ccfd74a99 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:55.396083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-95ef7654-496a-4b27-a85a-759ccfd74a99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:55.396083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:55.396083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:55.396083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:55.396083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:55.396083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:55.396083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:55.396083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:55.396083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:55.396083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:55.396083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:55.396083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:55.396083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:36:55.396083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:36:55.396083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:55.396952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:55.398092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:36:55.398092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:36:55.398092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:55.398092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:55.398092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 258/520] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:55.398470 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef7654-496a-4b27-a85a-759ccfd74a99 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:55.433150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540ae87-eb47-4350-b07b-dce397289636 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:55.466526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9bfe6-96f3-4fa2-9188-65f219e3f546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:55.467238 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9bfe6-96f3-4fa2-9188-65f219e3f546 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:55.467533 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9bfe6-96f3-4fa2-9188-65f219e3f546 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:55.467796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9bfe6-96f3-4fa2-9188-65f219e3f546 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:55.468492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9bfe6-96f3-4fa2-9188-65f219e3f546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:55.468791 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9bfe6-96f3-4fa2-9188-65f219e3f546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:55.470169 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9bfe6-96f3-4fa2-9188-65f219e3f546 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:55.486174 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 263/521] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:55.486174 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9bfe6-96f3-4fa2-9188-65f219e3f546 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:55.508620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef7654-496a-4b27-a85a-759ccfd74a99 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:55.543235 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3960d-1c0c-4c13-96b6-6f2999b50321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:55.543884 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3960d-1c0c-4c13-96b6-6f2999b50321 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:55.544197 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3960d-1c0c-4c13-96b6-6f2999b50321 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:55.544428 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3960d-1c0c-4c13-96b6-6f2999b50321 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:55.565963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:55.585630 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:55.590758 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9bfe6-96f3-4fa2-9188-65f219e3f546 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:55.613930 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3960d-1c0c-4c13-96b6-6f2999b50321 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:55.615366 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3960d-1c0c-4c13-96b6-6f2999b50321 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:55.637442 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b7186-961d-4459-afeb-845d9a8cbbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:55.638112 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b7186-961d-4459-afeb-845d9a8cbbb7 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:55.638416 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b7186-961d-4459-afeb-845d9a8cbbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:55.638686 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b7186-961d-4459-afeb-845d9a8cbbb7 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:55.639521 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b7186-961d-4459-afeb-845d9a8cbbb7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:55.639826 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b7186-961d-4459-afeb-845d9a8cbbb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:55.641277 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b7186-961d-4459-afeb-845d9a8cbbb7 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:55.659616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 264/522] 10.222.0.84 () {56 vars in 1139 bytes} [Thu May 19 19:36:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:55.660515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b7186-961d-4459-afeb-845d9a8cbbb7 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:55.730419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 259/523] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:55.731779 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3960d-1c0c-4c13-96b6-6f2999b50321 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:55.763914 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b7186-961d-4459-afeb-845d9a8cbbb7 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:55.787626 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:55.799380 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc1a15-2d57-4d58-a5a8-7261ee2116fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:55.800002 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc1a15-2d57-4d58-a5a8-7261ee2116fe None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:55.800275 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc1a15-2d57-4d58-a5a8-7261ee2116fe None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:55.800553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc1a15-2d57-4d58-a5a8-7261ee2116fe None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:55.801254 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc1a15-2d57-4d58-a5a8-7261ee2116fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:55.801549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc1a15-2d57-4d58-a5a8-7261ee2116fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:55.802953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc1a15-2d57-4d58-a5a8-7261ee2116fe None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:55.814689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 265/524] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:55.815856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc1a15-2d57-4d58-a5a8-7261ee2116fe None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:55.833201 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:55.846965 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3960d-1c0c-4c13-96b6-6f2999b50321 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:55.882139 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f39aa-4491-4843-ae27-dd3fc392b457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:55.882995 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f39aa-4491-4843-ae27-dd3fc392b457 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:55.883339 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f39aa-4491-4843-ae27-dd3fc392b457 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:55.883679 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f39aa-4491-4843-ae27-dd3fc392b457 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:55.890774 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f39aa-4491-4843-ae27-dd3fc392b457 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:55.892307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f39aa-4491-4843-ae27-dd3fc392b457 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:55.906888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-508f39aa-4491-4843-ae27-dd3fc392b457 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:36:55.906888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:55.906888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:55.906888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:55.906888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:55.906888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:55.906888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:55.906888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:55.906888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:55.906888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:55.906888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:55.906888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:55.906888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:36:55.906888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:36:55.906888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:55.907964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:55.909044 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:36:55.909044 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:36:55.909044 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:36:55.909044 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:36:55.909044 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:36:55.909044 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:36:55.909044 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:36:55.909044 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:55.909847 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 260/525] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:55.910710 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f39aa-4491-4843-ae27-dd3fc392b457 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:55.926638 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc1a15-2d57-4d58-a5a8-7261ee2116fe None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:55.962912 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07700902-7a62-4dd5-99e0-ebf4b9a2033f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:55.963861 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07700902-7a62-4dd5-99e0-ebf4b9a2033f None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:55.964134 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07700902-7a62-4dd5-99e0-ebf4b9a2033f None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:55.964412 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07700902-7a62-4dd5-99e0-ebf4b9a2033f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:56.016329 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f39aa-4491-4843-ae27-dd3fc392b457 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:56.029897 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:56.040334 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07700902-7a62-4dd5-99e0-ebf4b9a2033f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:56.042435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07700902-7a62-4dd5-99e0-ebf4b9a2033f None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:56.052475 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3446b-cce1-444d-a6a6-aad987d8e478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:56.053360 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3446b-cce1-444d-a6a6-aad987d8e478 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:56.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3446b-cce1-444d-a6a6-aad987d8e478 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:56.054046 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3446b-cce1-444d-a6a6-aad987d8e478 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:56.062752 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:56.066132 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3446b-cce1-444d-a6a6-aad987d8e478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:56.067727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3446b-cce1-444d-a6a6-aad987d8e478 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:56.085223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-7de3446b-cce1-444d-a6a6-aad987d8e478 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:56.085223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:56.085223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:56.085223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:56.085223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:56.085223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:56.085223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:56.085223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:56.085223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:56.085223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:56.085223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:56.085223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:56.085223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:36:56.085223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:36:56.085223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:56.086354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:56.087389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:36:56.087389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:36:56.087389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:56.087389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:56.088252 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 261/526] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:56.089024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3446b-cce1-444d-a6a6-aad987d8e478 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:56.157834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 266/527] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:56.158849 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07700902-7a62-4dd5-99e0-ebf4b9a2033f None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:56.190100 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3446b-cce1-444d-a6a6-aad987d8e478 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:56.223566 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bf31c-1583-485c-a8e6-dad0b92f7366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:56.224359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bf31c-1583-485c-a8e6-dad0b92f7366 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:56.224642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bf31c-1583-485c-a8e6-dad0b92f7366 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:56.224921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bf31c-1583-485c-a8e6-dad0b92f7366 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:56.232648 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bf31c-1583-485c-a8e6-dad0b92f7366 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:56.234101 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bf31c-1583-485c-a8e6-dad0b92f7366 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:56.248396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-707bf31c-1583-485c-a8e6-dad0b92f7366 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:36:56.248396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:56.248396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:56.248396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:56.248396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:56.248396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:56.248396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:56.248396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:56.248396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:56.248396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:56.248396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:56.248396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:56.248396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:36:56.248396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:36:56.248396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:56.249260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:56.250108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:36:56.250108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:36:56.250108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:36:56.250108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:36:56.250108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:36:56.250108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:36:56.250108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:36:56.250108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:56.250108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 262/528] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:56.251045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bf31c-1583-485c-a8e6-dad0b92f7366 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:56.268418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07700902-7a62-4dd5-99e0-ebf4b9a2033f None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:56.277848 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:56.287032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:56.305645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a1bfb-6853-4c63-96c6-5b833b48cb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:56.306473 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a1bfb-6853-4c63-96c6-5b833b48cb34 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:56.306844 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a1bfb-6853-4c63-96c6-5b833b48cb34 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:56.307204 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a1bfb-6853-4c63-96c6-5b833b48cb34 None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:56.308080 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a1bfb-6853-4c63-96c6-5b833b48cb34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:56.308473 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a1bfb-6853-4c63-96c6-5b833b48cb34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:56.310010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a1bfb-6853-4c63-96c6-5b833b48cb34 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:56.334870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 267/529] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:36:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:56.334870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a1bfb-6853-4c63-96c6-5b833b48cb34 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:56.363904 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bf31c-1583-485c-a8e6-dad0b92f7366 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:56.400129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb52091-e66b-4e10-9bca-f133f8845e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:56.401090 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb52091-e66b-4e10-9bca-f133f8845e05 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:56.401374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb52091-e66b-4e10-9bca-f133f8845e05 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:56.401646 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb52091-e66b-4e10-9bca-f133f8845e05 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:56.402406 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb52091-e66b-4e10-9bca-f133f8845e05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:56.402858 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb52091-e66b-4e10-9bca-f133f8845e05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:56.404727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb52091-e66b-4e10-9bca-f133f8845e05 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:56.417080 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 263/530] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:56.417650 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb52091-e66b-4e10-9bca-f133f8845e05 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:56.442980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a1bfb-6853-4c63-96c6-5b833b48cb34 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:56.479507 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b359e-daf1-4dd7-9061-9ad5bef7cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:56.480398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b359e-daf1-4dd7-9061-9ad5bef7cb3e None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:56.480667 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b359e-daf1-4dd7-9061-9ad5bef7cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:56.480928 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b359e-daf1-4dd7-9061-9ad5bef7cb3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:56.491340 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b359e-daf1-4dd7-9061-9ad5bef7cb3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:56.493033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b359e-daf1-4dd7-9061-9ad5bef7cb3e None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:56.507635 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:56.510492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-cf7b359e-daf1-4dd7-9061-9ad5bef7cb3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:56.510492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:56.510492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:56.510492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:56.510492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:56.510492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:56.510492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:56.510492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:56.510492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:56.510492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:56.510492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:56.510492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:56.510492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:36:56.510492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:36:56.510492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:56.511607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:56.512446 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:36:56.512446 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:36:56.512446 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:56.512446 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:56.512446 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 268/531] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:56.512945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b359e-daf1-4dd7-9061-9ad5bef7cb3e None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:56.531312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb52091-e66b-4e10-9bca-f133f8845e05 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:56.541411 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:56.567187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0c938-614a-4f60-86c1-6a1c29d2bc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:56.567964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0c938-614a-4f60-86c1-6a1c29d2bc7f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:56.568643 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0c938-614a-4f60-86c1-6a1c29d2bc7f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:56.568916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0c938-614a-4f60-86c1-6a1c29d2bc7f None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:56.569707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0c938-614a-4f60-86c1-6a1c29d2bc7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:56.570013 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0c938-614a-4f60-86c1-6a1c29d2bc7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:56.571490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0c938-614a-4f60-86c1-6a1c29d2bc7f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:56.594202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 264/532] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:36:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:56.594202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0c938-614a-4f60-86c1-6a1c29d2bc7f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:56.624347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b359e-daf1-4dd7-9061-9ad5bef7cb3e None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:56.660175 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da98426-1377-4785-9cf6-3a8f4557324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:56.660921 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da98426-1377-4785-9cf6-3a8f4557324c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:56.661295 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da98426-1377-4785-9cf6-3a8f4557324c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:56.661672 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da98426-1377-4785-9cf6-3a8f4557324c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:56.662624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da98426-1377-4785-9cf6-3a8f4557324c None admin] RBAC: Adding query filter params (role.id=cefdc5c73ec347438e5b4687f191382c, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=3029e939ee1c4569ba877d05547faa65) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:56.663046 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da98426-1377-4785-9cf6-3a8f4557324c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:56.664576 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da98426-1377-4785-9cf6-3a8f4557324c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:56.670418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 269/533] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:56 2022] GET /identity/v3/role_assignments?user.id=3029e939ee1c4569ba877d05547faa65&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=cefdc5c73ec347438e5b4687f191382c => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:56.671044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da98426-1377-4785-9cf6-3a8f4557324c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:56.721536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0c938-614a-4f60-86c1-6a1c29d2bc7f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:56.775429 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:56.778545 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58228419-f26e-4c86-8f2e-746331227043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:56.779919 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58228419-f26e-4c86-8f2e-746331227043 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:56.780353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58228419-f26e-4c86-8f2e-746331227043 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:56.781047 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58228419-f26e-4c86-8f2e-746331227043 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:56.787306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da98426-1377-4785-9cf6-3a8f4557324c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:56.793031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58228419-f26e-4c86-8f2e-746331227043 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:56.794781 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58228419-f26e-4c86-8f2e-746331227043 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:56.816935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-58228419-f26e-4c86-8f2e-746331227043 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:36:56.816935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:56.816935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:56.816935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:56.816935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:56.816935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:56.816935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:56.816935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:56.816935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:56.816935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:56.816935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:56.816935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:56.816935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:36:56.816935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:36:56.816935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:56.818032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:56.819087 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:36:56.819087 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:36:56.819087 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:36:56.819087 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:36:56.819087 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:36:56.819087 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:36:56.819087 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:36:56.819087 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:56.822122 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 265/534] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:56.823217 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58228419-f26e-4c86-8f2e-746331227043 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:56.827784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:56.839496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e7d68-aa0c-4315-a1e5-d40098e93340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:56.840437 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e7d68-aa0c-4315-a1e5-d40098e93340 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:56.841345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e7d68-aa0c-4315-a1e5-d40098e93340 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:56.841773 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e7d68-aa0c-4315-a1e5-d40098e93340 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:56.843189 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e7d68-aa0c-4315-a1e5-d40098e93340 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:56.843652 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e7d68-aa0c-4315-a1e5-d40098e93340 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:56.845565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e7d68-aa0c-4315-a1e5-d40098e93340 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:56.863758 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 270/535] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:56.864959 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e7d68-aa0c-4315-a1e5-d40098e93340 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:56.974217 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58228419-f26e-4c86-8f2e-746331227043 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:56.990686 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e7d68-aa0c-4315-a1e5-d40098e93340 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:57.022575 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee7cef-b3ba-4a8c-977a-fee94db7f367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:57.023242 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee7cef-b3ba-4a8c-977a-fee94db7f367 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:57.023524 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee7cef-b3ba-4a8c-977a-fee94db7f367 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:57.023798 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee7cef-b3ba-4a8c-977a-fee94db7f367 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:57.024813 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee7cef-b3ba-4a8c-977a-fee94db7f367 None admin] RBAC: Adding query filter params (role.id=cefdc5c73ec347438e5b4687f191382c, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=abde7cc02fae4d9bb561a6642c5252a6) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:57.025128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee7cef-b3ba-4a8c-977a-fee94db7f367 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:57.027205 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee7cef-b3ba-4a8c-977a-fee94db7f367 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:57.038357 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 266/536] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:56 2022] GET /identity/v3/role_assignments?user.id=abde7cc02fae4d9bb561a6642c5252a6&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=cefdc5c73ec347438e5b4687f191382c => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:57.039632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee7cef-b3ba-4a8c-977a-fee94db7f367 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:57.047954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e95a0-99e3-47f6-bb6a-e3aadaa6e7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:57.049043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e95a0-99e3-47f6-bb6a-e3aadaa6e7db None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:57.049756 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e95a0-99e3-47f6-bb6a-e3aadaa6e7db None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:57.053868 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e95a0-99e3-47f6-bb6a-e3aadaa6e7db None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:57.055131 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e95a0-99e3-47f6-bb6a-e3aadaa6e7db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:57.055645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e95a0-99e3-47f6-bb6a-e3aadaa6e7db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:57.057360 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e95a0-99e3-47f6-bb6a-e3aadaa6e7db None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:57.082291 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 271/537] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:36:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:57.088382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e95a0-99e3-47f6-bb6a-e3aadaa6e7db None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:57.118073 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:57.191740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:57.253145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e95a0-99e3-47f6-bb6a-e3aadaa6e7db None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:57.262445 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee7cef-b3ba-4a8c-977a-fee94db7f367 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:57.317118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e69a4b-d2db-4180-8d75-176523290136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:57.318131 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e69a4b-d2db-4180-8d75-176523290136 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:57.318482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e69a4b-d2db-4180-8d75-176523290136 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:57.318807 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e69a4b-d2db-4180-8d75-176523290136 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:57.323729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d885722-e8f9-4ac9-9715-dea076649f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:57.324681 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d885722-e8f9-4ac9-9715-dea076649f8f None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:57.325182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d885722-e8f9-4ac9-9715-dea076649f8f None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:57.325651 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d885722-e8f9-4ac9-9715-dea076649f8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:57.334298 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e69a4b-d2db-4180-8d75-176523290136 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:57.334298 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e69a4b-d2db-4180-8d75-176523290136 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:57.335129 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d885722-e8f9-4ac9-9715-dea076649f8f None admin] RBAC: Adding query filter params (role.id=cefdc5c73ec347438e5b4687f191382c, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=423cd0a1227940d1bd7e4263348f45c9) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:57.335672 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d885722-e8f9-4ac9-9715-dea076649f8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:57.337357 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d885722-e8f9-4ac9-9715-dea076649f8f None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:57.346003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-e7e69a4b-d2db-4180-8d75-176523290136 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:36:57.346003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:57.346003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:57.346003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:57.346003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:57.346003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:57.346003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:57.346003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:57.346003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:57.346003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:57.346003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:57.346003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:57.346003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:36:57.346003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:36:57.346003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:57.346983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:57.347943 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:36:57.347943 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:36:57.347943 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:36:57.347943 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:36:57.347943 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:36:57.347943 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:36:57.347943 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:36:57.347943 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:57.348343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 267/538] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:36:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:57.352465 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 272/539] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:57 2022] GET /identity/v3/role_assignments?user.id=423cd0a1227940d1bd7e4263348f45c9&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=cefdc5c73ec347438e5b4687f191382c => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:57.355271 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e69a4b-d2db-4180-8d75-176523290136 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:57.511761 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e69a4b-d2db-4180-8d75-176523290136 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:57.552585 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:57.561857 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d32e93-72cd-4129-b490-3f232a6669ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:57.562550 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d32e93-72cd-4129-b490-3f232a6669ed None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:57.562832 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d32e93-72cd-4129-b490-3f232a6669ed None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:57.563105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d32e93-72cd-4129-b490-3f232a6669ed None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:57.563895 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d32e93-72cd-4129-b490-3f232a6669ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:57.564202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d32e93-72cd-4129-b490-3f232a6669ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:57.565809 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d32e93-72cd-4129-b490-3f232a6669ed None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:57.578697 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 268/540] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:36:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:57.584467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d885722-e8f9-4ac9-9715-dea076649f8f None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:57.637146 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc1ec5-91b1-4560-a082-27b8f8a9bfd6 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:57.637428 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc1ec5-91b1-4560-a082-27b8f8a9bfd6 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:57.637699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc1ec5-91b1-4560-a082-27b8f8a9bfd6 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:57.639409 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 269/541] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:57.644397 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170a23e-2bae-4a42-95be-1bb1baf29f98 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:57.644688 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170a23e-2bae-4a42-95be-1bb1baf29f98 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:57.644952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170a23e-2bae-4a42-95be-1bb1baf29f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:57.679399 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-2170a23e-2bae-4a42-95be-1bb1baf29f98 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:57.680606 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-2170a23e-2bae-4a42-95be-1bb1baf29f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:57.695257 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:57.721346 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d885722-e8f9-4ac9-9715-dea076649f8f None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:57.762088 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a4fd2-491a-40bf-915b-fb58f53379ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:57.762705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a4fd2-491a-40bf-915b-fb58f53379ba None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:57.762982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a4fd2-491a-40bf-915b-fb58f53379ba None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:57.763254 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a4fd2-491a-40bf-915b-fb58f53379ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:57.764091 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a4fd2-491a-40bf-915b-fb58f53379ba None admin] RBAC: Adding query filter params (role.id=cefdc5c73ec347438e5b4687f191382c, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=d2b98b1e4024410a87addde112237f32) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:57.764399 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a4fd2-491a-40bf-915b-fb58f53379ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:57.765869 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a4fd2-491a-40bf-915b-fb58f53379ba None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:57.773551 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 273/542] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:36:57 2022] GET /identity/v3/role_assignments?user.id=d2b98b1e4024410a87addde112237f32&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=cefdc5c73ec347438e5b4687f191382c => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:57.802460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 270/543] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:36:57.846594 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a4fd2-491a-40bf-915b-fb58f53379ba None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:57.974139 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a4fd2-491a-40bf-915b-fb58f53379ba None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:58.020994 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa08a4-4534-4f43-a8e4-f5829642035d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:58.021748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa08a4-4534-4f43-a8e4-f5829642035d None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:58.022030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa08a4-4534-4f43-a8e4-f5829642035d None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:58.022328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa08a4-4534-4f43-a8e4-f5829642035d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:58.031684 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:58.038456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa08a4-4534-4f43-a8e4-f5829642035d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-28fa08a4-4534-4f43-a8e4-f5829642035d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:58.052482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:58.053422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:58.054179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:58.054179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:58.054179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:58.054179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:58.054179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:58.054179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:58.054179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:58.054179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:58.054179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:58.054179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:58.054179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:36:58.054179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:58.054179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 274/544] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:36:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:58.061802 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170a23e-2bae-4a42-95be-1bb1baf29f98 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:58.184196 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170a23e-2bae-4a42-95be-1bb1baf29f98 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:58.202030 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:58.219124 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c3cd3-b006-4475-b5ff-b5df54e7e654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:58.219797 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c3cd3-b006-4475-b5ff-b5df54e7e654 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:58.220083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c3cd3-b006-4475-b5ff-b5df54e7e654 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:58.220356 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c3cd3-b006-4475-b5ff-b5df54e7e654 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:58.221148 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c3cd3-b006-4475-b5ff-b5df54e7e654 None admin] RBAC: Adding query filter params (name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:58.221453 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c3cd3-b006-4475-b5ff-b5df54e7e654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:58.222863 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c3cd3-b006-4475-b5ff-b5df54e7e654 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:58.231618 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 271/545] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:36:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:58.237012 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa08a4-4534-4f43-a8e4-f5829642035d None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:58.358197 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa08a4-4534-4f43-a8e4-f5829642035d None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:58.404987 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efe3f0-26ab-4abe-809d-7e7cc878dfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:58.405861 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efe3f0-26ab-4abe-809d-7e7cc878dfa6 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:58.406143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efe3f0-26ab-4abe-809d-7e7cc878dfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:58.406440 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efe3f0-26ab-4abe-809d-7e7cc878dfa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:58.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efe3f0-26ab-4abe-809d-7e7cc878dfa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:58.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efe3f0-26ab-4abe-809d-7e7cc878dfa6 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:58.445870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-28efe3f0-26ab-4abe-809d-7e7cc878dfa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:58.445870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:58.445870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:58.445870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:58.445870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:58.445870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:58.445870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:58.445870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:58.445870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:58.445870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:58.445870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:58.445870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:58.445870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:36:58.445870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:36:58.445870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:58.446799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:58.447605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:36:58.447605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:36:58.447605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:58.447605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:58.449833 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 275/546] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:58.455202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c3cd3-b006-4475-b5ff-b5df54e7e654 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:58.643176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c3cd3-b006-4475-b5ff-b5df54e7e654 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:58.693788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d399c4-420d-4c4c-afb8-8fca208c598f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:58.694826 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d399c4-420d-4c4c-afb8-8fca208c598f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:58.695314 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d399c4-420d-4c4c-afb8-8fca208c598f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:58.695790 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d399c4-420d-4c4c-afb8-8fca208c598f None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:58.696700 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d399c4-420d-4c4c-afb8-8fca208c598f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:58.697189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d399c4-420d-4c4c-afb8-8fca208c598f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:58.699001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d399c4-420d-4c4c-afb8-8fca208c598f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:58.714995 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 272/547] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:58.723375 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efe3f0-26ab-4abe-809d-7e7cc878dfa6 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:58.817622 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:58.880696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efe3f0-26ab-4abe-809d-7e7cc878dfa6 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:58.931976 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77e9f1-1db2-4b12-bcea-ab673d4e6792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:58.932792 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77e9f1-1db2-4b12-bcea-ab673d4e6792 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:58.933070 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77e9f1-1db2-4b12-bcea-ab673d4e6792 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:58.933339 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77e9f1-1db2-4b12-bcea-ab673d4e6792 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:58.944253 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77e9f1-1db2-4b12-bcea-ab673d4e6792 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:58.945742 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77e9f1-1db2-4b12-bcea-ab673d4e6792 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-8d77e9f1-1db2-4b12-bcea-ab673d4e6792 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:36:58.960886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:58.962469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:58.963518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:58.963518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:58.963518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:36:58.963518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:36:58.963518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:36:58.963518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:36:58.963518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:36:58.963518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:36:58.963518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:36:58.963518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:58.963518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 276/548] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:36:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:58.968027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d399c4-420d-4c4c-afb8-8fca208c598f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:59.024231 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:59.113573 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d399c4-420d-4c4c-afb8-8fca208c598f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:59.164733 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c812e-8cd1-4df3-b6b0-ef479523aeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:59.165382 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c812e-8cd1-4df3-b6b0-ef479523aeb8 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:59.165660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c812e-8cd1-4df3-b6b0-ef479523aeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:59.165976 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c812e-8cd1-4df3-b6b0-ef479523aeb8 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:59.166847 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c812e-8cd1-4df3-b6b0-ef479523aeb8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:59.167155 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c812e-8cd1-4df3-b6b0-ef479523aeb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:59.168593 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c812e-8cd1-4df3-b6b0-ef479523aeb8 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:59.222067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 273/549] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 19:36:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:59.227919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77e9f1-1db2-4b12-bcea-ab673d4e6792 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:59.320373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecdc4e-c499-4a25-9a8c-e791bad610b0 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:59.321106 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecdc4e-c499-4a25-9a8c-e791bad610b0 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:59.321584 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecdc4e-c499-4a25-9a8c-e791bad610b0 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:59.323570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 274/550] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:36:59.328988 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfb8af-3423-4596-beba-a804c5e4b4cd None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:59.329501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfb8af-3423-4596-beba-a804c5e4b4cd None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:59.329993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfb8af-3423-4596-beba-a804c5e4b4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:59.354215 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77e9f1-1db2-4b12-bcea-ab673d4e6792 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:59.359418 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-a9cfb8af-3423-4596-beba-a804c5e4b4cd None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:36:59.360615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-a9cfb8af-3423-4596-beba-a804c5e4b4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:36:59.376385 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:59.408625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfba65e-bbe8-420e-b02b-f40daf2084e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:59.409441 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfba65e-bbe8-420e-b02b-f40daf2084e3 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:59.409732 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfba65e-bbe8-420e-b02b-f40daf2084e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:59.410008 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfba65e-bbe8-420e-b02b-f40daf2084e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:59.444032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 275/551] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 19:36:59.470213 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfb8af-3423-4596-beba-a804c5e4b4cd None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:59.507296 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:59.568063 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfba65e-bbe8-420e-b02b-f40daf2084e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:59.569582 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfba65e-bbe8-420e-b02b-f40daf2084e3 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:59.666047 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfb8af-3423-4596-beba-a804c5e4b4cd None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:59.723078 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2c0cc-7160-4cd2-bd3c-1f13812c7918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:59.723802 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2c0cc-7160-4cd2-bd3c-1f13812c7918 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:59.724079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2c0cc-7160-4cd2-bd3c-1f13812c7918 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:59.724348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2c0cc-7160-4cd2-bd3c-1f13812c7918 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:59.738466 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 277/552] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:36:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:36:59.740156 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfba65e-bbe8-420e-b02b-f40daf2084e3 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:59.756813 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:59.757734 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca2c0cc-7160-4cd2-bd3c-1f13812c7918 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-3ca2c0cc-7160-4cd2-bd3c-1f13812c7918 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:36:59.766434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:59.767660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:59.768824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:59.768824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:59.768824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:59.768824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:59.768824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:36:59.768824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:36:59.768824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:36:59.768824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:36:59.768824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:36:59.768824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:36:59.768824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:36:59.768824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:36:59.768824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 276/553] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:36:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:36:59.774171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2c0cc-7160-4cd2-bd3c-1f13812c7918 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:59.893941 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:36:59.907140 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfba65e-bbe8-420e-b02b-f40daf2084e3 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:59.908970 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2c0cc-7160-4cd2-bd3c-1f13812c7918 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:36:59.949970 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cf73b-1c39-4bb5-97ee-173bb7e42fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:59.950771 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cf73b-1c39-4bb5-97ee-173bb7e42fa5 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:59.951067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cf73b-1c39-4bb5-97ee-173bb7e42fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:59.951351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cf73b-1c39-4bb5-97ee-173bb7e42fa5 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:59.952209 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cf73b-1c39-4bb5-97ee-173bb7e42fa5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:36:59.952618 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cf73b-1c39-4bb5-97ee-173bb7e42fa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:59.954332 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cf73b-1c39-4bb5-97ee-173bb7e42fa5 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:59.954833 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63e0f7-b492-45bd-806e-0d8f2c10232d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:36:59.955917 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63e0f7-b492-45bd-806e-0d8f2c10232d None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:59.956405 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63e0f7-b492-45bd-806e-0d8f2c10232d None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:59.956875 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63e0f7-b492-45bd-806e-0d8f2c10232d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:59.965980 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 277/554] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:36:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:36:59.967840 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01793d-3cbe-4b97-884f-3325ae45cffb None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:59.968129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01793d-3cbe-4b97-884f-3325ae45cffb None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:59.968408 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01793d-3cbe-4b97-884f-3325ae45cffb None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:36:59.968750 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63e0f7-b492-45bd-806e-0d8f2c10232d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:36:59.970600 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63e0f7-b492-45bd-806e-0d8f2c10232d None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:36:59.974395 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 278/555] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:36:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:36:59.975489 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01793d-3cbe-4b97-884f-3325ae45cffb None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:36:59.994478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-bb63e0f7-b492-45bd-806e-0d8f2c10232d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:59.994478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:36:59.994478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:36:59.994478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:36:59.994478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:36:59.994478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:36:59.994478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:36:59.994478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:36:59.994478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:36:59.994478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:36:59.994478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:36:59.994478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:36:59.994478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:36:59.994478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:36:59.994478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:36:59.995395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:36:59.996864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:36:59.996864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:36:59.996864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:36:59.996864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:36:59.996864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 278/556] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:36:59.997293 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8528b4d-a7f2-44ba-8a0b-070a2f9525a8 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:36:59.997648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8528b4d-a7f2-44ba-8a0b-070a2f9525a8 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:36:59.997963 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8528b4d-a7f2-44ba-8a0b-070a2f9525a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:00.015254 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-d8528b4d-a7f2-44ba-8a0b-070a2f9525a8 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:00.016536 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-d8528b4d-a7f2-44ba-8a0b-070a2f9525a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:00.080361 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:00.097784 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 279/557] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:36:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:37:00.097784 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8528b4d-a7f2-44ba-8a0b-070a2f9525a8 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:00.137939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01793d-3cbe-4b97-884f-3325ae45cffb None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:00.173540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1addd-2dc0-4437-b62a-7d5f1954d209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:00.174529 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1addd-2dc0-4437-b62a-7d5f1954d209 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:00.174529 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1addd-2dc0-4437-b62a-7d5f1954d209 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:00.175198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1addd-2dc0-4437-b62a-7d5f1954d209 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:00.185855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1addd-2dc0-4437-b62a-7d5f1954d209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:00.187351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1addd-2dc0-4437-b62a-7d5f1954d209 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:00.208198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-14e1addd-2dc0-4437-b62a-7d5f1954d209 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:00.208198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:00.208198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:00.208198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:00.208198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:00.208198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:00.208198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:00.208198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:00.208198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:00.208198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:00.208198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:00.208198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:00.208198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:00.208198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:00.208198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:00.209706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:00.211177 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:00.211177 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:00.211177 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:00.211177 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:00.211177 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 279/558] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:36:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:00.211177 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1addd-2dc0-4437-b62a-7d5f1954d209 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:00.214521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:00.258304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8528b4d-a7f2-44ba-8a0b-070a2f9525a8 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:00.294426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fa408-dcc2-4a0c-b3d3-dae49f4f9e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:00.295096 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fa408-dcc2-4a0c-b3d3-dae49f4f9e01 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:00.295374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fa408-dcc2-4a0c-b3d3-dae49f4f9e01 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:00.295644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fa408-dcc2-4a0c-b3d3-dae49f4f9e01 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:00.296367 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fa408-dcc2-4a0c-b3d3-dae49f4f9e01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:00.296670 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fa408-dcc2-4a0c-b3d3-dae49f4f9e01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:00.298742 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fa408-dcc2-4a0c-b3d3-dae49f4f9e01 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:00.317147 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 280/559] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:00.317147 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fa408-dcc2-4a0c-b3d3-dae49f4f9e01 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:00.333552 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1addd-2dc0-4437-b62a-7d5f1954d209 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:00.369457 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177e023-13b6-495b-9caa-0e9712497822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:00.370454 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177e023-13b6-495b-9caa-0e9712497822 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:00.370744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177e023-13b6-495b-9caa-0e9712497822 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:00.371022 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177e023-13b6-495b-9caa-0e9712497822 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:00.382843 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177e023-13b6-495b-9caa-0e9712497822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:00.383079 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-7177e023-13b6-495b-9caa-0e9712497822 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:00.390604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:00.391786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:00.392657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:00.392657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:00.392657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:00.392657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:00.392657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:00.392657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:00.392657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:00.392657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:00.392657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:00.392657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:00.392657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:00.392657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:00.392657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 280/560] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:37:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:00.393710 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef89aeb-b97b-47af-987e-170c8a749ba9 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:00.393914 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef89aeb-b97b-47af-987e-170c8a749ba9 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:00.394465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef89aeb-b97b-47af-987e-170c8a749ba9 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:00.395667 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 281/561] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:00.396143 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef89aeb-b97b-47af-987e-170c8a749ba9 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:00.437206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fa408-dcc2-4a0c-b3d3-dae49f4f9e01 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:00.459909 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:00.471422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb9182-5c02-466b-b3bc-d493a2f89075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:00.472064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb9182-5c02-466b-b3bc-d493a2f89075 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:00.472340 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb9182-5c02-466b-b3bc-d493a2f89075 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:00.472605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb9182-5c02-466b-b3bc-d493a2f89075 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:00.473325 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb9182-5c02-466b-b3bc-d493a2f89075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:00.473625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb9182-5c02-466b-b3bc-d493a2f89075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:00.475045 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb9182-5c02-466b-b3bc-d493a2f89075 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:00.487038 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 281/562] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:00.488447 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc7c59-7991-4434-b473-a57d31dd1846 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:00.488735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc7c59-7991-4434-b473-a57d31dd1846 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:00.489001 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc7c59-7991-4434-b473-a57d31dd1846 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:00.490397 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 282/563] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:00.490926 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc7c59-7991-4434-b473-a57d31dd1846 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:00.501607 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef89aeb-b97b-47af-987e-170c8a749ba9 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:00.537170 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41497a7-954e-473e-9527-23fd48ee7ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:00.537969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41497a7-954e-473e-9527-23fd48ee7ade None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:00.538249 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41497a7-954e-473e-9527-23fd48ee7ade None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:00.538551 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41497a7-954e-473e-9527-23fd48ee7ade None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:00.545142 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41497a7-954e-473e-9527-23fd48ee7ade None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:00.546641 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41497a7-954e-473e-9527-23fd48ee7ade None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:00.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-a41497a7-954e-473e-9527-23fd48ee7ade None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:00.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:00.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:00.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:00.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:00.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:00.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:00.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:00.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:00.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:00.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:00.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:00.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:00.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:00.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:00.561292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:00.562159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:00.562159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:00.562159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:00.562159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:00.562159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:00.562159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:00.562159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:00.562159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:00.562159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 282/564] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:00.562968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4afc1d-9b0d-4a71-b146-456a6cd9da27 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:00.562968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4afc1d-9b0d-4a71-b146-456a6cd9da27 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:00.563257 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4afc1d-9b0d-4a71-b146-456a6cd9da27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:00.580003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-ab4afc1d-9b0d-4a71-b146-456a6cd9da27 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:00.581399 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-ab4afc1d-9b0d-4a71-b146-456a6cd9da27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:00.590053 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc7c59-7991-4434-b473-a57d31dd1846 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:00.604565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:00.628828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87df1d8-8099-419a-9441-b712683a7113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:00.629503 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87df1d8-8099-419a-9441-b712683a7113 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:00.629770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87df1d8-8099-419a-9441-b712683a7113 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:00.630035 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87df1d8-8099-419a-9441-b712683a7113 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:00.630813 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87df1d8-8099-419a-9441-b712683a7113 None admin] RBAC: Adding query filter params (name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:00.631114 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87df1d8-8099-419a-9441-b712683a7113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:00.632498 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87df1d8-8099-419a-9441-b712683a7113 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:00.640553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 283/565] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:37:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:00.641317 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87df1d8-8099-419a-9441-b712683a7113 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:00.653229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 283/566] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:37:00.654549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf1874-fcbc-4bcc-8858-65977a57068c None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:00.654838 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf1874-fcbc-4bcc-8858-65977a57068c None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:00.655120 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf1874-fcbc-4bcc-8858-65977a57068c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:00.672860 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-72bf1874-fcbc-4bcc-8858-65977a57068c None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:00.674807 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-72bf1874-fcbc-4bcc-8858-65977a57068c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:00.720919 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:00.741486 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 284/567] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:37:00.742331 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87df1d8-8099-419a-9441-b712683a7113 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:00.742442 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf1874-fcbc-4bcc-8858-65977a57068c None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:00.778856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b9c44-f72c-43a3-a47a-cd3ed7ad226b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:00.779660 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b9c44-f72c-43a3-a47a-cd3ed7ad226b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:00.779938 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b9c44-f72c-43a3-a47a-cd3ed7ad226b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:00.780206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b9c44-f72c-43a3-a47a-cd3ed7ad226b None admin] PATH_INFO: `/v3/users/glance` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:00.788420 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b9c44-f72c-43a3-a47a-cd3ed7ad226b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:00.789810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b9c44-f72c-43a3-a47a-cd3ed7ad226b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-7c6b9c44-f72c-43a3-a47a-cd3ed7ad226b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:00.804589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:00.805658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:00.806615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:00.806615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:00.806615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:00.806615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:00.806615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:00.806615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:00.806615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:00.806615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:00.806615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 19:37:00.806615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:00.806615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 284/568] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:37:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:00.806615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b9c44-f72c-43a3-a47a-cd3ed7ad226b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:00.841097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:00.857391 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf1874-fcbc-4bcc-8858-65977a57068c None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:00.892038 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c864c60-d0a1-4c8b-99cb-4d9c6159a827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:00.892760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c864c60-d0a1-4c8b-99cb-4d9c6159a827 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:00.893044 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c864c60-d0a1-4c8b-99cb-4d9c6159a827 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:00.893459 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c864c60-d0a1-4c8b-99cb-4d9c6159a827 None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:00.894249 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c864c60-d0a1-4c8b-99cb-4d9c6159a827 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:00.894584 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c864c60-d0a1-4c8b-99cb-4d9c6159a827 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:00.896043 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c864c60-d0a1-4c8b-99cb-4d9c6159a827 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:00.905585 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b9c44-f72c-43a3-a47a-cd3ed7ad226b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:00.907358 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 285/569] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:00.908142 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c864c60-d0a1-4c8b-99cb-4d9c6159a827 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:00.938444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122ffc9-1c2b-4fe5-96a4-02fbf5ff5c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:00.939234 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122ffc9-1c2b-4fe5-96a4-02fbf5ff5c2f None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:00.939537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122ffc9-1c2b-4fe5-96a4-02fbf5ff5c2f None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:00.939810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122ffc9-1c2b-4fe5-96a4-02fbf5ff5c2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:00.950296 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:00.951740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122ffc9-1c2b-4fe5-96a4-02fbf5ff5c2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:00.953392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122ffc9-1c2b-4fe5-96a4-02fbf5ff5c2f None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:00.971835 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-4122ffc9-1c2b-4fe5-96a4-02fbf5ff5c2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:00.971835 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:00.971835 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:00.971835 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:00.971835 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:00.971835 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:00.971835 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:00.971835 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:00.971835 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:00.971835 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:00.971835 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:00.971835 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:00.971835 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:00.971835 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:00.971835 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:00.972716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:00.973571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:00.973571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:00.973571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:00.973571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:00.973571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 285/570] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:00.973836 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122ffc9-1c2b-4fe5-96a4-02fbf5ff5c2f None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:01.012995 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c864c60-d0a1-4c8b-99cb-4d9c6159a827 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:01.047557 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f46e1-1f0a-4283-b1b0-23f3a6fe0703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:01.048282 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f46e1-1f0a-4283-b1b0-23f3a6fe0703 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:01.048560 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f46e1-1f0a-4283-b1b0-23f3a6fe0703 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:01.048857 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f46e1-1f0a-4283-b1b0-23f3a6fe0703 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:01.060237 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7f46e1-1f0a-4283-b1b0-23f3a6fe0703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:01.065309 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-5b7f46e1-1f0a-4283-b1b0-23f3a6fe0703 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:01.068714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:01.069704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:01.070601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:01.070601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:01.070601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:01.070601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:01.070601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:01.070601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:01.070601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:01.070601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:01.070601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:01.070601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:01.070601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:01.070601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:01.070601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 286/571] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:37:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:01.071352 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f46e1-1f0a-4283-b1b0-23f3a6fe0703 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:01.081992 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122ffc9-1c2b-4fe5-96a4-02fbf5ff5c2f None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:01.114657 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56388af6-2f37-4b82-a2be-5345a7762bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:01.115448 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388af6-2f37-4b82-a2be-5345a7762bd8 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:01.115728 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388af6-2f37-4b82-a2be-5345a7762bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:01.115993 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388af6-2f37-4b82-a2be-5345a7762bd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:01.126022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56388af6-2f37-4b82-a2be-5345a7762bd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-56388af6-2f37-4b82-a2be-5345a7762bd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:01.134414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:01.135579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:01.136404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:01.136404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:01.136404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:01.136404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:01.136404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:01.136404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:01.136404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:01.136404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:01.136404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:01.136404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:01.136404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:01.136404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:01.136404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 286/572] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:37:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:01.137074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56388af6-2f37-4b82-a2be-5345a7762bd8 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:01.175282 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f46e1-1f0a-4283-b1b0-23f3a6fe0703 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:01.176806 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:01.209936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4d19a-aa10-4028-953c-e182ef241c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:01.210629 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4d19a-aa10-4028-953c-e182ef241c8a None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:01.210910 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4d19a-aa10-4028-953c-e182ef241c8a None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:01.211181 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4d19a-aa10-4028-953c-e182ef241c8a None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:01.212011 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4d19a-aa10-4028-953c-e182ef241c8a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:01.212316 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4d19a-aa10-4028-953c-e182ef241c8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:01.213772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4d19a-aa10-4028-953c-e182ef241c8a None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:01.231029 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 287/573] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 19:37:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:01.231812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4d19a-aa10-4028-953c-e182ef241c8a None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:01.242136 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56388af6-2f37-4b82-a2be-5345a7762bd8 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:01.276238 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748aa9b3-98ad-4a93-9f57-025f2305652b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:01.276895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748aa9b3-98ad-4a93-9f57-025f2305652b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:01.277162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748aa9b3-98ad-4a93-9f57-025f2305652b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:01.277423 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748aa9b3-98ad-4a93-9f57-025f2305652b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:01.278216 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748aa9b3-98ad-4a93-9f57-025f2305652b None admin] RBAC: Adding query filter params (role.id=cefdc5c73ec347438e5b4687f191382c, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=4d1c8759f0564f28b388106dbf615cb5) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:01.278577 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748aa9b3-98ad-4a93-9f57-025f2305652b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:01.280014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748aa9b3-98ad-4a93-9f57-025f2305652b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:01.286090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 287/574] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:01 2022] GET /identity/v3/role_assignments?user.id=4d1c8759f0564f28b388106dbf615cb5&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=cefdc5c73ec347438e5b4687f191382c => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:01.286851 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748aa9b3-98ad-4a93-9f57-025f2305652b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:01.310814 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:01.347257 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4d19a-aa10-4028-953c-e182ef241c8a None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:01.389413 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10926b76-e0a5-44cd-9f71-d44033f80438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:01.390551 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10926b76-e0a5-44cd-9f71-d44033f80438 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:01.390938 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10926b76-e0a5-44cd-9f71-d44033f80438 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:01.391283 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10926b76-e0a5-44cd-9f71-d44033f80438 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:01.392047 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10926b76-e0a5-44cd-9f71-d44033f80438 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:01.392362 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10926b76-e0a5-44cd-9f71-d44033f80438 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:01.394130 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10926b76-e0a5-44cd-9f71-d44033f80438 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:01.414982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 288/575] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:01.414982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10926b76-e0a5-44cd-9f71-d44033f80438 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:01.419266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748aa9b3-98ad-4a93-9f57-025f2305652b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:01.441005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:01.461475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff9bcd-cf40-4710-acfc-9278711eb860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:01.462710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff9bcd-cf40-4710-acfc-9278711eb860 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:01.463004 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff9bcd-cf40-4710-acfc-9278711eb860 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:01.463291 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff9bcd-cf40-4710-acfc-9278711eb860 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:01.464042 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff9bcd-cf40-4710-acfc-9278711eb860 None admin] RBAC: Adding query filter params (name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:01.464729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff9bcd-cf40-4710-acfc-9278711eb860 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:01.466489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff9bcd-cf40-4710-acfc-9278711eb860 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:01.477743 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 288/576] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:37:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:01.478499 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff9bcd-cf40-4710-acfc-9278711eb860 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:01.532364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10926b76-e0a5-44cd-9f71-d44033f80438 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:01.561056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:01.568694 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494ffe1-e1f6-4789-932b-81d22db188c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:01.569376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494ffe1-e1f6-4789-932b-81d22db188c1 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:01.569652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494ffe1-e1f6-4789-932b-81d22db188c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:01.569925 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494ffe1-e1f6-4789-932b-81d22db188c1 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:01.570704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494ffe1-e1f6-4789-932b-81d22db188c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:01.571008 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494ffe1-e1f6-4789-932b-81d22db188c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:01.572391 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494ffe1-e1f6-4789-932b-81d22db188c1 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:01.581899 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 289/577] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:37:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:01.582762 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494ffe1-e1f6-4789-932b-81d22db188c1 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:01.592692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff9bcd-cf40-4710-acfc-9278711eb860 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:01.637313 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7521c3-e6fe-4690-9d0b-ae68d49a254f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:01.638229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7521c3-e6fe-4690-9d0b-ae68d49a254f None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:01.638537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7521c3-e6fe-4690-9d0b-ae68d49a254f None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:01.638808 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7521c3-e6fe-4690-9d0b-ae68d49a254f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:01.650361 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7521c3-e6fe-4690-9d0b-ae68d49a254f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:01.651822 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7521c3-e6fe-4690-9d0b-ae68d49a254f None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:01.669462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-ea7521c3-e6fe-4690-9d0b-ae68d49a254f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:01.669462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:01.669462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:01.669462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:01.669462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:01.669462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:01.669462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:01.669462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:01.669462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:01.669462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:01.669462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:01.669462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:01.669462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:01.669462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:01.669462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:01.670426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:01.671320 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:01.671320 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:01.671320 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:01.671320 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:01.671320 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 289/578] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:01.671624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7521c3-e6fe-4690-9d0b-ae68d49a254f None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:01.678469 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:01.699253 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494ffe1-e1f6-4789-932b-81d22db188c1 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:01.734858 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266e972-6303-491e-bd74-383daed6986b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:01.736053 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266e972-6303-491e-bd74-383daed6986b None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:01.736454 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266e972-6303-491e-bd74-383daed6986b None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:01.736788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266e972-6303-491e-bd74-383daed6986b None admin] PATH_INFO: `/v3/users/nova` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:01.744941 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266e972-6303-491e-bd74-383daed6986b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:01.746496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266e972-6303-491e-bd74-383daed6986b None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-a266e972-6303-491e-bd74-383daed6986b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:01.760955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:01.762103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:01.763281 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:01.763281 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:01.763281 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:01.763281 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:01.763281 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:01.763281 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:01.763281 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:01.763281 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:01.763281 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:37:01.763281 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:01.766622 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 290/579] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:37:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:01.767413 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266e972-6303-491e-bd74-383daed6986b None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:01.781052 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7521c3-e6fe-4690-9d0b-ae68d49a254f None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:01.814167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:01.825016 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcc8ff-fc7d-4c38-89b4-ed2a90faa1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:01.825819 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcc8ff-fc7d-4c38-89b4-ed2a90faa1a1 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:01.826091 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcc8ff-fc7d-4c38-89b4-ed2a90faa1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:01.826381 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcc8ff-fc7d-4c38-89b4-ed2a90faa1a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:01.838715 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcc8ff-fc7d-4c38-89b4-ed2a90faa1a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:01.841229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcc8ff-fc7d-4c38-89b4-ed2a90faa1a1 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:01.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-bcbcc8ff-fc7d-4c38-89b4-ed2a90faa1a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:01.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:01.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:01.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:01.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:01.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:01.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:01.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:01.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:01.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:01.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:01.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:01.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:01.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:01.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:01.863735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:01.864805 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:01.864805 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:01.864805 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:01.864805 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:01.866518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 290/580] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:01.867500 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcc8ff-fc7d-4c38-89b4-ed2a90faa1a1 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:01.885633 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266e972-6303-491e-bd74-383daed6986b None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:01.918051 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45e084-ffdd-4e67-b861-3dd89d05b58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:01.918965 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45e084-ffdd-4e67-b861-3dd89d05b58d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:01.919244 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45e084-ffdd-4e67-b861-3dd89d05b58d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:01.919517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45e084-ffdd-4e67-b861-3dd89d05b58d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:01.929979 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45e084-ffdd-4e67-b861-3dd89d05b58d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:01.931474 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45e084-ffdd-4e67-b861-3dd89d05b58d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:01.933157 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:01.950789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-3d45e084-ffdd-4e67-b861-3dd89d05b58d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:01.950789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:01.950789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:01.950789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:01.950789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:01.950789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:01.950789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:01.950789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:01.950789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:01.950789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:01.950789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:01.950789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:01.950789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:01.950789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:01.950789 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:01.951921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:01.952850 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:01.952850 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:01.952850 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:01.952850 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:01.953618 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 291/581] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:01.954562 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45e084-ffdd-4e67-b861-3dd89d05b58d None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:01.980745 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcc8ff-fc7d-4c38-89b4-ed2a90faa1a1 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:02.016246 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c209e4-28f5-4b41-9d85-485689ad0f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:02.016909 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c209e4-28f5-4b41-9d85-485689ad0f53 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:02.017209 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c209e4-28f5-4b41-9d85-485689ad0f53 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:02.017450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c209e4-28f5-4b41-9d85-485689ad0f53 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:02.018277 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c209e4-28f5-4b41-9d85-485689ad0f53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:02.018594 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c209e4-28f5-4b41-9d85-485689ad0f53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:02.020043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c209e4-28f5-4b41-9d85-485689ad0f53 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:02.037075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 291/582] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:02.037075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c209e4-28f5-4b41-9d85-485689ad0f53 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:02.061434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45e084-ffdd-4e67-b861-3dd89d05b58d None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:02.076747 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:02.095420 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee170eb0-e5d8-4254-8b19-fd2c1b745fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:02.096209 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee170eb0-e5d8-4254-8b19-fd2c1b745fb3 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:02.096527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee170eb0-e5d8-4254-8b19-fd2c1b745fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:02.096757 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee170eb0-e5d8-4254-8b19-fd2c1b745fb3 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:02.097581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee170eb0-e5d8-4254-8b19-fd2c1b745fb3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:02.097886 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee170eb0-e5d8-4254-8b19-fd2c1b745fb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:02.099368 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee170eb0-e5d8-4254-8b19-fd2c1b745fb3 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:02.121780 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 292/583] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:37:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:02.121780 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee170eb0-e5d8-4254-8b19-fd2c1b745fb3 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:02.146305 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c209e4-28f5-4b41-9d85-485689ad0f53 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:02.171260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:02.182049 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce20c2-be90-4d9b-8aea-4bd66b3889f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:02.182745 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce20c2-be90-4d9b-8aea-4bd66b3889f1 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:02.183022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce20c2-be90-4d9b-8aea-4bd66b3889f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:02.183291 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce20c2-be90-4d9b-8aea-4bd66b3889f1 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:02.184007 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce20c2-be90-4d9b-8aea-4bd66b3889f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:02.184308 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce20c2-be90-4d9b-8aea-4bd66b3889f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:02.185687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce20c2-be90-4d9b-8aea-4bd66b3889f1 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:02.197147 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 292/584] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:02.197861 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce20c2-be90-4d9b-8aea-4bd66b3889f1 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:02.223404 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee170eb0-e5d8-4254-8b19-fd2c1b745fb3 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:02.258387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ae1e5-cae4-480b-9476-cb9ab8fb32ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:02.259045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ae1e5-cae4-480b-9476-cb9ab8fb32ed None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:02.259321 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ae1e5-cae4-480b-9476-cb9ab8fb32ed None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:02.259588 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ae1e5-cae4-480b-9476-cb9ab8fb32ed None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:02.260298 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ae1e5-cae4-480b-9476-cb9ab8fb32ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:02.260597 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ae1e5-cae4-480b-9476-cb9ab8fb32ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:02.261968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ae1e5-cae4-480b-9476-cb9ab8fb32ed None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:02.278522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 293/585] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:02.278522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ae1e5-cae4-480b-9476-cb9ab8fb32ed None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:02.300234 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:02.304978 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce20c2-be90-4d9b-8aea-4bd66b3889f1 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:02.339348 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eab5d-7583-42dc-abf0-9208788eff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:02.340070 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eab5d-7583-42dc-abf0-9208788eff09 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:02.340353 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eab5d-7583-42dc-abf0-9208788eff09 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:02.340611 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eab5d-7583-42dc-abf0-9208788eff09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:02.347785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eab5d-7583-42dc-abf0-9208788eff09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:02.349240 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eab5d-7583-42dc-abf0-9208788eff09 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:02.364469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-2e4eab5d-7583-42dc-abf0-9208788eff09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:02.364469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:02.364469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:02.364469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:02.364469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:02.364469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:02.364469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:02.364469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:02.364469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:02.364469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:02.364469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:02.364469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:02.364469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:02.364469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:02.364469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:02.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:02.366479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:02.366479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:02.366479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:02.366479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:02.366479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:02.366479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:02.366479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:02.366479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:02.366479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 293/586] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:02.366479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eab5d-7583-42dc-abf0-9208788eff09 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:02.390773 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ae1e5-cae4-480b-9476-cb9ab8fb32ed None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:02.406149 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:02.431330 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40323ea6-5d44-48c8-aa5f-6aefcf4f62a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:02.432463 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40323ea6-5d44-48c8-aa5f-6aefcf4f62a2 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:02.432929 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40323ea6-5d44-48c8-aa5f-6aefcf4f62a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:02.433274 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40323ea6-5d44-48c8-aa5f-6aefcf4f62a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:02.444865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40323ea6-5d44-48c8-aa5f-6aefcf4f62a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:02.445859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40323ea6-5d44-48c8-aa5f-6aefcf4f62a2 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:02.464422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-40323ea6-5d44-48c8-aa5f-6aefcf4f62a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:02.464422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:02.464422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:02.464422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:02.464422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:02.464422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:02.464422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:02.464422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:02.464422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:02.464422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:02.464422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:02.464422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:02.464422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:02.464422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:02.464422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:02.465343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:02.466195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:02.466195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:02.466195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:02.466195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:02.470245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 294/587] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:02.470944 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40323ea6-5d44-48c8-aa5f-6aefcf4f62a2 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:02.477665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eab5d-7583-42dc-abf0-9208788eff09 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:02.514032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c32516-1462-42d4-9c0f-023d9455fa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:02.514873 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c32516-1462-42d4-9c0f-023d9455fa82 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:02.515146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c32516-1462-42d4-9c0f-023d9455fa82 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:02.515408 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c32516-1462-42d4-9c0f-023d9455fa82 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:02.523482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c32516-1462-42d4-9c0f-023d9455fa82 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:02.524940 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c32516-1462-42d4-9c0f-023d9455fa82 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-54c32516-1462-42d4-9c0f-023d9455fa82 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:02.539476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:02.540459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:02.541366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:02.541366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:02.541366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:02.541366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:02.541366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:02.541366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:02.541366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:02.541366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:02.541366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:37:02.541366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:02.541366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 294/588] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:37:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:02.541953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c32516-1462-42d4-9c0f-023d9455fa82 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:02.560576 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:02.586028 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40323ea6-5d44-48c8-aa5f-6aefcf4f62a2 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:02.624961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f0247-fa9c-4f96-868c-7002a7edaf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:02.626059 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f0247-fa9c-4f96-868c-7002a7edaf99 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:02.626582 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f0247-fa9c-4f96-868c-7002a7edaf99 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:02.626956 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f0247-fa9c-4f96-868c-7002a7edaf99 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:02.637652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f0247-fa9c-4f96-868c-7002a7edaf99 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:02.639214 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f0247-fa9c-4f96-868c-7002a7edaf99 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:02.650220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c32516-1462-42d4-9c0f-023d9455fa82 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-c24f0247-fa9c-4f96-868c-7002a7edaf99 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:02.653671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:02.654833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:02.655969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:02.655969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:02.655969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:02.655969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:02.655969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:02.655969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:02.655969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:02.655969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:02.655969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:37:02.655969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:02.661542 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 295/589] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:37:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:02.661542 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f0247-fa9c-4f96-868c-7002a7edaf99 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:02.688319 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6154f-e3b2-4e14-9eaf-3e221527cb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:02.689005 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6154f-e3b2-4e14-9eaf-3e221527cb0c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:02.689282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6154f-e3b2-4e14-9eaf-3e221527cb0c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:02.689652 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6154f-e3b2-4e14-9eaf-3e221527cb0c None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:02.690458 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6154f-e3b2-4e14-9eaf-3e221527cb0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:02.690762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6154f-e3b2-4e14-9eaf-3e221527cb0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:02.690896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:02.692205 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6154f-e3b2-4e14-9eaf-3e221527cb0c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:02.704285 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 295/590] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:02.709565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6154f-e3b2-4e14-9eaf-3e221527cb0c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:02.773067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f0247-fa9c-4f96-868c-7002a7edaf99 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:02.790994 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:02.811173 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0d96e-dd9e-4960-8d48-866073fbeacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:02.811834 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0d96e-dd9e-4960-8d48-866073fbeacc None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:02.812133 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0d96e-dd9e-4960-8d48-866073fbeacc None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:02.812382 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0d96e-dd9e-4960-8d48-866073fbeacc None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:02.813117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0d96e-dd9e-4960-8d48-866073fbeacc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:02.813404 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0d96e-dd9e-4960-8d48-866073fbeacc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:02.814996 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0d96e-dd9e-4960-8d48-866073fbeacc None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:02.816133 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6154f-e3b2-4e14-9eaf-3e221527cb0c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:02.827176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 296/591] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:02.831716 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0d96e-dd9e-4960-8d48-866073fbeacc None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:02.856879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464d7ff-7d94-47a4-984b-2b477c22c0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:02.857760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464d7ff-7d94-47a4-984b-2b477c22c0c9 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:02.858060 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464d7ff-7d94-47a4-984b-2b477c22c0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:02.861347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464d7ff-7d94-47a4-984b-2b477c22c0c9 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:02.861347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464d7ff-7d94-47a4-984b-2b477c22c0c9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:02.861347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464d7ff-7d94-47a4-984b-2b477c22c0c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:02.861347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464d7ff-7d94-47a4-984b-2b477c22c0c9 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:02.881228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 296/592] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 19:37:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:02.881228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464d7ff-7d94-47a4-984b-2b477c22c0c9 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:02.942801 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:02.964876 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0d96e-dd9e-4960-8d48-866073fbeacc None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:02.989450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464d7ff-7d94-47a4-984b-2b477c22c0c9 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:03.005263 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc35933-a5ae-49d6-9050-9d00c3944b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:03.005943 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc35933-a5ae-49d6-9050-9d00c3944b1c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:03.006218 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc35933-a5ae-49d6-9050-9d00c3944b1c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:03.006511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc35933-a5ae-49d6-9050-9d00c3944b1c None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:03.007339 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc35933-a5ae-49d6-9050-9d00c3944b1c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:03.007645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc35933-a5ae-49d6-9050-9d00c3944b1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:03.009088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc35933-a5ae-49d6-9050-9d00c3944b1c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:03.027002 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61237ddc-33c3-442d-ac03-6ef08c8fddd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:03.027895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61237ddc-33c3-442d-ac03-6ef08c8fddd2 None admin] REQUEST_METHOD: `PUT` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:03.028173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61237ddc-33c3-442d-ac03-6ef08c8fddd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:03.028442 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61237ddc-33c3-442d-ac03-6ef08c8fddd2 None admin] PATH_INFO: `/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/3029e939ee1c4569ba877d05547faa65/roles/cefdc5c73ec347438e5b4687f191382c` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:03.028898 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 297/593] 10.222.0.84 () {56 vars in 1139 bytes} [Thu May 19 19:37:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:03.029735 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc35933-a5ae-49d6-9050-9d00c3944b1c None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:03.037073 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:03.049959 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61237ddc-33c3-442d-ac03-6ef08c8fddd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecf87429758641eeb462b3e409cadc9e, user_id=3029e939ee1c4569ba877d05547faa65, role_id=cefdc5c73ec347438e5b4687f191382c)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:03.051439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61237ddc-33c3-442d-ac03-6ef08c8fddd2 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:03.066392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 297/594] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:37:02 2022] PUT /identity/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/3029e939ee1c4569ba877d05547faa65/roles/cefdc5c73ec347438e5b4687f191382c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:37:03.067521 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61237ddc-33c3-442d-ac03-6ef08c8fddd2 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:03.166871 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc35933-a5ae-49d6-9050-9d00c3944b1c None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:03.190142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61237ddc-33c3-442d-ac03-6ef08c8fddd2 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:03.225488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f521f-a6ff-4511-9afa-8175a780c52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:03.226324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f521f-a6ff-4511-9afa-8175a780c52a None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:03.226925 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f521f-a6ff-4511-9afa-8175a780c52a None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:03.227453 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f521f-a6ff-4511-9afa-8175a780c52a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:03.239623 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f521f-a6ff-4511-9afa-8175a780c52a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:03.241546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f521f-a6ff-4511-9afa-8175a780c52a None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:03.245858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2482b-3e15-42df-9f1e-b4e16e2ab005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:03.247020 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2482b-3e15-42df-9f1e-b4e16e2ab005 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:03.247543 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2482b-3e15-42df-9f1e-b4e16e2ab005 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:03.248024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2482b-3e15-42df-9f1e-b4e16e2ab005 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:03.256918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-426f521f-a6ff-4511-9afa-8175a780c52a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:03.256918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:03.256918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:03.256918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:03.256918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:03.256918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:03.256918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:03.256918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:03.256918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:03.256918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:03.256918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:03.256918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:03.256918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:03.256918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:03.256918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:03.257862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:03.258764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:03.258764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:03.258764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:03.258764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:03.258764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:03.258764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:03.258764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:03.258764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:03.258764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 298/595] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:03.260205 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f521f-a6ff-4511-9afa-8175a780c52a None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:03.263063 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:03.267057 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf2482b-3e15-42df-9f1e-b4e16e2ab005 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:03.269058 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf2482b-3e15-42df-9f1e-b4e16e2ab005 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:03.294207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-6cf2482b-3e15-42df-9f1e-b4e16e2ab005 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:03.294207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:03.294207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:03.294207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:03.294207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:03.294207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:03.294207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:03.294207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:03.294207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:03.294207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:03.294207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:03.294207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:03.294207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:03.294207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:03.294207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:03.295355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:03.296428 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:03.296428 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:03.296428 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:03.296428 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:03.298650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 298/596] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:03.299888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2482b-3e15-42df-9f1e-b4e16e2ab005 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:03.357168 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:03.393389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f521f-a6ff-4511-9afa-8175a780c52a None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:03.427377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2482b-3e15-42df-9f1e-b4e16e2ab005 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:03.440257 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54302fd9-3a1c-4791-bb17-492fe7af914f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:03.441567 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54302fd9-3a1c-4791-bb17-492fe7af914f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:03.442304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54302fd9-3a1c-4791-bb17-492fe7af914f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:03.442744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54302fd9-3a1c-4791-bb17-492fe7af914f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:03.460301 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54302fd9-3a1c-4791-bb17-492fe7af914f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:03.461981 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54302fd9-3a1c-4791-bb17-492fe7af914f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:03.463685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf1e4b-7040-4790-b63a-50be106328c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:03.464349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf1e4b-7040-4790-b63a-50be106328c8 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:03.464622 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf1e4b-7040-4790-b63a-50be106328c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:03.464884 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf1e4b-7040-4790-b63a-50be106328c8 None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:03.465676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf1e4b-7040-4790-b63a-50be106328c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:03.465952 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf1e4b-7040-4790-b63a-50be106328c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:03.467411 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf1e4b-7040-4790-b63a-50be106328c8 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:03.480547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 299/597] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:03.481449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf1e4b-7040-4790-b63a-50be106328c8 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:03.485992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-54302fd9-3a1c-4791-bb17-492fe7af914f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:03.485992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:03.485992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:03.485992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:03.485992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:03.485992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:03.485992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:03.485992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:03.485992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:03.485992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:03.485992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:03.485992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:03.485992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:03.485992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:03.485992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:03.487172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:03.488237 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:03.488237 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:03.488237 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:03.488237 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:03.490113 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 299/598] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:03.495147 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54302fd9-3a1c-4791-bb17-492fe7af914f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:03.562302 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:03.604078 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:03.608013 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf1e4b-7040-4790-b63a-50be106328c8 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:03.609506 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54302fd9-3a1c-4791-bb17-492fe7af914f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:03.650408 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59397641-fd46-483c-96bb-d9572cb8331c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:03.651081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a904529-2b0c-45a8-ae92-cb237a7a9946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:03.651222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59397641-fd46-483c-96bb-d9572cb8331c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:03.651417 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59397641-fd46-483c-96bb-d9572cb8331c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:03.651666 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59397641-fd46-483c-96bb-d9572cb8331c None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:03.651969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a904529-2b0c-45a8-ae92-cb237a7a9946 None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:03.652245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a904529-2b0c-45a8-ae92-cb237a7a9946 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:03.652405 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59397641-fd46-483c-96bb-d9572cb8331c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:03.652507 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a904529-2b0c-45a8-ae92-cb237a7a9946 None admin] PATH_INFO: `/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/abde7cc02fae4d9bb561a6642c5252a6/roles/cefdc5c73ec347438e5b4687f191382c` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:03.652678 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59397641-fd46-483c-96bb-d9572cb8331c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:03.654345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59397641-fd46-483c-96bb-d9572cb8331c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:03.671970 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 300/599] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:03.672156 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a904529-2b0c-45a8-ae92-cb237a7a9946 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecf87429758641eeb462b3e409cadc9e, user_id=abde7cc02fae4d9bb561a6642c5252a6, role_id=cefdc5c73ec347438e5b4687f191382c)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:03.672269 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59397641-fd46-483c-96bb-d9572cb8331c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:03.672380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a904529-2b0c-45a8-ae92-cb237a7a9946 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:03.690413 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 300/600] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:37:03 2022] PUT /identity/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/abde7cc02fae4d9bb561a6642c5252a6/roles/cefdc5c73ec347438e5b4687f191382c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:37:03.690413 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa27f3-bae6-4542-8493-613e2a5614d2 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:03.690413 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa27f3-bae6-4542-8493-613e2a5614d2 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:03.690413 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa27f3-bae6-4542-8493-613e2a5614d2 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:03.690413 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 301/601] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:03.690413 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa27f3-bae6-4542-8493-613e2a5614d2 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:03.816622 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59397641-fd46-483c-96bb-d9572cb8331c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:03.819132 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa27f3-bae6-4542-8493-613e2a5614d2 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:03.822694 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:03.863412 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4961f73-4b4d-4a5a-a43b-47f147ddd714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:03.864237 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4961f73-4b4d-4a5a-a43b-47f147ddd714 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:03.864515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4961f73-4b4d-4a5a-a43b-47f147ddd714 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:03.864783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4961f73-4b4d-4a5a-a43b-47f147ddd714 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:03.869264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5915b-e3bf-44f5-b17f-d91636e47e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:03.870424 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5915b-e3bf-44f5-b17f-d91636e47e6c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:03.870910 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5915b-e3bf-44f5-b17f-d91636e47e6c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:03.871382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5915b-e3bf-44f5-b17f-d91636e47e6c None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:03.872628 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5915b-e3bf-44f5-b17f-d91636e47e6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:03.873135 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5915b-e3bf-44f5-b17f-d91636e47e6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:03.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4961f73-4b4d-4a5a-a43b-47f147ddd714 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:03.874891 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4961f73-4b4d-4a5a-a43b-47f147ddd714 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:03.875144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5915b-e3bf-44f5-b17f-d91636e47e6c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:03.892282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 301/602] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:03.893666 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da2767-8f5a-4d26-aa31-51d84e68553f None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:03.893952 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da2767-8f5a-4d26-aa31-51d84e68553f None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:03.894221 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da2767-8f5a-4d26-aa31-51d84e68553f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:03.894642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-b4961f73-4b4d-4a5a-a43b-47f147ddd714 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:03.894642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:03.894642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:03.894642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:03.894642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:03.894642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:03.894642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:03.894642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:03.894642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:03.894642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:03.894642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:03.894642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:03.894642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:03.894642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:03.894642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:03.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:03.896410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:03.896410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:03.896410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:03.896410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:03.896410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:03.896410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:03.896410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:03.896410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:03.897000 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 302/603] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:03.901818 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4961f73-4b4d-4a5a-a43b-47f147ddd714 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:03.919042 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-d0da2767-8f5a-4d26-aa31-51d84e68553f None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:03.920769 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-d0da2767-8f5a-4d26-aa31-51d84e68553f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:03.936266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:03.995360 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 302/604] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:37:04.002281 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da2767-8f5a-4d26-aa31-51d84e68553f None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:04.019263 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4961f73-4b4d-4a5a-a43b-47f147ddd714 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:04.060102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c691f-39ce-494e-9660-8e8057e8d442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:04.060933 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c691f-39ce-494e-9660-8e8057e8d442 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:04.061211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c691f-39ce-494e-9660-8e8057e8d442 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:04.061477 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c691f-39ce-494e-9660-8e8057e8d442 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:04.068455 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c691f-39ce-494e-9660-8e8057e8d442 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:04.069899 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c691f-39ce-494e-9660-8e8057e8d442 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:04.084372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-5d3c691f-39ce-494e-9660-8e8057e8d442 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:04.084372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:04.084372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:04.084372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:04.084372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:04.084372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:04.084372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:04.084372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:04.084372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:04.084372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:04.084372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:04.084372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:04.084372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:04.084372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:04.084372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:04.085289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:04.086159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:04.086159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:04.086159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:04.086159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:04.086159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:04.086159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:04.086159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:04.086159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:04.086159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 303/605] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:04.086666 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c691f-39ce-494e-9660-8e8057e8d442 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:04.113728 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:04.125434 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da2767-8f5a-4d26-aa31-51d84e68553f None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:04.162610 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a15bcd-48e3-43a3-b239-be900aa99e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:04.163562 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a15bcd-48e3-43a3-b239-be900aa99e8c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:04.163981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a15bcd-48e3-43a3-b239-be900aa99e8c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:04.164388 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a15bcd-48e3-43a3-b239-be900aa99e8c None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:04.165306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a15bcd-48e3-43a3-b239-be900aa99e8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:04.165796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a15bcd-48e3-43a3-b239-be900aa99e8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:04.167498 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a15bcd-48e3-43a3-b239-be900aa99e8c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:04.180587 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 303/606] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:04.181599 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a15bcd-48e3-43a3-b239-be900aa99e8c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:04.198713 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c691f-39ce-494e-9660-8e8057e8d442 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:04.199349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:04.245041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1560cd3-e08c-4d30-8603-cfb2d0179004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:04.246227 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1560cd3-e08c-4d30-8603-cfb2d0179004 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:04.248142 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1560cd3-e08c-4d30-8603-cfb2d0179004 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:04.248435 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1560cd3-e08c-4d30-8603-cfb2d0179004 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:04.268194 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1560cd3-e08c-4d30-8603-cfb2d0179004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-f1560cd3-e08c-4d30-8603-cfb2d0179004 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:04.278750 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:04.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:04.280511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:04.280511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:04.280511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:04.280511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:04.280511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:04.280511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:04.280511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:04.280511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:04.280511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:04.280511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:04.280511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:04.280511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:04.280511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 304/607] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:37:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:04.281183 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1560cd3-e08c-4d30-8603-cfb2d0179004 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:04.317290 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a15bcd-48e3-43a3-b239-be900aa99e8c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:04.356614 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b33c9-35ad-4476-9c00-7858ef8c0dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:04.357308 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b33c9-35ad-4476-9c00-7858ef8c0dc2 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:04.357527 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b33c9-35ad-4476-9c00-7858ef8c0dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:04.357801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b33c9-35ad-4476-9c00-7858ef8c0dc2 None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:04.358642 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b33c9-35ad-4476-9c00-7858ef8c0dc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:04.358950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b33c9-35ad-4476-9c00-7858ef8c0dc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:04.360607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b33c9-35ad-4476-9c00-7858ef8c0dc2 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:04.374075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 304/608] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:04.374994 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b33c9-35ad-4476-9c00-7858ef8c0dc2 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:04.390093 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:04.412321 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1560cd3-e08c-4d30-8603-cfb2d0179004 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:04.449714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:04.452524 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf19aa-d761-411c-a581-b982117b18a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:04.453656 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf19aa-d761-411c-a581-b982117b18a8 None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:04.454134 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf19aa-d761-411c-a581-b982117b18a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:04.454622 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf19aa-d761-411c-a581-b982117b18a8 None admin] PATH_INFO: `/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/423cd0a1227940d1bd7e4263348f45c9/roles/cefdc5c73ec347438e5b4687f191382c` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:04.471140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf19aa-d761-411c-a581-b982117b18a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecf87429758641eeb462b3e409cadc9e, user_id=423cd0a1227940d1bd7e4263348f45c9, role_id=cefdc5c73ec347438e5b4687f191382c)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:04.472985 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf19aa-d761-411c-a581-b982117b18a8 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:04.488566 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b33c9-35ad-4476-9c00-7858ef8c0dc2 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:04.492051 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 305/609] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:37:04 2022] PUT /identity/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/423cd0a1227940d1bd7e4263348f45c9/roles/cefdc5c73ec347438e5b4687f191382c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:37:04.493198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf19aa-d761-411c-a581-b982117b18a8 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:04.536774 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8adf5-0215-4c89-873d-5e27256565e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:04.537455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8adf5-0215-4c89-873d-5e27256565e9 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:04.537755 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8adf5-0215-4c89-873d-5e27256565e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:04.538027 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8adf5-0215-4c89-873d-5e27256565e9 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:04.538811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8adf5-0215-4c89-873d-5e27256565e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:04.539110 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8adf5-0215-4c89-873d-5e27256565e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:04.540551 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8adf5-0215-4c89-873d-5e27256565e9 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:04.550671 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 305/610] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:37:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:04.556201 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8adf5-0215-4c89-873d-5e27256565e9 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:04.649715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf19aa-d761-411c-a581-b982117b18a8 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:04.694146 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6beac-b051-475d-99ff-62ecf47c3421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:04.695087 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6beac-b051-475d-99ff-62ecf47c3421 None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:04.695367 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6beac-b051-475d-99ff-62ecf47c3421 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:04.695636 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6beac-b051-475d-99ff-62ecf47c3421 None admin] PATH_INFO: `/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/d2b98b1e4024410a87addde112237f32/roles/cefdc5c73ec347438e5b4687f191382c` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:04.698762 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:04.701245 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8adf5-0215-4c89-873d-5e27256565e9 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:04.717047 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6beac-b051-475d-99ff-62ecf47c3421 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecf87429758641eeb462b3e409cadc9e, user_id=d2b98b1e4024410a87addde112237f32, role_id=cefdc5c73ec347438e5b4687f191382c)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:04.718554 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6beac-b051-475d-99ff-62ecf47c3421 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:04.736945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 306/611] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:37:04 2022] PUT /identity/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/d2b98b1e4024410a87addde112237f32/roles/cefdc5c73ec347438e5b4687f191382c => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:37:04.753236 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:04.761185 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450eaf20-5732-4891-91d0-ed857f53640b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:04.762509 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450eaf20-5732-4891-91d0-ed857f53640b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:04.763092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450eaf20-5732-4891-91d0-ed857f53640b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:04.766802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450eaf20-5732-4891-91d0-ed857f53640b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:04.787092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450eaf20-5732-4891-91d0-ed857f53640b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:04.788911 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450eaf20-5732-4891-91d0-ed857f53640b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:04.815304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-450eaf20-5732-4891-91d0-ed857f53640b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:04.815304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:04.815304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:04.815304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:04.815304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:04.815304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:04.815304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:04.815304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:04.815304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:04.815304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:04.815304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:04.815304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:04.815304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:04.815304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:04.815304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:04.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:04.817098 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:04.817098 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:04.817098 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:04.817098 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:04.819064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 306/612] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:04.824578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6beac-b051-475d-99ff-62ecf47c3421 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:04.938488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6beac-b051-475d-99ff-62ecf47c3421 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:04.982634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eda88d-a2ee-4c2d-8521-477ed75dcef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:04.983498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eda88d-a2ee-4c2d-8521-477ed75dcef1 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:04.983875 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eda88d-a2ee-4c2d-8521-477ed75dcef1 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:04.984248 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eda88d-a2ee-4c2d-8521-477ed75dcef1 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:04.985090 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eda88d-a2ee-4c2d-8521-477ed75dcef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:04.985519 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eda88d-a2ee-4c2d-8521-477ed75dcef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:04.987202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eda88d-a2ee-4c2d-8521-477ed75dcef1 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:05.001805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 307/613] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:05.007677 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450eaf20-5732-4891-91d0-ed857f53640b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:05.130944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450eaf20-5732-4891-91d0-ed857f53640b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:05.165504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2115fe-72bc-4cd7-9a73-80e5f00f7586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:05.166321 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2115fe-72bc-4cd7-9a73-80e5f00f7586 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:05.166598 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2115fe-72bc-4cd7-9a73-80e5f00f7586 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:05.166864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2115fe-72bc-4cd7-9a73-80e5f00f7586 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:05.175012 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2115fe-72bc-4cd7-9a73-80e5f00f7586 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:05.176473 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2115fe-72bc-4cd7-9a73-80e5f00f7586 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-9f2115fe-72bc-4cd7-9a73-80e5f00f7586 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:05.191025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:05.191969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:05.192872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:05.192872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:05.192872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:05.192872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:05.192872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:05.192872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:05.192872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:05.192872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:05.192872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:37:05.192872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:05.192872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 307/614] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:37:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:05.198404 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eda88d-a2ee-4c2d-8521-477ed75dcef1 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:05.282882 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:05.357360 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eda88d-a2ee-4c2d-8521-477ed75dcef1 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:05.404659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6e7d1-9aa7-4925-8c38-57ad583e7185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:05.405244 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6e7d1-9aa7-4925-8c38-57ad583e7185 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:05.405519 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6e7d1-9aa7-4925-8c38-57ad583e7185 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:05.405790 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6e7d1-9aa7-4925-8c38-57ad583e7185 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:05.406641 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6e7d1-9aa7-4925-8c38-57ad583e7185 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:05.406941 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6e7d1-9aa7-4925-8c38-57ad583e7185 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:05.408457 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6e7d1-9aa7-4925-8c38-57ad583e7185 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:05.435971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 308/615] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 19:37:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:05.442799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2115fe-72bc-4cd7-9a73-80e5f00f7586 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:05.454809 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2fdc2-7479-4c18-9e5a-48f64a796bde None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:05.456086 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2fdc2-7479-4c18-9e5a-48f64a796bde None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:05.456591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2fdc2-7479-4c18-9e5a-48f64a796bde None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:05.458444 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 309/616] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:37:05.464341 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96545313-f184-4910-b36e-0c9f5363992d None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:05.464899 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96545313-f184-4910-b36e-0c9f5363992d None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:05.465386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96545313-f184-4910-b36e-0c9f5363992d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:05.483015 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-96545313-f184-4910-b36e-0c9f5363992d None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:05.484519 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-96545313-f184-4910-b36e-0c9f5363992d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:05.506657 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:05.594248 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 310/617] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:37:05.598121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2115fe-72bc-4cd7-9a73-80e5f00f7586 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:05.638832 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96545313-f184-4910-b36e-0c9f5363992d None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:05.647006 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75542bd-f405-4fae-9e7b-722bc841e24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:05.648112 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75542bd-f405-4fae-9e7b-722bc841e24b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:05.648589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75542bd-f405-4fae-9e7b-722bc841e24b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:05.649075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75542bd-f405-4fae-9e7b-722bc841e24b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:05.673923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75542bd-f405-4fae-9e7b-722bc841e24b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:05.675766 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75542bd-f405-4fae-9e7b-722bc841e24b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:05.695639 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:05.716191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-b75542bd-f405-4fae-9e7b-722bc841e24b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:05.716191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:05.716191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:05.716191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:05.716191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:05.716191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:05.716191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:05.716191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:05.716191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:05.716191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:05.716191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:05.716191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:05.716191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:05.716191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:05.716191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:05.717178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:05.718040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:05.718040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:05.718040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:05.718040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:05.724480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 308/618] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:05.727414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75542bd-f405-4fae-9e7b-722bc841e24b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:05.790830 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96545313-f184-4910-b36e-0c9f5363992d None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:05.833937 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd09ac-6d05-4a3b-bcab-199b8391aea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:05.835393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd09ac-6d05-4a3b-bcab-199b8391aea2 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:05.835889 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd09ac-6d05-4a3b-bcab-199b8391aea2 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:05.836355 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd09ac-6d05-4a3b-bcab-199b8391aea2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:05.856150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd09ac-6d05-4a3b-bcab-199b8391aea2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-99bd09ac-6d05-4a3b-bcab-199b8391aea2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:05.869659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:05.870671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:05.871485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:05.871485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:05.871485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:05.871485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:05.871485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:05.871485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:05.871485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:05.871485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:05.871485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:05.871485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:05.871485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:05.871485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:05.875005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 311/619] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:37:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:05.881600 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd09ac-6d05-4a3b-bcab-199b8391aea2 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:05.930781 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:05.933863 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75542bd-f405-4fae-9e7b-722bc841e24b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:06.000102 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812db00-c26b-4675-a10b-9969e8627aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:06.000763 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812db00-c26b-4675-a10b-9969e8627aec None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:06.001037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812db00-c26b-4675-a10b-9969e8627aec None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:06.001306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812db00-c26b-4675-a10b-9969e8627aec None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:06.002191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812db00-c26b-4675-a10b-9969e8627aec None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:06.002522 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812db00-c26b-4675-a10b-9969e8627aec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:06.003912 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812db00-c26b-4675-a10b-9969e8627aec None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:06.030811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 309/620] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:06.039291 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812db00-c26b-4675-a10b-9969e8627aec None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:06.062814 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:06.078706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd09ac-6d05-4a3b-bcab-199b8391aea2 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:06.176946 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35869322-457a-423d-bc92-cacea3475ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:06.177623 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35869322-457a-423d-bc92-cacea3475ba0 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:06.177900 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35869322-457a-423d-bc92-cacea3475ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:06.178169 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35869322-457a-423d-bc92-cacea3475ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:06.178948 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35869322-457a-423d-bc92-cacea3475ba0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:06.179254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35869322-457a-423d-bc92-cacea3475ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:06.180647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35869322-457a-423d-bc92-cacea3475ba0 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:06.181870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812db00-c26b-4675-a10b-9969e8627aec None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:06.201997 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 312/621] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:37:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:06.208227 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35869322-457a-423d-bc92-cacea3475ba0 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:06.230483 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cad9b6-1317-44a7-881f-cbf83b41454d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:06.231654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cad9b6-1317-44a7-881f-cbf83b41454d None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:06.232130 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cad9b6-1317-44a7-881f-cbf83b41454d None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:06.232596 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cad9b6-1317-44a7-881f-cbf83b41454d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:06.244964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cad9b6-1317-44a7-881f-cbf83b41454d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:06.251935 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cad9b6-1317-44a7-881f-cbf83b41454d None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:06.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-a1cad9b6-1317-44a7-881f-cbf83b41454d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:06.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:06.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:06.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:06.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:06.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:06.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:06.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:06.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:06.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:06.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:06.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:06.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:06.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:06.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:06.275676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:06.276567 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:06.276567 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:06.276567 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:06.276567 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:06.276567 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:06.276567 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:06.276567 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:06.276567 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:06.276567 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 310/622] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:06.287290 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cad9b6-1317-44a7-881f-cbf83b41454d None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:06.309593 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:06.347736 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35869322-457a-423d-bc92-cacea3475ba0 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:06.386212 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda74c36-dcf0-4bf2-83ff-54c9cefb34b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:06.387473 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda74c36-dcf0-4bf2-83ff-54c9cefb34b3 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:06.387829 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda74c36-dcf0-4bf2-83ff-54c9cefb34b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:06.388167 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda74c36-dcf0-4bf2-83ff-54c9cefb34b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:06.400229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda74c36-dcf0-4bf2-83ff-54c9cefb34b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:06.401960 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda74c36-dcf0-4bf2-83ff-54c9cefb34b3 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:06.412325 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cad9b6-1317-44a7-881f-cbf83b41454d None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:06.419788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-cda74c36-dcf0-4bf2-83ff-54c9cefb34b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:06.419788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:06.419788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:06.419788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:06.419788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:06.419788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:06.419788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:06.419788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:06.419788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:06.419788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:06.419788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:06.419788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:06.419788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:06.419788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:06.419788 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:06.420661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:06.421539 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:06.421539 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:06.421539 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:06.421539 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:06.423870 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 313/623] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:06.425412 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad25699-92ab-4fce-bc3f-0a7866381743 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:06.425856 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad25699-92ab-4fce-bc3f-0a7866381743 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:06.426314 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad25699-92ab-4fce-bc3f-0a7866381743 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:06.428070 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 314/624] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:06.428923 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad25699-92ab-4fce-bc3f-0a7866381743 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:06.440011 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:06.461728 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a82ae5-54b9-4c68-8dbd-5b124bdfb5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:06.462877 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a82ae5-54b9-4c68-8dbd-5b124bdfb5e7 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:06.463994 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a82ae5-54b9-4c68-8dbd-5b124bdfb5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:06.465049 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a82ae5-54b9-4c68-8dbd-5b124bdfb5e7 None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:06.466042 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a82ae5-54b9-4c68-8dbd-5b124bdfb5e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:06.466806 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a82ae5-54b9-4c68-8dbd-5b124bdfb5e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:06.468980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a82ae5-54b9-4c68-8dbd-5b124bdfb5e7 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:06.487041 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 311/625] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:06.488681 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37145f-b482-4202-943c-9dda2a026f53 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:06.489028 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37145f-b482-4202-943c-9dda2a026f53 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:06.489368 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37145f-b482-4202-943c-9dda2a026f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:06.509746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-ed37145f-b482-4202-943c-9dda2a026f53 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:06.511284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-ed37145f-b482-4202-943c-9dda2a026f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:06.549143 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad25699-92ab-4fce-bc3f-0a7866381743 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:06.588357 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be005b9b-38c8-49b4-b9d2-731a46d43a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:06.589546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be005b9b-38c8-49b4-b9d2-731a46d43a57 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:06.590565 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be005b9b-38c8-49b4-b9d2-731a46d43a57 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:06.591014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be005b9b-38c8-49b4-b9d2-731a46d43a57 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:06.591951 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be005b9b-38c8-49b4-b9d2-731a46d43a57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:06.592447 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be005b9b-38c8-49b4-b9d2-731a46d43a57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:06.594046 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be005b9b-38c8-49b4-b9d2-731a46d43a57 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:06.600404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 312/626] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:37:06.601316 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37145f-b482-4202-943c-9dda2a026f53 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:06.606075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:06.608359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 315/627] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:06.613872 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be005b9b-38c8-49b4-b9d2-731a46d43a57 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:06.705221 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:06.735519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37145f-b482-4202-943c-9dda2a026f53 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:06.744829 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be005b9b-38c8-49b4-b9d2-731a46d43a57 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:06.779469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1a233-dae6-4687-9526-79c63467b146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:06.780458 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1a233-dae6-4687-9526-79c63467b146 None admin] REQUEST_METHOD: `PUT` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:06.780736 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1a233-dae6-4687-9526-79c63467b146 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:06.781009 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1a233-dae6-4687-9526-79c63467b146 None admin] PATH_INFO: `/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/4d1c8759f0564f28b388106dbf615cb5/roles/cefdc5c73ec347438e5b4687f191382c` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:06.781481 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346e57b-a263-4050-bc2b-0ac87952d162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:06.782565 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346e57b-a263-4050-bc2b-0ac87952d162 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:06.783098 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346e57b-a263-4050-bc2b-0ac87952d162 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:06.783709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346e57b-a263-4050-bc2b-0ac87952d162 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:06.792761 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346e57b-a263-4050-bc2b-0ac87952d162 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:06.794566 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346e57b-a263-4050-bc2b-0ac87952d162 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:06.798562 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1a233-dae6-4687-9526-79c63467b146 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecf87429758641eeb462b3e409cadc9e, user_id=4d1c8759f0564f28b388106dbf615cb5, role_id=cefdc5c73ec347438e5b4687f191382c)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:06.800077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1a233-dae6-4687-9526-79c63467b146 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-f346e57b-a263-4050-bc2b-0ac87952d162 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:06.811465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:06.812491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:06.813460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:06.813460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:06.813460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:06.813460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:06.813460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:06.813460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:06.813460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:06.813460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:06.813460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 19:37:06.813460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:06.815056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 313/628] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:37:06 2022] PUT /identity/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/4d1c8759f0564f28b388106dbf615cb5/roles/cefdc5c73ec347438e5b4687f191382c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:37:06.815200 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1a233-dae6-4687-9526-79c63467b146 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:06.817161 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 316/629] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:37:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:06.823482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346e57b-a263-4050-bc2b-0ac87952d162 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:06.904348 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:06.987213 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346e57b-a263-4050-bc2b-0ac87952d162 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:07.008359 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1a233-dae6-4687-9526-79c63467b146 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:07.042648 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:07.061778 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f670b44-dec0-4f7b-a261-2fd48ff9dd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:07.062869 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f670b44-dec0-4f7b-a261-2fd48ff9dd63 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:07.064051 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f670b44-dec0-4f7b-a261-2fd48ff9dd63 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:07.064815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f670b44-dec0-4f7b-a261-2fd48ff9dd63 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:07.066075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f670b44-dec0-4f7b-a261-2fd48ff9dd63 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:07.068029 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f670b44-dec0-4f7b-a261-2fd48ff9dd63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:07.069727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f670b44-dec0-4f7b-a261-2fd48ff9dd63 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:07.088503 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f18961-20ad-4e2d-aafa-996ca1b27f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:07.089319 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f18961-20ad-4e2d-aafa-996ca1b27f5d None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:07.089596 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f18961-20ad-4e2d-aafa-996ca1b27f5d None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:07.089866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f18961-20ad-4e2d-aafa-996ca1b27f5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:07.095293 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 317/630] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 19:37:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:07.101345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f670b44-dec0-4f7b-a261-2fd48ff9dd63 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:07.110295 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f18961-20ad-4e2d-aafa-996ca1b27f5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-40f18961-20ad-4e2d-aafa-996ca1b27f5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:07.120171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:07.121210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:07.122648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:07.122648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:07.122648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:07.122648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:07.122648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:07.122648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:07.122648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:07.122648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:07.122648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:07.122648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:07.122648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:07.122648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:07.122648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 314/631] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:37:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:07.132269 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f18961-20ad-4e2d-aafa-996ca1b27f5d None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:07.246513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f18961-20ad-4e2d-aafa-996ca1b27f5d None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:07.273045 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:07.313090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7845c6-6420-4f1a-8efe-7d6adbdecb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:07.313770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7845c6-6420-4f1a-8efe-7d6adbdecb97 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:07.314047 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7845c6-6420-4f1a-8efe-7d6adbdecb97 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:07.314693 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7845c6-6420-4f1a-8efe-7d6adbdecb97 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:07.315459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7845c6-6420-4f1a-8efe-7d6adbdecb97 None admin] RBAC: Adding query filter params (name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:07.315766 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7845c6-6420-4f1a-8efe-7d6adbdecb97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:07.317152 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7845c6-6420-4f1a-8efe-7d6adbdecb97 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:07.326616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 315/632] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:37:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:07.327542 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f670b44-dec0-4f7b-a261-2fd48ff9dd63 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:07.335368 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7845c6-6420-4f1a-8efe-7d6adbdecb97 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:07.383378 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6c31c-1007-4497-b696-b4a5edc95557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:07.384232 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6c31c-1007-4497-b696-b4a5edc95557 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:07.384582 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6c31c-1007-4497-b696-b4a5edc95557 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:07.384927 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6c31c-1007-4497-b696-b4a5edc95557 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:07.456684 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:07.460069 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7845c6-6420-4f1a-8efe-7d6adbdecb97 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:07.473896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6c31c-1007-4497-b696-b4a5edc95557 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:07.475400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6c31c-1007-4497-b696-b4a5edc95557 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:07.497220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddeb332-72d2-48f1-8296-a9ee93c4251c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:07.497939 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddeb332-72d2-48f1-8296-a9ee93c4251c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:07.498214 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddeb332-72d2-48f1-8296-a9ee93c4251c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:07.498510 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddeb332-72d2-48f1-8296-a9ee93c4251c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:07.510078 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddeb332-72d2-48f1-8296-a9ee93c4251c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:07.511577 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddeb332-72d2-48f1-8296-a9ee93c4251c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:07.529537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-6ddeb332-72d2-48f1-8296-a9ee93c4251c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:07.529537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:07.529537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:07.529537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:07.529537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:07.529537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:07.529537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:07.529537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:07.529537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:07.529537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:07.529537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:07.529537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:07.529537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:07.529537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:07.529537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:07.530496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:07.531397 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:07.531397 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:07.531397 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:07.531397 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:07.531397 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 316/633] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:07.532148 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1a929-5b0e-4a18-b253-19636f8734da None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:07.532433 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1a929-5b0e-4a18-b253-19636f8734da None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:07.532801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1a929-5b0e-4a18-b253-19636f8734da None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:07.538347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 317/634] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:07.538347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfbb6e-1f1c-41a9-a3e4-388b0aedcf42 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:07.538347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfbb6e-1f1c-41a9-a3e4-388b0aedcf42 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:07.538347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfbb6e-1f1c-41a9-a3e4-388b0aedcf42 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:07.538347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 318/635] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:07.538864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfbb6e-1f1c-41a9-a3e4-388b0aedcf42 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:07.569008 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:07.588346 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 318/636] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:37:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:37:07.589760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd462e10-22d3-475b-8731-ae5ae15e40e2 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:07.590045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd462e10-22d3-475b-8731-ae5ae15e40e2 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:07.590341 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd462e10-22d3-475b-8731-ae5ae15e40e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:07.607823 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-cd462e10-22d3-475b-8731-ae5ae15e40e2 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:07.609339 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-cd462e10-22d3-475b-8731-ae5ae15e40e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:07.655185 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfbb6e-1f1c-41a9-a3e4-388b0aedcf42 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:07.690285 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 319/637] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:37:07.690285 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4883a4-041a-4e2d-8556-0a8dcd468992 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:07.690285 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4883a4-041a-4e2d-8556-0a8dcd468992 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:07.690285 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4883a4-041a-4e2d-8556-0a8dcd468992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:07.700829 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21b781-e067-41c2-9036-17bacca876ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:07.701650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21b781-e067-41c2-9036-17bacca876ae None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:07.701993 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21b781-e067-41c2-9036-17bacca876ae None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:07.702443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21b781-e067-41c2-9036-17bacca876ae None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:07.703223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-ac4883a4-041a-4e2d-8556-0a8dcd468992 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:07.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21b781-e067-41c2-9036-17bacca876ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:07.703675 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21b781-e067-41c2-9036-17bacca876ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:07.704573 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-ac4883a4-041a-4e2d-8556-0a8dcd468992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:07.705372 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21b781-e067-41c2-9036-17bacca876ae None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:07.715695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:07.720120 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 319/638] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:07.720344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21b781-e067-41c2-9036-17bacca876ae None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:07.790369 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 320/639] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:37:07.790369 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4883a4-041a-4e2d-8556-0a8dcd468992 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:07.823736 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:07.837608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21b781-e067-41c2-9036-17bacca876ae None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:07.872878 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ace1e-572a-4709-8503-56d85f7ddf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:07.873771 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ace1e-572a-4709-8503-56d85f7ddf25 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:07.874169 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ace1e-572a-4709-8503-56d85f7ddf25 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:07.874473 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ace1e-572a-4709-8503-56d85f7ddf25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:07.885352 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ace1e-572a-4709-8503-56d85f7ddf25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:07.887189 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ace1e-572a-4709-8503-56d85f7ddf25 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:07.892621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4883a4-041a-4e2d-8556-0a8dcd468992 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:07.907206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-a47ace1e-572a-4709-8503-56d85f7ddf25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:07.907206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:07.907206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:07.907206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:07.907206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:07.907206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:07.907206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:07.907206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:07.907206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:07.907206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:07.907206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:07.907206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:07.907206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:07.907206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:07.907206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:07.908237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:07.909486 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:07.909486 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:07.909486 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:07.909486 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:07.909486 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 320/640] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:07.909486 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ace1e-572a-4709-8503-56d85f7ddf25 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:07.933703 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39be07-eaab-4523-b71c-29596e6d5733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:07.934530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39be07-eaab-4523-b71c-29596e6d5733 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:07.934811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39be07-eaab-4523-b71c-29596e6d5733 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:07.935081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39be07-eaab-4523-b71c-29596e6d5733 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:07.949395 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39be07-eaab-4523-b71c-29596e6d5733 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-8f39be07-eaab-4523-b71c-29596e6d5733 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:07.958436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:07.959340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:07.965140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:07.965140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:07.965140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:07.965140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:07.965140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:07.965140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:07.965140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:07.965140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:07.965140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:07.965140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:07.965140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:07.965140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:07.965140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 321/641] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:37:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:07.965140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39be07-eaab-4523-b71c-29596e6d5733 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:07.967861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:08.023653 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ace1e-572a-4709-8503-56d85f7ddf25 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:08.058609 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f8f64-e639-4345-b1f7-9a5d0fae6c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:08.059340 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f8f64-e639-4345-b1f7-9a5d0fae6c47 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:08.059603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f8f64-e639-4345-b1f7-9a5d0fae6c47 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:08.059872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f8f64-e639-4345-b1f7-9a5d0fae6c47 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:08.067137 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f8f64-e639-4345-b1f7-9a5d0fae6c47 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:08.068610 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f8f64-e639-4345-b1f7-9a5d0fae6c47 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:08.070347 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39be07-eaab-4523-b71c-29596e6d5733 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:08.071591 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-ef5f8f64-e639-4345-b1f7-9a5d0fae6c47 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:08.085827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:08.087123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:08.088050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:08.088050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:08.088050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:08.088050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:08.088050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:08.088050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:08.088050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:08.088050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:08.088050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:37:08.088050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:08.088050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 321/642] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:37:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:08.088050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f8f64-e639-4345-b1f7-9a5d0fae6c47 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:08.115443 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226dba1d-9832-4fd6-ad83-558aa12eea44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:08.116302 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226dba1d-9832-4fd6-ad83-558aa12eea44 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:08.116585 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226dba1d-9832-4fd6-ad83-558aa12eea44 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:08.116856 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226dba1d-9832-4fd6-ad83-558aa12eea44 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:08.128429 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226dba1d-9832-4fd6-ad83-558aa12eea44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-226dba1d-9832-4fd6-ad83-558aa12eea44 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:08.138308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:08.139250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:08.140129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:08.140129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:08.140129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:08.140129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:08.140129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:08.140129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:08.140129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:08.140129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:08.140129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:08.140129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:08.140129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:08.140129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:08.142327 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 322/643] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:37:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:08.143487 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226dba1d-9832-4fd6-ad83-558aa12eea44 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:08.193620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f8f64-e639-4345-b1f7-9a5d0fae6c47 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:08.213902 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:08.227373 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03658c-b0dd-4800-b9ff-f569b160c0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:08.228032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03658c-b0dd-4800-b9ff-f569b160c0fa None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:08.228307 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03658c-b0dd-4800-b9ff-f569b160c0fa None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:08.228574 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03658c-b0dd-4800-b9ff-f569b160c0fa None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:08.229288 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03658c-b0dd-4800-b9ff-f569b160c0fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:08.229589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03658c-b0dd-4800-b9ff-f569b160c0fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:08.230995 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03658c-b0dd-4800-b9ff-f569b160c0fa None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:08.246646 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 322/644] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:08.247875 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03658c-b0dd-4800-b9ff-f569b160c0fa None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:08.262964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226dba1d-9832-4fd6-ad83-558aa12eea44 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:08.306633 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:08.309826 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad58eab-4a2f-44ae-bda4-594f5cc4de04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:08.310839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad58eab-4a2f-44ae-bda4-594f5cc4de04 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:08.311571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad58eab-4a2f-44ae-bda4-594f5cc4de04 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:08.311851 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad58eab-4a2f-44ae-bda4-594f5cc4de04 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:08.312637 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad58eab-4a2f-44ae-bda4-594f5cc4de04 None admin] RBAC: Adding query filter params (name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:08.312942 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad58eab-4a2f-44ae-bda4-594f5cc4de04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:08.315701 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad58eab-4a2f-44ae-bda4-594f5cc4de04 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:08.330813 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 323/645] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:37:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:08.330813 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad58eab-4a2f-44ae-bda4-594f5cc4de04 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:08.349920 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03658c-b0dd-4800-b9ff-f569b160c0fa None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:08.385736 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c44c8-b07f-4117-8462-8e9169c98d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:08.386475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c44c8-b07f-4117-8462-8e9169c98d68 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:08.386747 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c44c8-b07f-4117-8462-8e9169c98d68 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:08.387019 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c44c8-b07f-4117-8462-8e9169c98d68 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:08.387848 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c44c8-b07f-4117-8462-8e9169c98d68 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:08.388150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c44c8-b07f-4117-8462-8e9169c98d68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:08.389566 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c44c8-b07f-4117-8462-8e9169c98d68 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:08.406379 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 323/646] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:37:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:08.407175 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c44c8-b07f-4117-8462-8e9169c98d68 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:08.448133 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad58eab-4a2f-44ae-bda4-594f5cc4de04 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:08.476009 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:08.485878 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe3144-a2f8-476c-8d66-675fb80890cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:08.486800 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe3144-a2f8-476c-8d66-675fb80890cc None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:08.487082 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe3144-a2f8-476c-8d66-675fb80890cc None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:08.487357 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe3144-a2f8-476c-8d66-675fb80890cc None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:08.488107 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe3144-a2f8-476c-8d66-675fb80890cc None admin] RBAC: Adding query filter params (name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:08.488413 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe3144-a2f8-476c-8d66-675fb80890cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:08.489814 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe3144-a2f8-476c-8d66-675fb80890cc None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:08.498099 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 324/647] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:37:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:08.498762 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe3144-a2f8-476c-8d66-675fb80890cc None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:08.518060 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c44c8-b07f-4117-8462-8e9169c98d68 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:08.536063 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:08.554201 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b3a8d-b960-4138-8112-a84e1ff88466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:08.555473 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b3a8d-b960-4138-8112-a84e1ff88466 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:08.555905 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b3a8d-b960-4138-8112-a84e1ff88466 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:08.556173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b3a8d-b960-4138-8112-a84e1ff88466 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:08.563561 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b3a8d-b960-4138-8112-a84e1ff88466 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:08.564986 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b3a8d-b960-4138-8112-a84e1ff88466 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:08.578783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-000b3a8d-b960-4138-8112-a84e1ff88466 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:08.578783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:08.578783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:08.578783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:08.578783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:08.578783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:08.578783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:08.578783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:08.578783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:08.578783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:08.578783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:08.578783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:08.578783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:08.578783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:08.578783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:08.579688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:08.580682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:08.580682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:08.580682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:08.580682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:08.580682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:08.580682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:08.580682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:08.580682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:08.580682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 324/648] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:08.581355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b3a8d-b960-4138-8112-a84e1ff88466 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:08.610746 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe3144-a2f8-476c-8d66-675fb80890cc None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:08.650223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a756d-dd3e-49ab-af86-43410182e2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:08.651061 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a756d-dd3e-49ab-af86-43410182e2b9 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:08.651341 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a756d-dd3e-49ab-af86-43410182e2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:08.651615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a756d-dd3e-49ab-af86-43410182e2b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:08.663997 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a756d-dd3e-49ab-af86-43410182e2b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:08.665453 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a756d-dd3e-49ab-af86-43410182e2b9 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:08.683578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-112a756d-dd3e-49ab-af86-43410182e2b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:08.683578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:08.683578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:08.683578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:08.683578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:08.683578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:08.683578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:08.683578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:08.683578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:08.683578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:08.683578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:08.683578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:08.683578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:08.683578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:08.683578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:08.684499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:08.685374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:08.685374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:08.685374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:08.685374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:08.685374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 325/649] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:08.685651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a756d-dd3e-49ab-af86-43410182e2b9 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:08.693310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b3a8d-b960-4138-8112-a84e1ff88466 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:08.726885 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:08.733649 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c56d4-13cb-47e6-aa38-d2bad5bbbe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:08.734339 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c56d4-13cb-47e6-aa38-d2bad5bbbe0b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:08.734784 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c56d4-13cb-47e6-aa38-d2bad5bbbe0b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:08.735063 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c56d4-13cb-47e6-aa38-d2bad5bbbe0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:08.789090 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a756d-dd3e-49ab-af86-43410182e2b9 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:08.793653 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:08.817469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c56d4-13cb-47e6-aa38-d2bad5bbbe0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:08.818924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c56d4-13cb-47e6-aa38-d2bad5bbbe0b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:08.824717 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac39849-ab4f-4c21-a267-db775157f086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:08.825516 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac39849-ab4f-4c21-a267-db775157f086 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:08.825793 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac39849-ab4f-4c21-a267-db775157f086 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:08.826061 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac39849-ab4f-4c21-a267-db775157f086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:08.837336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac39849-ab4f-4c21-a267-db775157f086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:08.839087 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac39849-ab4f-4c21-a267-db775157f086 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:08.857081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-1ac39849-ab4f-4c21-a267-db775157f086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:08.857081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:08.857081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:08.857081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:08.857081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:08.857081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:08.857081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:08.857081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:08.857081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:08.857081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:08.857081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:08.857081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:08.857081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:08.857081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:08.857081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:08.857963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:08.859532 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:08.859532 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:08.859532 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:08.859532 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:08.859532 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 326/650] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:08.859532 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac39849-ab4f-4c21-a267-db775157f086 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:08.926905 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 325/651] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:37:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:37:08.928172 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c56d4-13cb-47e6-aa38-d2bad5bbbe0b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:08.961028 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:08.971238 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac39849-ab4f-4c21-a267-db775157f086 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:09.010442 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490198e-62f1-4259-8baf-3d0d3384964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:09.011221 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490198e-62f1-4259-8baf-3d0d3384964d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:09.011504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490198e-62f1-4259-8baf-3d0d3384964d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:09.011767 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490198e-62f1-4259-8baf-3d0d3384964d None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:09.012556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c490198e-62f1-4259-8baf-3d0d3384964d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:09.012974 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c490198e-62f1-4259-8baf-3d0d3384964d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:09.014608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:09.015511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c490198e-62f1-4259-8baf-3d0d3384964d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:09.034612 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 327/652] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:09.034612 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490198e-62f1-4259-8baf-3d0d3384964d None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:09.044106 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c56d4-13cb-47e6-aa38-d2bad5bbbe0b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:09.079740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8d043-067e-45ca-a365-5fcfd7e9a126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:09.080386 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8d043-067e-45ca-a365-5fcfd7e9a126 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:09.080668 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8d043-067e-45ca-a365-5fcfd7e9a126 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:09.080941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8d043-067e-45ca-a365-5fcfd7e9a126 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:09.081667 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8d043-067e-45ca-a365-5fcfd7e9a126 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:09.081971 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8d043-067e-45ca-a365-5fcfd7e9a126 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:09.083386 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8d043-067e-45ca-a365-5fcfd7e9a126 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:09.094967 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 326/653] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:09.095667 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8d043-067e-45ca-a365-5fcfd7e9a126 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:09.146991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490198e-62f1-4259-8baf-3d0d3384964d None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:09.183734 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9f167-05b6-4a6d-9e5f-f9c65d7ee050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:09.184399 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9f167-05b6-4a6d-9e5f-f9c65d7ee050 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:09.184676 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9f167-05b6-4a6d-9e5f-f9c65d7ee050 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:09.184947 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9f167-05b6-4a6d-9e5f-f9c65d7ee050 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:09.185660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9f167-05b6-4a6d-9e5f-f9c65d7ee050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:09.185964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9f167-05b6-4a6d-9e5f-f9c65d7ee050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:09.187388 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9f167-05b6-4a6d-9e5f-f9c65d7ee050 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:09.199161 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 328/654] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:09.199960 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9f167-05b6-4a6d-9e5f-f9c65d7ee050 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:09.207676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8d043-067e-45ca-a365-5fcfd7e9a126 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:09.214325 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:09.244331 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f1f03-ad65-4a76-9c43-43ca0e30450c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:09.245130 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f1f03-ad65-4a76-9c43-43ca0e30450c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:09.245410 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f1f03-ad65-4a76-9c43-43ca0e30450c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:09.245688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f1f03-ad65-4a76-9c43-43ca0e30450c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:09.252442 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:09.257622 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f1f03-ad65-4a76-9c43-43ca0e30450c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:09.259129 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f1f03-ad65-4a76-9c43-43ca0e30450c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:09.276025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-1b6f1f03-ad65-4a76-9c43-43ca0e30450c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:09.276025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:09.276025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:09.276025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:09.276025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:09.276025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:09.276025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:09.276025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:09.276025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:09.276025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:09.276025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:09.276025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:09.276025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:09.276025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:09.276025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:09.276929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:09.277819 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:09.277819 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:09.277819 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:09.277819 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:09.277819 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 327/655] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:09.278102 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f1f03-ad65-4a76-9c43-43ca0e30450c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:09.305413 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9f167-05b6-4a6d-9e5f-f9c65d7ee050 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:09.344406 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57179d-a3e9-4c46-ae6c-7813ba0c02d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:09.345027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57179d-a3e9-4c46-ae6c-7813ba0c02d4 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:09.345300 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57179d-a3e9-4c46-ae6c-7813ba0c02d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:09.345596 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57179d-a3e9-4c46-ae6c-7813ba0c02d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:09.346620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57179d-a3e9-4c46-ae6c-7813ba0c02d4 None admin] RBAC: Adding query filter params (role.id=cefdc5c73ec347438e5b4687f191382c, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=3029e939ee1c4569ba877d05547faa65) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:09.346930 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57179d-a3e9-4c46-ae6c-7813ba0c02d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:09.348396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57179d-a3e9-4c46-ae6c-7813ba0c02d4 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:09.354584 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 329/656] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:09 2022] GET /identity/v3/role_assignments?user.id=3029e939ee1c4569ba877d05547faa65&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=cefdc5c73ec347438e5b4687f191382c => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:09.355933 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da233c72-c860-4286-99c0-5deff904da8f None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:09.356217 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da233c72-c860-4286-99c0-5deff904da8f None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:09.356535 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da233c72-c860-4286-99c0-5deff904da8f None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:09.357939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 330/657] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:09.358565 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da233c72-c860-4286-99c0-5deff904da8f None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:09.390242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f1f03-ad65-4a76-9c43-43ca0e30450c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:09.429112 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d5a39-91a8-4114-9e20-a63685b46acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:09.429637 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d5a39-91a8-4114-9e20-a63685b46acc None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:09.429914 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d5a39-91a8-4114-9e20-a63685b46acc None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:09.430184 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d5a39-91a8-4114-9e20-a63685b46acc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:09.438812 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d5a39-91a8-4114-9e20-a63685b46acc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:09.440422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d5a39-91a8-4114-9e20-a63685b46acc None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-024d5a39-91a8-4114-9e20-a63685b46acc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:09.455708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:09.456713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:09.457634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:09.457634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:09.457634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:09.457634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:09.457634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:09.457634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:09.457634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:09.457634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:09.457634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:37:09.457634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:09.457634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 328/658] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:37:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:09.458223 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d5a39-91a8-4114-9e20-a63685b46acc None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:09.460090 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:09.490887 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da233c72-c860-4286-99c0-5deff904da8f None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:09.515666 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:09.538205 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a2540-f200-4e3d-90e2-7421e4f082c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:09.539335 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a2540-f200-4e3d-90e2-7421e4f082c0 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:09.539755 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a2540-f200-4e3d-90e2-7421e4f082c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:09.540333 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a2540-f200-4e3d-90e2-7421e4f082c0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:09.548600 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a2540-f200-4e3d-90e2-7421e4f082c0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:09.550070 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a2540-f200-4e3d-90e2-7421e4f082c0 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-2b1a2540-f200-4e3d-90e2-7421e4f082c0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:09.566359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:09.567320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:09.568224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:09.568224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:09.568224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:09.568224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:09.568224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:09.568224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:09.568224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:09.568224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:09.568224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:37:09.568224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:09.568224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 331/659] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:37:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:09.568819 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb9e92-f4bc-4553-ad80-1c91fbed258b None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:09.569173 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb9e92-f4bc-4553-ad80-1c91fbed258b None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:09.569952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb9e92-f4bc-4553-ad80-1c91fbed258b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:09.579971 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d5a39-91a8-4114-9e20-a63685b46acc None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:09.587382 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-51eb9e92-f4bc-4553-ad80-1c91fbed258b None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:09.588563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-51eb9e92-f4bc-4553-ad80-1c91fbed258b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:09.623557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf2477-9e60-43d0-98f0-c18ae1d545b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:09.624237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf2477-9e60-43d0-98f0-c18ae1d545b0 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:09.624518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf2477-9e60-43d0-98f0-c18ae1d545b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:09.624788 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf2477-9e60-43d0-98f0-c18ae1d545b0 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:09.625527 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf2477-9e60-43d0-98f0-c18ae1d545b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:09.625830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf2477-9e60-43d0-98f0-c18ae1d545b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:09.627385 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf2477-9e60-43d0-98f0-c18ae1d545b0 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:09.640151 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 329/660] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:09.640982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf2477-9e60-43d0-98f0-c18ae1d545b0 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:09.676721 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 332/661] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:37:09.681629 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb9e92-f4bc-4553-ad80-1c91fbed258b None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:09.743195 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf2477-9e60-43d0-98f0-c18ae1d545b0 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:09.753883 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:09.756031 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:09.780087 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7dd2f0-17c4-4219-a627-bda16d9b00ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:09.780777 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7dd2f0-17c4-4219-a627-bda16d9b00ea None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:09.781054 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7dd2f0-17c4-4219-a627-bda16d9b00ea None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:09.781397 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7dd2f0-17c4-4219-a627-bda16d9b00ea None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:09.782155 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7dd2f0-17c4-4219-a627-bda16d9b00ea None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:09.782503 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7dd2f0-17c4-4219-a627-bda16d9b00ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:09.783971 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7dd2f0-17c4-4219-a627-bda16d9b00ea None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:09.793374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb9e92-f4bc-4553-ad80-1c91fbed258b None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:09.801423 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 330/662] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 19:37:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:09.802152 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7dd2f0-17c4-4219-a627-bda16d9b00ea None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:09.835158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baddd80-b424-440b-9e0c-9ffe31d183f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:09.835831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baddd80-b424-440b-9e0c-9ffe31d183f5 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:09.836111 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baddd80-b424-440b-9e0c-9ffe31d183f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:09.836545 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baddd80-b424-440b-9e0c-9ffe31d183f5 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:09.838872 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baddd80-b424-440b-9e0c-9ffe31d183f5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:09.839185 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baddd80-b424-440b-9e0c-9ffe31d183f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:09.840653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baddd80-b424-440b-9e0c-9ffe31d183f5 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:09.865564 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 333/663] 10.222.0.84 () {56 vars in 1139 bytes} [Thu May 19 19:37:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:09.866495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baddd80-b424-440b-9e0c-9ffe31d183f5 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:09.910097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7dd2f0-17c4-4219-a627-bda16d9b00ea None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:09.946666 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc9407-4ad9-4a6c-b968-67272b31db01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:09.947470 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc9407-4ad9-4a6c-b968-67272b31db01 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:09.947745 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc9407-4ad9-4a6c-b968-67272b31db01 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:09.948009 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc9407-4ad9-4a6c-b968-67272b31db01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:09.954927 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc9407-4ad9-4a6c-b968-67272b31db01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:09.956510 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc9407-4ad9-4a6c-b968-67272b31db01 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:09.970178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-b4dc9407-4ad9-4a6c-b968-67272b31db01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:09.970178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:09.970178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:09.970178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:09.970178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:09.970178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:09.970178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:09.970178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:09.970178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:09.970178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:09.970178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:09.970178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:09.970178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:09.970178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:09.970178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:09.971876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:09.972968 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:09.972968 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:09.972968 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:09.972968 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:09.972968 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:09.972968 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:09.972968 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:09.972968 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:09.976159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 331/664] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:09.977441 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc9407-4ad9-4a6c-b968-67272b31db01 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:09.978477 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baddd80-b424-440b-9e0c-9ffe31d183f5 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:10.004996 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:10.006579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:10.018460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fe15f-074f-43c9-9c9a-4d04a21766c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:10.019276 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fe15f-074f-43c9-9c9a-4d04a21766c8 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:10.019601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fe15f-074f-43c9-9c9a-4d04a21766c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:10.019876 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fe15f-074f-43c9-9c9a-4d04a21766c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:10.030863 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fe15f-074f-43c9-9c9a-4d04a21766c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-010fe15f-074f-43c9-9c9a-4d04a21766c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:10.038718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:10.039651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:10.040488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:10.040488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:10.040488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:10.040488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:10.040488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:10.040488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:10.040488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:10.040488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:10.040488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:10.040488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:10.040488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:10.040488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:10.040488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 334/665] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:37:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:10.041823 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fe15f-074f-43c9-9c9a-4d04a21766c8 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:10.087504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc9407-4ad9-4a6c-b968-67272b31db01 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:10.125220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29ae01-d3f5-4463-a642-a7af2698bb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:10.125870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29ae01-d3f5-4463-a642-a7af2698bb11 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:10.126155 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29ae01-d3f5-4463-a642-a7af2698bb11 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:10.126447 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29ae01-d3f5-4463-a642-a7af2698bb11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:10.146168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fe15f-074f-43c9-9c9a-4d04a21766c8 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:10.181642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58ed45-f4ba-4c2e-8c4c-b0610a025c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:10.182216 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58ed45-f4ba-4c2e-8c4c-b0610a025c66 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:10.182543 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58ed45-f4ba-4c2e-8c4c-b0610a025c66 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:10.182818 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58ed45-f4ba-4c2e-8c4c-b0610a025c66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:10.196621 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29ae01-d3f5-4463-a642-a7af2698bb11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:10.198087 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29ae01-d3f5-4463-a642-a7af2698bb11 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:10.226171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:10.244223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:10.262953 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58ed45-f4ba-4c2e-8c4c-b0610a025c66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:10.264429 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58ed45-f4ba-4c2e-8c4c-b0610a025c66 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:10.302747 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 332/666] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:37:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:37:10.303794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29ae01-d3f5-4463-a642-a7af2698bb11 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:10.375317 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 335/667] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:37:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:37:10.375317 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58ed45-f4ba-4c2e-8c4c-b0610a025c66 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:10.415736 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29ae01-d3f5-4463-a642-a7af2698bb11 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:10.444240 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:10.452957 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b61a11-caf3-4fe9-adcc-e7c093238083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:10.453617 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b61a11-caf3-4fe9-adcc-e7c093238083 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:10.453897 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b61a11-caf3-4fe9-adcc-e7c093238083 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:10.454171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b61a11-caf3-4fe9-adcc-e7c093238083 None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:10.455018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b61a11-caf3-4fe9-adcc-e7c093238083 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:10.455325 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b61a11-caf3-4fe9-adcc-e7c093238083 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:10.456795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b61a11-caf3-4fe9-adcc-e7c093238083 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:10.463813 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:10.473615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 333/668] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:10.473615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b61a11-caf3-4fe9-adcc-e7c093238083 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:10.486733 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58ed45-f4ba-4c2e-8c4c-b0610a025c66 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:10.521634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8415d-31bc-4836-9b50-fb52eb82f950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:10.522329 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8415d-31bc-4836-9b50-fb52eb82f950 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:10.522613 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8415d-31bc-4836-9b50-fb52eb82f950 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:10.522884 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8415d-31bc-4836-9b50-fb52eb82f950 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:10.523624 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8415d-31bc-4836-9b50-fb52eb82f950 None admin] RBAC: Adding query filter params (name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:10.523928 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8415d-31bc-4836-9b50-fb52eb82f950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:10.525314 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8415d-31bc-4836-9b50-fb52eb82f950 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:10.534777 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 336/669] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:37:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:10.535465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8415d-31bc-4836-9b50-fb52eb82f950 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:10.585089 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b61a11-caf3-4fe9-adcc-e7c093238083 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:10.619908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb0ad7-494b-44ad-998b-ce44bc2e44f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:10.621126 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb0ad7-494b-44ad-998b-ce44bc2e44f8 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:10.621407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb0ad7-494b-44ad-998b-ce44bc2e44f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:10.621675 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb0ad7-494b-44ad-998b-ce44bc2e44f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:10.636083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb0ad7-494b-44ad-998b-ce44bc2e44f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:10.637554 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb0ad7-494b-44ad-998b-ce44bc2e44f8 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:10.653068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8415d-31bc-4836-9b50-fb52eb82f950 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:10.655143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-c2bb0ad7-494b-44ad-998b-ce44bc2e44f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:10.655143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:10.655143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:10.655143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:10.655143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:10.655143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:10.655143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:10.655143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:10.655143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:10.655143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:10.655143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:10.655143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:10.655143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:10.655143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:10.655143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:10.655980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:10.656844 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:10.656844 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:10.656844 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:10.656844 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:10.661320 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 334/670] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:10.661978 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb0ad7-494b-44ad-998b-ce44bc2e44f8 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:10.691410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7796b-cf11-46f3-9768-f278aa15d682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:10.692210 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7796b-cf11-46f3-9768-f278aa15d682 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:10.692493 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7796b-cf11-46f3-9768-f278aa15d682 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:10.692768 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7796b-cf11-46f3-9768-f278aa15d682 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:10.703987 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7796b-cf11-46f3-9768-f278aa15d682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:10.704477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:10.705438 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7796b-cf11-46f3-9768-f278aa15d682 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:10.712573 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:10.724088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-8ff7796b-cf11-46f3-9768-f278aa15d682 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:10.724088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:10.724088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:10.724088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:10.724088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:10.724088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:10.724088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:10.724088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:10.724088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:10.724088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:10.724088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:10.724088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:10.724088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:10.724088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:10.724088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:10.725168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:10.726040 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:10.726040 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:10.726040 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:10.726040 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:10.730026 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 337/671] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:10.730892 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7796b-cf11-46f3-9768-f278aa15d682 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:10.771572 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb0ad7-494b-44ad-998b-ce44bc2e44f8 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:10.808844 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc6564-d6b8-4929-97bf-09dae64b1cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:10.809606 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc6564-d6b8-4929-97bf-09dae64b1cfb None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:10.809881 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc6564-d6b8-4929-97bf-09dae64b1cfb None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:10.810250 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc6564-d6b8-4929-97bf-09dae64b1cfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:10.811110 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc6564-d6b8-4929-97bf-09dae64b1cfb None admin] RBAC: Adding query filter params (role.id=cefdc5c73ec347438e5b4687f191382c, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=abde7cc02fae4d9bb561a6642c5252a6) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:10.811436 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc6564-d6b8-4929-97bf-09dae64b1cfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:10.812996 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc6564-d6b8-4929-97bf-09dae64b1cfb None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:10.819250 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 335/672] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:10 2022] GET /identity/v3/role_assignments?user.id=abde7cc02fae4d9bb561a6642c5252a6&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=cefdc5c73ec347438e5b4687f191382c => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:10.820026 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc6564-d6b8-4929-97bf-09dae64b1cfb None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:10.841706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7796b-cf11-46f3-9768-f278aa15d682 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:10.880031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afb9ee-489c-413d-936e-8faafc4ff204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:10.880863 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afb9ee-489c-413d-936e-8faafc4ff204 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:10.881143 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afb9ee-489c-413d-936e-8faafc4ff204 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:10.881413 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afb9ee-489c-413d-936e-8faafc4ff204 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:10.894183 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afb9ee-489c-413d-936e-8faafc4ff204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:10.902216 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afb9ee-489c-413d-936e-8faafc4ff204 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:10.930292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-b1afb9ee-489c-413d-936e-8faafc4ff204 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:10.930292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:10.930292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:10.930292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:10.930292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:10.930292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:10.930292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:10.930292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:10.930292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:10.930292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:10.930292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:10.930292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:10.930292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:10.930292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:10.930292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:10.931229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:10.932097 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:10.932097 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:10.932097 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:10.932097 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:10.932097 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 338/673] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:10.932097 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afb9ee-489c-413d-936e-8faafc4ff204 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:10.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc6564-d6b8-4929-97bf-09dae64b1cfb None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:10.971803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:10.987512 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:11.009768 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b69bd7-5e3d-4292-9692-78f658804a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:11.010740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b69bd7-5e3d-4292-9692-78f658804a62 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:11.011190 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b69bd7-5e3d-4292-9692-78f658804a62 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:11.011604 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b69bd7-5e3d-4292-9692-78f658804a62 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:11.012541 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b69bd7-5e3d-4292-9692-78f658804a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:11.012991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b69bd7-5e3d-4292-9692-78f658804a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:11.014685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b69bd7-5e3d-4292-9692-78f658804a62 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:11.030909 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 336/674] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:11.032312 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b69bd7-5e3d-4292-9692-78f658804a62 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:11.054681 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afb9ee-489c-413d-936e-8faafc4ff204 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:11.100379 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ec5bb-c352-4d23-adfd-75295b857d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:11.100982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ec5bb-c352-4d23-adfd-75295b857d30 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:11.101261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ec5bb-c352-4d23-adfd-75295b857d30 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:11.101861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ec5bb-c352-4d23-adfd-75295b857d30 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:11.102632 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ec5bb-c352-4d23-adfd-75295b857d30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:11.102935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ec5bb-c352-4d23-adfd-75295b857d30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:11.104313 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ec5bb-c352-4d23-adfd-75295b857d30 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:11.122418 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 339/675] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:11.123292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ec5bb-c352-4d23-adfd-75295b857d30 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:11.179215 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b69bd7-5e3d-4292-9692-78f658804a62 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:11.229704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4e100-16a3-4490-9cef-869a07b78b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:11.230938 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4e100-16a3-4490-9cef-869a07b78b12 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:11.231409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4e100-16a3-4490-9cef-869a07b78b12 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:11.232089 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4e100-16a3-4490-9cef-869a07b78b12 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:11.233445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4e100-16a3-4490-9cef-869a07b78b12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:11.234230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4e100-16a3-4490-9cef-869a07b78b12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:11.236303 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4e100-16a3-4490-9cef-869a07b78b12 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:11.253275 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ec5bb-c352-4d23-adfd-75295b857d30 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:11.254120 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 337/676] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:11.255491 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4e100-16a3-4490-9cef-869a07b78b12 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:11.267524 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:11.292015 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:11.293980 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbf2a6-d2c7-452a-ac68-4d5adb7902c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:11.294898 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbf2a6-d2c7-452a-ac68-4d5adb7902c3 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:11.295250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbf2a6-d2c7-452a-ac68-4d5adb7902c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:11.295593 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbf2a6-d2c7-452a-ac68-4d5adb7902c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:11.304021 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbf2a6-d2c7-452a-ac68-4d5adb7902c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:11.305637 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbf2a6-d2c7-452a-ac68-4d5adb7902c3 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:11.330294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-5dbbf2a6-d2c7-452a-ac68-4d5adb7902c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:11.330294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:11.330294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:11.330294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:11.330294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:11.330294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:11.330294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:11.330294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:11.330294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:11.330294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:11.330294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:11.330294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:11.330294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:11.330294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:11.330294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:11.331226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:11.332092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:11.332092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:11.332092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:11.332092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:11.332092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:11.332092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:11.332092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:11.332092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:11.339133 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 340/677] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:11.339263 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbf2a6-d2c7-452a-ac68-4d5adb7902c3 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:11.383770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4e100-16a3-4490-9cef-869a07b78b12 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:11.419296 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338aff9c-b17e-4d65-a75b-a20e05edbbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:11.420111 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338aff9c-b17e-4d65-a75b-a20e05edbbd6 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:11.420387 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338aff9c-b17e-4d65-a75b-a20e05edbbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:11.421018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338aff9c-b17e-4d65-a75b-a20e05edbbd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:11.428844 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338aff9c-b17e-4d65-a75b-a20e05edbbd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:11.431420 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338aff9c-b17e-4d65-a75b-a20e05edbbd6 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:11.447514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-338aff9c-b17e-4d65-a75b-a20e05edbbd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:11.447514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:11.447514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:11.447514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:11.447514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:11.447514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:11.447514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:11.447514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:11.447514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:11.447514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:11.447514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:11.447514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:11.447514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:11.447514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:11.447514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:11.448484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:11.449340 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:11.449340 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:11.449340 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:11.449340 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:11.449340 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:11.449340 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:11.449340 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:11.449340 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:11.453356 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 338/678] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:11.454208 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338aff9c-b17e-4d65-a75b-a20e05edbbd6 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:11.474606 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbf2a6-d2c7-452a-ac68-4d5adb7902c3 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:11.518648 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900c7a0-99ed-4565-a993-9afd59a675f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:11.519775 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900c7a0-99ed-4565-a993-9afd59a675f6 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:11.520335 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900c7a0-99ed-4565-a993-9afd59a675f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:11.520817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900c7a0-99ed-4565-a993-9afd59a675f6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:11.529476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900c7a0-99ed-4565-a993-9afd59a675f6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:11.531355 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900c7a0-99ed-4565-a993-9afd59a675f6 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-9900c7a0-99ed-4565-a993-9afd59a675f6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:11.547294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:11.548230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:11.549299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:11.549299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:11.549299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:11.549299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:11.549299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:11.549299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:11.549299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:11.549299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:11.549299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:37:11.549299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:11.550616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:11.552691 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 341/679] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:37:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:11.554066 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900c7a0-99ed-4565-a993-9afd59a675f6 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:11.568396 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338aff9c-b17e-4d65-a75b-a20e05edbbd6 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:11.588299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:11.605612 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f985737-5652-4047-8f8f-84bd79de7448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:11.606457 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f985737-5652-4047-8f8f-84bd79de7448 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:11.606804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f985737-5652-4047-8f8f-84bd79de7448 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:11.607131 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f985737-5652-4047-8f8f-84bd79de7448 None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:11.607983 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f985737-5652-4047-8f8f-84bd79de7448 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:11.608356 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f985737-5652-4047-8f8f-84bd79de7448 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:11.609926 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f985737-5652-4047-8f8f-84bd79de7448 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:11.628702 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 339/680] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:11.628702 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f985737-5652-4047-8f8f-84bd79de7448 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:11.679225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900c7a0-99ed-4565-a993-9afd59a675f6 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:11.716387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769f8ae-0b6f-4e60-a94c-88ea1f393a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:11.717063 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769f8ae-0b6f-4e60-a94c-88ea1f393a92 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:11.717343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769f8ae-0b6f-4e60-a94c-88ea1f393a92 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:11.717619 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769f8ae-0b6f-4e60-a94c-88ea1f393a92 None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:11.718432 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769f8ae-0b6f-4e60-a94c-88ea1f393a92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:11.718740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769f8ae-0b6f-4e60-a94c-88ea1f393a92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:11.720261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769f8ae-0b6f-4e60-a94c-88ea1f393a92 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:11.732808 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 342/681] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:11.733612 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769f8ae-0b6f-4e60-a94c-88ea1f393a92 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:11.733917 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f985737-5652-4047-8f8f-84bd79de7448 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:11.777815 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90cc0b-8d87-4f09-9b4a-b85fead57342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:11.778435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90cc0b-8d87-4f09-9b4a-b85fead57342 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:11.778714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90cc0b-8d87-4f09-9b4a-b85fead57342 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:11.779117 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90cc0b-8d87-4f09-9b4a-b85fead57342 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:11.779868 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90cc0b-8d87-4f09-9b4a-b85fead57342 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:11.780217 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90cc0b-8d87-4f09-9b4a-b85fead57342 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:11.781603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90cc0b-8d87-4f09-9b4a-b85fead57342 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:11.801790 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 340/682] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 19:37:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:11.803260 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4e500-70c6-4744-a665-fb0a81e92ed9 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:11.803617 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4e500-70c6-4744-a665-fb0a81e92ed9 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:11.803958 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4e500-70c6-4744-a665-fb0a81e92ed9 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:11.805391 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 341/683] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:11.806016 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4e500-70c6-4744-a665-fb0a81e92ed9 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:11.808879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:11.835926 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:11.846524 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769f8ae-0b6f-4e60-a94c-88ea1f393a92 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:11.881819 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0280ac-5401-4a97-a284-12cadfacb59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:11.882621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0280ac-5401-4a97-a284-12cadfacb59c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:11.882954 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0280ac-5401-4a97-a284-12cadfacb59c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:11.883293 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0280ac-5401-4a97-a284-12cadfacb59c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:11.884141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0280ac-5401-4a97-a284-12cadfacb59c None admin] RBAC: Adding query filter params (role.id=cefdc5c73ec347438e5b4687f191382c, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=423cd0a1227940d1bd7e4263348f45c9) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:11.884443 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0280ac-5401-4a97-a284-12cadfacb59c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:11.885865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0280ac-5401-4a97-a284-12cadfacb59c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:11.891825 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 343/684] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:11 2022] GET /identity/v3/role_assignments?user.id=423cd0a1227940d1bd7e4263348f45c9&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=cefdc5c73ec347438e5b4687f191382c => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:11.892882 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0280ac-5401-4a97-a284-12cadfacb59c None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:11.932690 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4e500-70c6-4744-a665-fb0a81e92ed9 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:11.981628 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8f3d0-7258-48d8-9c18-fc521d969607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:11.982716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8f3d0-7258-48d8-9c18-fc521d969607 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:11.983165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8f3d0-7258-48d8-9c18-fc521d969607 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:11.983875 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8f3d0-7258-48d8-9c18-fc521d969607 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:11.985597 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8f3d0-7258-48d8-9c18-fc521d969607 None admin] RBAC: Adding query filter params (role.id=cefdc5c73ec347438e5b4687f191382c, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=d2b98b1e4024410a87addde112237f32) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:11.986702 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8f3d0-7258-48d8-9c18-fc521d969607 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:11.988754 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8f3d0-7258-48d8-9c18-fc521d969607 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:12.003050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 342/685] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:11 2022] GET /identity/v3/role_assignments?user.id=d2b98b1e4024410a87addde112237f32&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=cefdc5c73ec347438e5b4687f191382c => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:12.005095 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61825246-4788-4db7-8734-aefad88dbd79 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:12.005429 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61825246-4788-4db7-8734-aefad88dbd79 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:12.005647 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61825246-4788-4db7-8734-aefad88dbd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:12.041007 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-61825246-4788-4db7-8734-aefad88dbd79 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:12.042279 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-61825246-4788-4db7-8734-aefad88dbd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:12.054318 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0280ac-5401-4a97-a284-12cadfacb59c None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:12.121178 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b7601-8e66-43b1-89d9-982fafe15559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:12.121845 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b7601-8e66-43b1-89d9-982fafe15559 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:12.122119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b7601-8e66-43b1-89d9-982fafe15559 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:12.122410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b7601-8e66-43b1-89d9-982fafe15559 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:12.156728 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:12.165972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:12.178377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 343/686] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:37:12.204235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61825246-4788-4db7-8734-aefad88dbd79 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:12.215154 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b7601-8e66-43b1-89d9-982fafe15559 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:12.216188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b7601-8e66-43b1-89d9-982fafe15559 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:12.337725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61825246-4788-4db7-8734-aefad88dbd79 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:12.350432 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 344/687] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:37:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:37:12.357762 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b7601-8e66-43b1-89d9-982fafe15559 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:12.377184 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022ec18-78fa-4e7e-b9f0-c999115f3f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:12.378504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022ec18-78fa-4e7e-b9f0-c999115f3f68 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:12.379043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022ec18-78fa-4e7e-b9f0-c999115f3f68 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:12.379512 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022ec18-78fa-4e7e-b9f0-c999115f3f68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:12.400608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c022ec18-78fa-4e7e-b9f0-c999115f3f68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:12.403177 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c022ec18-78fa-4e7e-b9f0-c999115f3f68 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:12.435011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-c022ec18-78fa-4e7e-b9f0-c999115f3f68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:12.435011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:12.435011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:12.435011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:12.435011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:12.435011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:12.435011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:12.435011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:12.435011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:12.435011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:12.435011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:12.435011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:12.435011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:12.435011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:12.435011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:12.435961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:12.436872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:12.436872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:12.436872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:12.436872 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:12.443279 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 344/688] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:12.450241 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022ec18-78fa-4e7e-b9f0-c999115f3f68 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:12.455766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:12.530320 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:12.556532 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b7601-8e66-43b1-89d9-982fafe15559 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:12.628328 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb1787-b0f2-4fdc-85b7-90b13c4053ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:12.628682 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb1787-b0f2-4fdc-85b7-90b13c4053ca None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:12.629098 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb1787-b0f2-4fdc-85b7-90b13c4053ca None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:12.629512 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb1787-b0f2-4fdc-85b7-90b13c4053ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:12.639274 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022ec18-78fa-4e7e-b9f0-c999115f3f68 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:12.661446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb1787-b0f2-4fdc-85b7-90b13c4053ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:12.663406 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb1787-b0f2-4fdc-85b7-90b13c4053ca None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:12.716647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-9fdb1787-b0f2-4fdc-85b7-90b13c4053ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:12.716647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:12.716647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:12.716647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:12.716647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:12.716647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:12.716647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:12.716647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:12.716647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:12.716647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:12.716647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:12.716647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:12.716647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:12.716647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:12.716647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:12.719515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:12.721526 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:12.721526 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:12.721526 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:12.721526 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:12.721526 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 345/689] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:12.728956 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb1787-b0f2-4fdc-85b7-90b13c4053ca None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:12.755142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb4e9b-d89b-4bdd-9ffe-52bbe119db63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:12.756283 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb4e9b-d89b-4bdd-9ffe-52bbe119db63 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:12.756766 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb4e9b-d89b-4bdd-9ffe-52bbe119db63 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:12.757249 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb4e9b-d89b-4bdd-9ffe-52bbe119db63 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:12.758321 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb4e9b-d89b-4bdd-9ffe-52bbe119db63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:12.758840 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb4e9b-d89b-4bdd-9ffe-52bbe119db63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:12.760440 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb4e9b-d89b-4bdd-9ffe-52bbe119db63 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:12.797130 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 345/690] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:12.798538 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb4e9b-d89b-4bdd-9ffe-52bbe119db63 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:12.854285 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb1787-b0f2-4fdc-85b7-90b13c4053ca None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:12.860130 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:12.891346 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4174a6-7aa1-4929-bf0e-2fc21fca205c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:12.892523 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4174a6-7aa1-4929-bf0e-2fc21fca205c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:12.893095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4174a6-7aa1-4929-bf0e-2fc21fca205c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:12.893581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4174a6-7aa1-4929-bf0e-2fc21fca205c None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:12.894565 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4174a6-7aa1-4929-bf0e-2fc21fca205c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:12.895075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4174a6-7aa1-4929-bf0e-2fc21fca205c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:12.896748 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4174a6-7aa1-4929-bf0e-2fc21fca205c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:12.910073 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 346/691] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:12.915324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4174a6-7aa1-4929-bf0e-2fc21fca205c None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:13.012395 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb4e9b-d89b-4bdd-9ffe-52bbe119db63 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:13.032628 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:13.035608 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4174a6-7aa1-4929-bf0e-2fc21fca205c None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:13.076814 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0ca9a-dfe3-4fec-9f24-f47ad128f002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:13.078020 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0ca9a-dfe3-4fec-9f24-f47ad128f002 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:13.078578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0ca9a-dfe3-4fec-9f24-f47ad128f002 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:13.079050 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0ca9a-dfe3-4fec-9f24-f47ad128f002 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:13.089031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0ca9a-dfe3-4fec-9f24-f47ad128f002 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:13.090915 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0ca9a-dfe3-4fec-9f24-f47ad128f002 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:13.108765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-e0c0ca9a-dfe3-4fec-9f24-f47ad128f002 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:13.108765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:13.108765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:13.108765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:13.108765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:13.108765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:13.108765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:13.108765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:13.108765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:13.108765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:13.108765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:13.108765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:13.108765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:13.108765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:13.108765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:13.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:13.110693 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:13.110693 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:13.110693 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:13.110693 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:13.110693 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:13.110693 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:13.110693 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:13.110693 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:13.113503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 347/692] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:13.118814 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0ca9a-dfe3-4fec-9f24-f47ad128f002 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:13.118972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd261b5-49cb-4a50-ab0a-1f26c0593530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:13.119665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd261b5-49cb-4a50-ab0a-1f26c0593530 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:13.119945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd261b5-49cb-4a50-ab0a-1f26c0593530 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:13.120319 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd261b5-49cb-4a50-ab0a-1f26c0593530 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:13.121028 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd261b5-49cb-4a50-ab0a-1f26c0593530 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:13.122520 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd261b5-49cb-4a50-ab0a-1f26c0593530 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:13.151933 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:13.163808 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 346/693] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:37:12 2022] POST /identity/v3/users => generated 254 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:37:13.234609 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0ca9a-dfe3-4fec-9f24-f47ad128f002 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:13.272945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f37a5d-8914-4455-b0d0-f3a6348caba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:13.274070 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f37a5d-8914-4455-b0d0-f3a6348caba0 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:13.274567 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f37a5d-8914-4455-b0d0-f3a6348caba0 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:13.275036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f37a5d-8914-4455-b0d0-f3a6348caba0 None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:13.276038 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f37a5d-8914-4455-b0d0-f3a6348caba0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:13.276541 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f37a5d-8914-4455-b0d0-f3a6348caba0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:13.278218 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f37a5d-8914-4455-b0d0-f3a6348caba0 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:13.291963 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 348/694] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:13.297782 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd261b5-49cb-4a50-ab0a-1f26c0593530 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:13.303852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90073e-7182-419f-81c6-a00328f5096e None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:13.304486 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90073e-7182-419f-81c6-a00328f5096e None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:13.305080 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90073e-7182-419f-81c6-a00328f5096e None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:13.306903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 349/695] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:37:13.324759 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83e9f9-1cf2-48f3-aa12-1ff4deddc293 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:13.325472 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83e9f9-1cf2-48f3-aa12-1ff4deddc293 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:13.325971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83e9f9-1cf2-48f3-aa12-1ff4deddc293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:13.352853 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-0e83e9f9-1cf2-48f3-aa12-1ff4deddc293 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:13.354601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-0e83e9f9-1cf2-48f3-aa12-1ff4deddc293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:13.418374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd261b5-49cb-4a50-ab0a-1f26c0593530 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:13.455509 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bee04-a5d5-4fe8-b0f4-8ae560658e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:13.456176 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bee04-a5d5-4fe8-b0f4-8ae560658e5a None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:13.456449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bee04-a5d5-4fe8-b0f4-8ae560658e5a None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:13.456714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bee04-a5d5-4fe8-b0f4-8ae560658e5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:13.457559 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bee04-a5d5-4fe8-b0f4-8ae560658e5a None admin] RBAC: Adding query filter params (role.id=cefdc5c73ec347438e5b4687f191382c, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=4d1c8759f0564f28b388106dbf615cb5) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:13.457862 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bee04-a5d5-4fe8-b0f4-8ae560658e5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:13.459359 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bee04-a5d5-4fe8-b0f4-8ae560658e5a None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:13.474857 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 347/696] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:13 2022] GET /identity/v3/role_assignments?user.id=4d1c8759f0564f28b388106dbf615cb5&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=cefdc5c73ec347438e5b4687f191382c => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:13.518463 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 350/697] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:37:13.536109 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bee04-a5d5-4fe8-b0f4-8ae560658e5a None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:13.569419 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:13.649713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bee04-a5d5-4fe8-b0f4-8ae560658e5a None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:13.687456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d597a-7cf8-4a42-9b15-d4a79bc5c108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:13.688247 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d597a-7cf8-4a42-9b15-d4a79bc5c108 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:13.688494 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d597a-7cf8-4a42-9b15-d4a79bc5c108 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:13.688757 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d597a-7cf8-4a42-9b15-d4a79bc5c108 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:13.700628 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d597a-7cf8-4a42-9b15-d4a79bc5c108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-794d597a-7cf8-4a42-9b15-d4a79bc5c108 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:13.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:13.709350 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:13.710166 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:13.710166 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:13.710166 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:13.710166 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:13.710166 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:13.710166 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:13.710166 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:13.710166 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:13.710166 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:13.710166 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:13.710166 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:37:13.710166 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:13.711955 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 348/698] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:37:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:13.715856 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83e9f9-1cf2-48f3-aa12-1ff4deddc293 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:13.734592 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:13.828617 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83e9f9-1cf2-48f3-aa12-1ff4deddc293 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:13.864082 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b649dc-cad4-44e3-a741-f8d3dd370213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:13.865591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b649dc-cad4-44e3-a741-f8d3dd370213 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:13.866068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b649dc-cad4-44e3-a741-f8d3dd370213 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:13.866553 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b649dc-cad4-44e3-a741-f8d3dd370213 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:13.867522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b649dc-cad4-44e3-a741-f8d3dd370213 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:13.868028 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b649dc-cad4-44e3-a741-f8d3dd370213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:13.869611 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b649dc-cad4-44e3-a741-f8d3dd370213 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:13.879171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 351/699] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:37:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:13.884376 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d597a-7cf8-4a42-9b15-d4a79bc5c108 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:13.990009 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:14.004723 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d597a-7cf8-4a42-9b15-d4a79bc5c108 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:14.040490 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5c1ed-e7cb-4e06-ae2e-0de06db779c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:14.041659 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5c1ed-e7cb-4e06-ae2e-0de06db779c9 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:14.042194 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5c1ed-e7cb-4e06-ae2e-0de06db779c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:14.042710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5c1ed-e7cb-4e06-ae2e-0de06db779c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:14.198756 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf5c1ed-e7cb-4e06-ae2e-0de06db779c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:14.200664 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf5c1ed-e7cb-4e06-ae2e-0de06db779c9 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:14.219459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-fdf5c1ed-e7cb-4e06-ae2e-0de06db779c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:14.219459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:14.219459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:14.219459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:14.219459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:14.219459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:14.219459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:14.219459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:14.219459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:14.219459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:14.219459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:14.219459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:14.219459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:14.219459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:14.219459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:14.220495 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:14.221863 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:14.221863 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:14.221863 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:14.221863 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:14.224288 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 349/700] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:14.231636 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b649dc-cad4-44e3-a741-f8d3dd370213 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:14.332094 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:14.369817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b649dc-cad4-44e3-a741-f8d3dd370213 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:14.417538 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c35051-f1da-4bdf-8e03-cb9dd936d635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:14.418200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c35051-f1da-4bdf-8e03-cb9dd936d635 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:14.418496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c35051-f1da-4bdf-8e03-cb9dd936d635 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:14.418768 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c35051-f1da-4bdf-8e03-cb9dd936d635 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:14.419478 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c35051-f1da-4bdf-8e03-cb9dd936d635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:14.419775 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c35051-f1da-4bdf-8e03-cb9dd936d635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:14.421127 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c35051-f1da-4bdf-8e03-cb9dd936d635 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:14.437992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 352/701] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:14.440035 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5c1ed-e7cb-4e06-ae2e-0de06db779c9 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:14.576455 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8757bfa-dee0-4adf-963f-eb639ff51fe1 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:14.576826 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8757bfa-dee0-4adf-963f-eb639ff51fe1 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:14.577057 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8757bfa-dee0-4adf-963f-eb639ff51fe1 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:14.577159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5c1ed-e7cb-4e06-ae2e-0de06db779c9 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:14.578769 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 353/702] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:37:14.587015 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74f742-07d7-4432-b382-2525e2cf8028 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:14.587197 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74f742-07d7-4432-b382-2525e2cf8028 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:14.587479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74f742-07d7-4432-b382-2525e2cf8028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:14.609547 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-de74f742-07d7-4432-b382-2525e2cf8028 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:14.610781 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-de74f742-07d7-4432-b382-2525e2cf8028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:14.641163 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb27ed0-6255-40a7-baeb-78f661af0fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:14.642466 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb27ed0-6255-40a7-baeb-78f661af0fac None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:14.643071 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb27ed0-6255-40a7-baeb-78f661af0fac None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:14.643520 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb27ed0-6255-40a7-baeb-78f661af0fac None admin] PATH_INFO: `/v3/users/nova` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:14.664046 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb27ed0-6255-40a7-baeb-78f661af0fac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:14.665600 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb27ed0-6255-40a7-baeb-78f661af0fac None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:14.676171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-ecb27ed0-6255-40a7-baeb-78f661af0fac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:14.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:14.693097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:14.694100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:14.694100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:14.694100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:14.694100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:14.694100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:14.694100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:14.694100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:14.694100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:14.694100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:37:14.694100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:14.701250 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 350/703] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:37:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:14.708367 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb27ed0-6255-40a7-baeb-78f661af0fac None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:14.715396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 354/704] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:37:14.763460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74f742-07d7-4432-b382-2525e2cf8028 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:14.864707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb27ed0-6255-40a7-baeb-78f661af0fac None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:14.887495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:14.902074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1535c0-3350-4a8e-af86-bb93eaa250d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:14.903130 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1535c0-3350-4a8e-af86-bb93eaa250d8 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:14.903631 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1535c0-3350-4a8e-af86-bb93eaa250d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:14.904098 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1535c0-3350-4a8e-af86-bb93eaa250d8 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:14.905126 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1535c0-3350-4a8e-af86-bb93eaa250d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:14.905627 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1535c0-3350-4a8e-af86-bb93eaa250d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:14.907287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1535c0-3350-4a8e-af86-bb93eaa250d8 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:14.928297 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 351/705] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:37:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:14.938875 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1535c0-3350-4a8e-af86-bb93eaa250d8 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:14.965132 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74f742-07d7-4432-b382-2525e2cf8028 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:15.003273 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6939ce5-8edc-4721-80f9-8d0ebc4ef5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:15.004063 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6939ce5-8edc-4721-80f9-8d0ebc4ef5b3 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:15.004338 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6939ce5-8edc-4721-80f9-8d0ebc4ef5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:15.004606 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6939ce5-8edc-4721-80f9-8d0ebc4ef5b3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:15.005252 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6939ce5-8edc-4721-80f9-8d0ebc4ef5b3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:15.006643 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6939ce5-8edc-4721-80f9-8d0ebc4ef5b3 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:15.013815 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-f6939ce5-8edc-4721-80f9-8d0ebc4ef5b3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:15.020815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:15.021816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:15.022674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 355/706] 10.222.0.84 () {56 vars in 1129 bytes} [Thu May 19 19:37:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:15.024048 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62481f58-00c3-47ed-809e-ff5d128e9ea2 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:15.024048 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62481f58-00c3-47ed-809e-ff5d128e9ea2 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:15.024203 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62481f58-00c3-47ed-809e-ff5d128e9ea2 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:15.029538 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 356/707] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:15.030339 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62481f58-00c3-47ed-809e-ff5d128e9ea2 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:15.069964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1535c0-3350-4a8e-af86-bb93eaa250d8 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:15.116463 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4160a-9ec7-4eed-924d-035983be419c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:15.117808 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4160a-9ec7-4eed-924d-035983be419c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:15.118388 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4160a-9ec7-4eed-924d-035983be419c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:15.118820 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4160a-9ec7-4eed-924d-035983be419c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:15.140307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62481f58-00c3-47ed-809e-ff5d128e9ea2 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:15.181713 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:15.189906 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c9c99-d17f-4e2b-ba7b-1fa930c101c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:15.191116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c9c99-d17f-4e2b-ba7b-1fa930c101c6 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:15.191546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c9c99-d17f-4e2b-ba7b-1fa930c101c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:15.192556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c9c99-d17f-4e2b-ba7b-1fa930c101c6 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:15.193421 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c9c99-d17f-4e2b-ba7b-1fa930c101c6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:15.194108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c9c99-d17f-4e2b-ba7b-1fa930c101c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:15.194716 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4160a-9ec7-4eed-924d-035983be419c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:15.195908 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c9c99-d17f-4e2b-ba7b-1fa930c101c6 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:15.196336 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4160a-9ec7-4eed-924d-035983be419c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:15.207738 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 357/708] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 19:37:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:15.211124 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60bcc1-12cd-416b-a073-6c94e8e25276 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:15.211837 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60bcc1-12cd-416b-a073-6c94e8e25276 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:15.212775 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60bcc1-12cd-416b-a073-6c94e8e25276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:15.238760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-7c60bcc1-12cd-416b-a073-6c94e8e25276 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:15.240209 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-7c60bcc1-12cd-416b-a073-6c94e8e25276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:15.260881 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:15.312993 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 352/709] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:37:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:37:15.314235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4160a-9ec7-4eed-924d-035983be419c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:15.330329 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 358/710] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:37:15.330329 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60bcc1-12cd-416b-a073-6c94e8e25276 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:15.438453 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4160a-9ec7-4eed-924d-035983be419c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:15.463699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60bcc1-12cd-416b-a073-6c94e8e25276 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:15.475021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e28aae-ad71-4c31-b498-a194c6e6fa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:15.475709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e28aae-ad71-4c31-b498-a194c6e6fa0a None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:15.475985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e28aae-ad71-4c31-b498-a194c6e6fa0a None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:15.476252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e28aae-ad71-4c31-b498-a194c6e6fa0a None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:15.477299 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e28aae-ad71-4c31-b498-a194c6e6fa0a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:15.477603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e28aae-ad71-4c31-b498-a194c6e6fa0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:15.479060 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e28aae-ad71-4c31-b498-a194c6e6fa0a None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:15.486173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 353/711] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 19:37:15 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:15.487481 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e28aae-ad71-4c31-b498-a194c6e6fa0a None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:15.522369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:15.612196 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e28aae-ad71-4c31-b498-a194c6e6fa0a None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:15.634989 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:15.646775 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd23c6-e96f-488a-b10e-25ed9b490168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:15.648664 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd23c6-e96f-488a-b10e-25ed9b490168 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:15.649108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd23c6-e96f-488a-b10e-25ed9b490168 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:15.649543 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd23c6-e96f-488a-b10e-25ed9b490168 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:15.657788 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538dce6-5e4d-4d6d-bb7c-c00d1bed86d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:15.657788 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538dce6-5e4d-4d6d-bb7c-c00d1bed86d0 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:15.657788 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538dce6-5e4d-4d6d-bb7c-c00d1bed86d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:15.657788 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538dce6-5e4d-4d6d-bb7c-c00d1bed86d0 None admin] PATH_INFO: `/v3/services/network` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:15.658414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538dce6-5e4d-4d6d-bb7c-c00d1bed86d0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:15.658414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538dce6-5e4d-4d6d-bb7c-c00d1bed86d0 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:15.682935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd23c6-e96f-488a-b10e-25ed9b490168 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:15.684420 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd23c6-e96f-488a-b10e-25ed9b490168 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-1538dce6-5e4d-4d6d-bb7c-c00d1bed86d0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:15.685077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:15.686435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:15.687344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:15.687344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:15.687344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:15.687344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:15.687344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:15.687344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:15.687344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:15.687344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:15.687344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:15.687344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:15.687344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:15.687344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:15.687344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:37:15.687344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:15.689783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 354/712] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:15 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:15.693277 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538dce6-5e4d-4d6d-bb7c-c00d1bed86d0 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:15.719005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-9cdd23c6-e96f-488a-b10e-25ed9b490168 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:15.719005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:15.719005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:15.719005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:15.719005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:15.719005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:15.719005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:15.719005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:15.719005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:15.719005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:15.719005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:15.719005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:15.719005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:15.719005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:15.719005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:15.720014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:15.721520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:15.721520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:15.721520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:15.721520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:15.723150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 359/713] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:15.731596 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd23c6-e96f-488a-b10e-25ed9b490168 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:15.853048 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd23c6-e96f-488a-b10e-25ed9b490168 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:15.883398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:15.895713 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9852a2-2c16-4795-92de-f2edfae7f4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:15.897574 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9852a2-2c16-4795-92de-f2edfae7f4ac None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:15.898352 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9852a2-2c16-4795-92de-f2edfae7f4ac None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:15.898876 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9852a2-2c16-4795-92de-f2edfae7f4ac None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:15.899821 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9852a2-2c16-4795-92de-f2edfae7f4ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:15.899927 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538dce6-5e4d-4d6d-bb7c-c00d1bed86d0 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:15.900422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9852a2-2c16-4795-92de-f2edfae7f4ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:15.903229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9852a2-2c16-4795-92de-f2edfae7f4ac None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:15.919765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 360/714] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:15.927683 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9852a2-2c16-4795-92de-f2edfae7f4ac None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:15.947518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f59c7-35ea-493f-a91d-d67a22bcbd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:15.948110 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f59c7-35ea-493f-a91d-d67a22bcbd2b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:15.948387 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f59c7-35ea-493f-a91d-d67a22bcbd2b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:15.948701 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f59c7-35ea-493f-a91d-d67a22bcbd2b None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:15.949405 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f59c7-35ea-493f-a91d-d67a22bcbd2b None admin] RBAC: Adding query filter params (name=network) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:15.949792 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f59c7-35ea-493f-a91d-d67a22bcbd2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:15.951200 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f59c7-35ea-493f-a91d-d67a22bcbd2b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:15.960278 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 355/715] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:37:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:15.966523 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f59c7-35ea-493f-a91d-d67a22bcbd2b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:15.998085 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:16.098252 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9852a2-2c16-4795-92de-f2edfae7f4ac None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:16.162727 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f59c7-35ea-493f-a91d-d67a22bcbd2b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:16.169405 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21a180-fee6-4ded-b9da-3dbf4696dc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:16.170218 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21a180-fee6-4ded-b9da-3dbf4696dc88 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:16.170520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21a180-fee6-4ded-b9da-3dbf4696dc88 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:16.170787 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21a180-fee6-4ded-b9da-3dbf4696dc88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:16.186535 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21a180-fee6-4ded-b9da-3dbf4696dc88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:16.187998 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21a180-fee6-4ded-b9da-3dbf4696dc88 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:16.214726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-7c21a180-fee6-4ded-b9da-3dbf4696dc88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:16.214726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:16.214726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:16.214726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:16.214726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:16.214726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:16.214726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:16.214726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:16.214726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:16.214726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:16.214726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:16.214726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:16.214726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:16.214726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:16.214726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:16.215628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:16.216495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:16.216495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:16.216495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:16.216495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:16.216495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:16.216495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:16.216495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:16.216495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:16.217384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 361/716] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:16.225487 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21a180-fee6-4ded-b9da-3dbf4696dc88 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:16.242747 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796cc67-415b-4a61-951c-4c59303af7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:16.243438 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796cc67-415b-4a61-951c-4c59303af7d7 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:16.243710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796cc67-415b-4a61-951c-4c59303af7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:16.243976 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796cc67-415b-4a61-951c-4c59303af7d7 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:16.244701 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796cc67-415b-4a61-951c-4c59303af7d7 None admin] RBAC: Adding query filter params (type=network) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:16.244999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796cc67-415b-4a61-951c-4c59303af7d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:16.246499 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796cc67-415b-4a61-951c-4c59303af7d7 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:16.257281 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 356/717] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:37:15 2022] GET /identity/v3/services?type=network => generated 124 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:16.343836 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27193d-fb7c-4fd8-8f1c-d651905a94d0 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:16.344159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27193d-fb7c-4fd8-8f1c-d651905a94d0 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:16.344379 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27193d-fb7c-4fd8-8f1c-d651905a94d0 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:16.346571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 357/718] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:37:16.349762 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:16.357820 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21a180-fee6-4ded-b9da-3dbf4696dc88 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:16.360266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc65838-c8e2-4497-a252-706d0c139561 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:16.360532 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc65838-c8e2-4497-a252-706d0c139561 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:16.360850 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc65838-c8e2-4497-a252-706d0c139561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:16.387860 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-bcc65838-c8e2-4497-a252-706d0c139561 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:16.389501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-bcc65838-c8e2-4497-a252-706d0c139561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:16.418674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:16.442718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49856d44-f6dc-4f06-bb0b-28f7740dbe45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:16.443734 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49856d44-f6dc-4f06-bb0b-28f7740dbe45 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:16.444196 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49856d44-f6dc-4f06-bb0b-28f7740dbe45 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:16.445787 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49856d44-f6dc-4f06-bb0b-28f7740dbe45 None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:16.446855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49856d44-f6dc-4f06-bb0b-28f7740dbe45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:16.448671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49856d44-f6dc-4f06-bb0b-28f7740dbe45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:16.450460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49856d44-f6dc-4f06-bb0b-28f7740dbe45 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:16.469016 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 362/719] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:16.470981 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ebfe5-205b-4a74-a305-13d17c494458 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:16.471619 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ebfe5-205b-4a74-a305-13d17c494458 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:16.472076 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ebfe5-205b-4a74-a305-13d17c494458 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:16.473889 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 363/720] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:16.482410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ebfe5-205b-4a74-a305-13d17c494458 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:16.492312 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 358/721] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:37:16.494184 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471954c4-b836-46a6-b463-5b1756260999 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:16.494678 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471954c4-b836-46a6-b463-5b1756260999 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:16.495112 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471954c4-b836-46a6-b463-5b1756260999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:16.518440 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-471954c4-b836-46a6-b463-5b1756260999 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:16.520163 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-471954c4-b836-46a6-b463-5b1756260999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:16.603816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 359/722] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:37:16.605144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471954c4-b836-46a6-b463-5b1756260999 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:16.615722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ebfe5-205b-4a74-a305-13d17c494458 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:16.652161 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59932fa9-3954-46be-a3a1-5b154038859c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:16.652636 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59932fa9-3954-46be-a3a1-5b154038859c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:16.652913 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59932fa9-3954-46be-a3a1-5b154038859c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:16.653175 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59932fa9-3954-46be-a3a1-5b154038859c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:16.653997 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59932fa9-3954-46be-a3a1-5b154038859c None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=abde7cc02fae4d9bb561a6642c5252a6) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:16.654324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59932fa9-3954-46be-a3a1-5b154038859c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:16.655766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59932fa9-3954-46be-a3a1-5b154038859c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:16.666320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 364/723] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:16 2022] GET /identity/v3/role_assignments?user.id=abde7cc02fae4d9bb561a6642c5252a6&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:16.666320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59932fa9-3954-46be-a3a1-5b154038859c None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:16.694979 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:16.712101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:16.753425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471954c4-b836-46a6-b463-5b1756260999 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:16.834632 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41838bed-b036-430d-9343-4e42c831cdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:16.835452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41838bed-b036-430d-9343-4e42c831cdae None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:16.835730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41838bed-b036-430d-9343-4e42c831cdae None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:16.836001 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41838bed-b036-430d-9343-4e42c831cdae None admin] PATH_INFO: `/v3/roles/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:16.856116 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41838bed-b036-430d-9343-4e42c831cdae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:16.866357 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59932fa9-3954-46be-a3a1-5b154038859c None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-41838bed-b036-430d-9343-4e42c831cdae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:16.871472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:16.872418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:16.873237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:16.873237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:16.873237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:16.873237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:16.873237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:16.873237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:16.873237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:16.873237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:16.873237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:16.873237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:16.873237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:16.873237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:16.884319 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 360/724] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:37:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:16.886847 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41838bed-b036-430d-9343-4e42c831cdae None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:16.919986 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c0cd7-6788-45bb-a9ef-94b0a39439e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:16.920918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c0cd7-6788-45bb-a9ef-94b0a39439e8 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:16.921299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c0cd7-6788-45bb-a9ef-94b0a39439e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:16.921580 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c0cd7-6788-45bb-a9ef-94b0a39439e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:16.946350 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c0cd7-6788-45bb-a9ef-94b0a39439e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-9a2c0cd7-6788-45bb-a9ef-94b0a39439e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:16.954615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:16.955594 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:16.956491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:16.956491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:16.956491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:16.956491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:16.956491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:16.956491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:16.956491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:16.956491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:16.956491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:16.956491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:16.956491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:37:16.956491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:16.959471 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 365/725] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:37:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:16.965471 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c0cd7-6788-45bb-a9ef-94b0a39439e8 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:17.054295 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41838bed-b036-430d-9343-4e42c831cdae None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:17.097971 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33202e3f-b808-4266-a17d-b7c773c7e696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:17.099829 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33202e3f-b808-4266-a17d-b7c773c7e696 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:17.101915 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33202e3f-b808-4266-a17d-b7c773c7e696 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:17.104012 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33202e3f-b808-4266-a17d-b7c773c7e696 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:17.105000 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33202e3f-b808-4266-a17d-b7c773c7e696 None admin] RBAC: Adding query filter params (name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:17.108718 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33202e3f-b808-4266-a17d-b7c773c7e696 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:17.110466 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33202e3f-b808-4266-a17d-b7c773c7e696 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:17.118592 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:17.127107 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 361/726] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:37:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:17.127532 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:17.134150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33202e3f-b808-4266-a17d-b7c773c7e696 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:17.164197 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c0cd7-6788-45bb-a9ef-94b0a39439e8 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:17.258103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33202e3f-b808-4266-a17d-b7c773c7e696 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:17.267088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8a599-1706-4e37-8047-94de90e72240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:17.268122 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8a599-1706-4e37-8047-94de90e72240 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:17.268415 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8a599-1706-4e37-8047-94de90e72240 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:17.269904 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8a599-1706-4e37-8047-94de90e72240 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:17.270734 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8a599-1706-4e37-8047-94de90e72240 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:17.271049 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8a599-1706-4e37-8047-94de90e72240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:17.272457 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8a599-1706-4e37-8047-94de90e72240 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:17.285978 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 366/727] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:37:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:17.291092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8a599-1706-4e37-8047-94de90e72240 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:17.295420 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420c2d8-5085-4a13-ac16-10353a938a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:17.297123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420c2d8-5085-4a13-ac16-10353a938a0a None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:17.297400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420c2d8-5085-4a13-ac16-10353a938a0a None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:17.297668 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420c2d8-5085-4a13-ac16-10353a938a0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:17.309709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420c2d8-5085-4a13-ac16-10353a938a0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:17.311184 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420c2d8-5085-4a13-ac16-10353a938a0a None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:17.331941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-7420c2d8-5085-4a13-ac16-10353a938a0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:17.331941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:17.331941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:17.331941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:17.331941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:17.331941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:17.331941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:17.331941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:17.331941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:17.331941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:17.331941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:17.331941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:17.331941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:17.331941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:17.331941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:17.332892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:17.333740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:17.333740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:17.333740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:17.333740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:17.333971 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 362/728] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:17.338791 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420c2d8-5085-4a13-ac16-10353a938a0a None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:17.434595 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:17.465100 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8a599-1706-4e37-8047-94de90e72240 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:17.500598 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421f2e1-e490-4b47-ba70-64c1f1dcd8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:17.501471 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421f2e1-e490-4b47-ba70-64c1f1dcd8dd None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:17.501751 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421f2e1-e490-4b47-ba70-64c1f1dcd8dd None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:17.502023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421f2e1-e490-4b47-ba70-64c1f1dcd8dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:17.513295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421f2e1-e490-4b47-ba70-64c1f1dcd8dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:17.514802 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421f2e1-e490-4b47-ba70-64c1f1dcd8dd None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:17.517373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:17.534072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-b421f2e1-e490-4b47-ba70-64c1f1dcd8dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:17.534072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:17.534072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:17.534072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:17.534072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:17.534072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:17.534072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:17.534072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:17.534072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:17.534072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:17.534072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:17.534072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:17.534072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:17.534072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:17.534072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:17.535625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:17.536969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:17.536969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:17.536969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:17.536969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:17.536969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 367/729] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:17.540765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421f2e1-e490-4b47-ba70-64c1f1dcd8dd None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:17.559894 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420c2d8-5085-4a13-ac16-10353a938a0a None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:17.638484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76899b23-5b49-4a2d-8354-1a9c339e9599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:17.639161 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76899b23-5b49-4a2d-8354-1a9c339e9599 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:17.639437 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76899b23-5b49-4a2d-8354-1a9c339e9599 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:17.639705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76899b23-5b49-4a2d-8354-1a9c339e9599 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:17.640437 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76899b23-5b49-4a2d-8354-1a9c339e9599 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:17.640745 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76899b23-5b49-4a2d-8354-1a9c339e9599 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:17.642280 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76899b23-5b49-4a2d-8354-1a9c339e9599 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:17.663981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 363/730] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:17.669326 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76899b23-5b49-4a2d-8354-1a9c339e9599 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:17.714405 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421f2e1-e490-4b47-ba70-64c1f1dcd8dd None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:17.749358 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc41ca4-8bae-42c8-8a38-7a8a85d8a724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:17.750023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc41ca4-8bae-42c8-8a38-7a8a85d8a724 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:17.750327 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc41ca4-8bae-42c8-8a38-7a8a85d8a724 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:17.750601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc41ca4-8bae-42c8-8a38-7a8a85d8a724 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:17.751391 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc41ca4-8bae-42c8-8a38-7a8a85d8a724 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:17.751706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc41ca4-8bae-42c8-8a38-7a8a85d8a724 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:17.753195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc41ca4-8bae-42c8-8a38-7a8a85d8a724 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:17.766040 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 368/731] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:17.775394 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc41ca4-8bae-42c8-8a38-7a8a85d8a724 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:17.854437 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:17.861029 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76899b23-5b49-4a2d-8354-1a9c339e9599 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:17.880313 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:17.900567 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc41ca4-8bae-42c8-8a38-7a8a85d8a724 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:17.935131 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82395e-97f0-4a81-94b3-a14be44434b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:17.936751 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82395e-97f0-4a81-94b3-a14be44434b6 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:17.937226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82395e-97f0-4a81-94b3-a14be44434b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:17.937705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82395e-97f0-4a81-94b3-a14be44434b6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:17.945549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82395e-97f0-4a81-94b3-a14be44434b6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:17.947283 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82395e-97f0-4a81-94b3-a14be44434b6 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:17.948911 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0d2f7-de7f-41b1-9482-46b8dfdebc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:17.949738 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0d2f7-de7f-41b1-9482-46b8dfdebc43 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:17.950011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0d2f7-de7f-41b1-9482-46b8dfdebc43 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:17.950294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0d2f7-de7f-41b1-9482-46b8dfdebc43 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-9f82395e-97f0-4a81-94b3-a14be44434b6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:17.963344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:17.964304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:17.965247 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:17.965247 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:17.965247 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:17.965247 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:17.965247 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:17.965247 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:17.965247 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:17.965247 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:17.965247 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:37:17.965247 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:17.967902 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 369/732] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:37:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:17.971386 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0d2f7-de7f-41b1-9482-46b8dfdebc43 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:17.973082 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0d2f7-de7f-41b1-9482-46b8dfdebc43 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:17.974048 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82395e-97f0-4a81-94b3-a14be44434b6 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-ebe0d2f7-de7f-41b1-9482-46b8dfdebc43 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:18.007338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:18.008342 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:18.009358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:18.009358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:18.009358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:18.009358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:18.009358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:18.009358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:18.009358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:18.009358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:18.009358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:37:18.009358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:18.018742 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 364/733] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:37:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:18.024082 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0d2f7-de7f-41b1-9482-46b8dfdebc43 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:18.094347 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82395e-97f0-4a81-94b3-a14be44434b6 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:18.116686 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:18.158190 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043e42e-5b93-4552-89cd-944a17a749fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:18.159050 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043e42e-5b93-4552-89cd-944a17a749fd None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:18.159330 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043e42e-5b93-4552-89cd-944a17a749fd None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:18.159600 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043e42e-5b93-4552-89cd-944a17a749fd None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:18.160444 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043e42e-5b93-4552-89cd-944a17a749fd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:18.160749 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043e42e-5b93-4552-89cd-944a17a749fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:18.163218 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043e42e-5b93-4552-89cd-944a17a749fd None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:18.205323 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 370/734] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 19:37:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:18.215696 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043e42e-5b93-4552-89cd-944a17a749fd None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:18.257030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0d2f7-de7f-41b1-9482-46b8dfdebc43 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:18.295870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092fd69-8771-4406-8f28-e15dfe659f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:18.296545 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092fd69-8771-4406-8f28-e15dfe659f12 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:18.296822 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092fd69-8771-4406-8f28-e15dfe659f12 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:18.297094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092fd69-8771-4406-8f28-e15dfe659f12 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:18.298033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092fd69-8771-4406-8f28-e15dfe659f12 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:18.298424 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092fd69-8771-4406-8f28-e15dfe659f12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:18.299974 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092fd69-8771-4406-8f28-e15dfe659f12 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:18.317197 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043e42e-5b93-4552-89cd-944a17a749fd None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:18.318794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 365/735] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 19:37:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:18.320305 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f8286-bd29-4837-817d-1daf2ef584ba None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:18.320594 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f8286-bd29-4837-817d-1daf2ef584ba None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:18.320870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f8286-bd29-4837-817d-1daf2ef584ba None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:18.322439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 366/736] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:18.324846 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f8286-bd29-4837-817d-1daf2ef584ba None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:18.354063 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:18.357435 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d97c7-623a-4376-bf3a-319ab6b63230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:18.358628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d97c7-623a-4376-bf3a-319ab6b63230 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:18.359023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d97c7-623a-4376-bf3a-319ab6b63230 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:18.359351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d97c7-623a-4376-bf3a-319ab6b63230 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:18.415843 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:18.443512 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d97c7-623a-4376-bf3a-319ab6b63230 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:18.445148 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d97c7-623a-4376-bf3a-319ab6b63230 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:18.449698 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f8286-bd29-4837-817d-1daf2ef584ba None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:18.495107 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da629cdd-38cb-4de5-9749-b59761be9e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:18.496260 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da629cdd-38cb-4de5-9749-b59761be9e09 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:18.496711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da629cdd-38cb-4de5-9749-b59761be9e09 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:18.497489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da629cdd-38cb-4de5-9749-b59761be9e09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:18.552153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 371/737] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:37:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:37:18.553631 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a60d01-0d8c-42e1-b70c-5db536ff788d None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:18.554009 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a60d01-0d8c-42e1-b70c-5db536ff788d None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:18.554724 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a60d01-0d8c-42e1-b70c-5db536ff788d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:18.572155 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-43a60d01-0d8c-42e1-b70c-5db536ff788d None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:18.573459 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-43a60d01-0d8c-42e1-b70c-5db536ff788d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:18.586540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da629cdd-38cb-4de5-9749-b59761be9e09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:18.588901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da629cdd-38cb-4de5-9749-b59761be9e09 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:18.642909 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:18.662407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 372/738] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:37:18.663546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a60d01-0d8c-42e1-b70c-5db536ff788d None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:18.671082 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:18.734289 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 367/739] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:37:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:37:18.734289 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da629cdd-38cb-4de5-9749-b59761be9e09 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:18.799317 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a60d01-0d8c-42e1-b70c-5db536ff788d None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:18.836307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae6f22-21b6-44c4-a485-ba6a68f4a020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:18.837372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae6f22-21b6-44c4-a485-ba6a68f4a020 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:18.837654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae6f22-21b6-44c4-a485-ba6a68f4a020 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:18.837928 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae6f22-21b6-44c4-a485-ba6a68f4a020 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:18.843085 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da629cdd-38cb-4de5-9749-b59761be9e09 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:18.849269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae6f22-21b6-44c4-a485-ba6a68f4a020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:18.850765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae6f22-21b6-44c4-a485-ba6a68f4a020 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:18.870852 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:18.872266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-4eae6f22-21b6-44c4-a485-ba6a68f4a020 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:18.872266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:18.872266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:18.872266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:18.872266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:18.872266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:18.872266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:18.872266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:18.872266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:18.872266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:18.872266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:18.872266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:18.872266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:18.872266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:18.872266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:18.873324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:18.874236 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:18.874236 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:18.874236 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:18.874236 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:18.876129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 373/740] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:18.877160 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae6f22-21b6-44c4-a485-ba6a68f4a020 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:18.884432 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234da0e3-6c1a-487c-afd9-17f30d603ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:18.885110 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234da0e3-6c1a-487c-afd9-17f30d603ce0 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:18.885388 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234da0e3-6c1a-487c-afd9-17f30d603ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:18.885670 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234da0e3-6c1a-487c-afd9-17f30d603ce0 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:18.886429 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234da0e3-6c1a-487c-afd9-17f30d603ce0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:18.887812 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234da0e3-6c1a-487c-afd9-17f30d603ce0 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:18.906776 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 368/741] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:37:18 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:37:18.908076 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234da0e3-6c1a-487c-afd9-17f30d603ce0 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:18.974011 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:19.013102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae6f22-21b6-44c4-a485-ba6a68f4a020 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:19.042290 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234da0e3-6c1a-487c-afd9-17f30d603ce0 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:19.060928 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb32ec-5584-4231-9574-c573cd0e6f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:19.061748 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb32ec-5584-4231-9574-c573cd0e6f90 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:19.062029 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb32ec-5584-4231-9574-c573cd0e6f90 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:19.062359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb32ec-5584-4231-9574-c573cd0e6f90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:19.079104 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb32ec-5584-4231-9574-c573cd0e6f90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:19.079907 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef209c0c-c259-46cf-9b95-354452a05e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:19.080714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef209c0c-c259-46cf-9b95-354452a05e18 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:19.081095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb32ec-5584-4231-9574-c573cd0e6f90 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:19.081197 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef209c0c-c259-46cf-9b95-354452a05e18 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:19.081450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef209c0c-c259-46cf-9b95-354452a05e18 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:19.082253 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef209c0c-c259-46cf-9b95-354452a05e18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:19.082645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef209c0c-c259-46cf-9b95-354452a05e18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:19.085005 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef209c0c-c259-46cf-9b95-354452a05e18 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:19.100737 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 369/742] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:19.102681 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c834ef-c5f4-41f0-9bfe-71684e936513 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:19.103049 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c834ef-c5f4-41f0-9bfe-71684e936513 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:19.103387 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c834ef-c5f4-41f0-9bfe-71684e936513 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:19.105041 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 370/743] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:19.106348 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c834ef-c5f4-41f0-9bfe-71684e936513 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:19.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-73fb32ec-5584-4231-9574-c573cd0e6f90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:19.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:19.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:19.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:19.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:19.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:19.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:19.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:19.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:19.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:19.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:19.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:19.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:19.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:19.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:19.109195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:19.110068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:19.110068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:19.110068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:19.110068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:19.111485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 374/744] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:19.118443 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf47b2-e7c1-4da1-98dd-5f56c779e4e7 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:19.118740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf47b2-e7c1-4da1-98dd-5f56c779e4e7 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:19.118999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf47b2-e7c1-4da1-98dd-5f56c779e4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:19.144695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-e5cf47b2-e7c1-4da1-98dd-5f56c779e4e7 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:19.145933 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-e5cf47b2-e7c1-4da1-98dd-5f56c779e4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:19.157710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:19.217148 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 375/745] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:37:19.219388 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf47b2-e7c1-4da1-98dd-5f56c779e4e7 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:19.221853 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c834ef-c5f4-41f0-9bfe-71684e936513 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:19.264389 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a8629-2fea-44c1-aae6-ade571883049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:19.265107 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a8629-2fea-44c1-aae6-ade571883049 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:19.265383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a8629-2fea-44c1-aae6-ade571883049 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:19.265653 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a8629-2fea-44c1-aae6-ade571883049 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:19.271857 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:19.273622 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a8629-2fea-44c1-aae6-ade571883049 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:19.275218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a8629-2fea-44c1-aae6-ade571883049 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:19.289874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-a04a8629-2fea-44c1-aae6-ade571883049 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:19.289874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:19.289874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:19.289874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:19.289874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:19.289874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:19.289874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:19.289874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:19.289874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:19.289874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:19.289874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:19.289874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:19.289874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:19.289874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:19.289874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:19.291065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:19.291953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:19.291953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:19.291953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:19.291953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:19.291953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:19.291953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:19.291953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:19.291953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:19.291953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 371/746] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:19.292540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d016d-2139-4fcd-b83b-4c5135e0ad38 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:19.292822 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d016d-2139-4fcd-b83b-4c5135e0ad38 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:19.293095 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d016d-2139-4fcd-b83b-4c5135e0ad38 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:19.298332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 372/747] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:19.298332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d016d-2139-4fcd-b83b-4c5135e0ad38 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:19.344419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf47b2-e7c1-4da1-98dd-5f56c779e4e7 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:19.381059 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31ec43-48d5-4f28-a06c-d5e0e8327ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:19.381728 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31ec43-48d5-4f28-a06c-d5e0e8327ecb None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:19.382001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31ec43-48d5-4f28-a06c-d5e0e8327ecb None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:19.382295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31ec43-48d5-4f28-a06c-d5e0e8327ecb None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:19.383095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31ec43-48d5-4f28-a06c-d5e0e8327ecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:19.383518 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31ec43-48d5-4f28-a06c-d5e0e8327ecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:19.384881 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31ec43-48d5-4f28-a06c-d5e0e8327ecb None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:19.396131 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 376/748] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:19.400599 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31ec43-48d5-4f28-a06c-d5e0e8327ecb None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:19.412952 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d016d-2139-4fcd-b83b-4c5135e0ad38 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:19.414783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:19.451114 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea4d88-e899-480f-9930-9243162564ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:19.451867 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea4d88-e899-480f-9930-9243162564ea None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:19.452155 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea4d88-e899-480f-9930-9243162564ea None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:19.452426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea4d88-e899-480f-9930-9243162564ea None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:19.453175 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea4d88-e899-480f-9930-9243162564ea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:19.454741 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea4d88-e899-480f-9930-9243162564ea None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:19.478489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 373/749] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:37:19 2022] POST /identity/v3/services => generated 240 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:37:19.479992 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb211f-95b9-42c4-b51f-b1fd4ad8359e None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:19.480274 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb211f-95b9-42c4-b51f-b1fd4ad8359e None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:19.480541 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb211f-95b9-42c4-b51f-b1fd4ad8359e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:19.505106 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-d0cb211f-95b9-42c4-b51f-b1fd4ad8359e None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:19.506527 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-d0cb211f-95b9-42c4-b51f-b1fd4ad8359e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:19.509817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:19.514472 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31ec43-48d5-4f28-a06c-d5e0e8327ecb None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:19.581758 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80d937-147d-4653-a514-6893ed5cf5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:19.582873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80d937-147d-4653-a514-6893ed5cf5f7 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:19.583350 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80d937-147d-4653-a514-6893ed5cf5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:19.583810 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80d937-147d-4653-a514-6893ed5cf5f7 None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:19.584854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80d937-147d-4653-a514-6893ed5cf5f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:19.585358 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80d937-147d-4653-a514-6893ed5cf5f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:19.587211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80d937-147d-4653-a514-6893ed5cf5f7 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:19.598291 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 374/750] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:19.598291 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb211f-95b9-42c4-b51f-b1fd4ad8359e None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:19.614028 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 377/751] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:19.622166 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80d937-147d-4653-a514-6893ed5cf5f7 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:19.723596 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:19.738210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb211f-95b9-42c4-b51f-b1fd4ad8359e None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:19.749522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80d937-147d-4653-a514-6893ed5cf5f7 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:19.775674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e382d-c28d-4c97-89a1-22748e1230bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:19.776473 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e382d-c28d-4c97-89a1-22748e1230bd None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:19.776813 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e382d-c28d-4c97-89a1-22748e1230bd None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:19.777145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e382d-c28d-4c97-89a1-22748e1230bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:19.784828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e382d-c28d-4c97-89a1-22748e1230bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:19.786544 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e382d-c28d-4c97-89a1-22748e1230bd None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:19.794050 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbcb3c-7d53-46b4-9937-5fd9c7cccfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:19.794740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbcb3c-7d53-46b4-9937-5fd9c7cccfd2 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:19.795022 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbcb3c-7d53-46b4-9937-5fd9c7cccfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:19.795293 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbcb3c-7d53-46b4-9937-5fd9c7cccfd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:19.796122 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbcb3c-7d53-46b4-9937-5fd9c7cccfd2 None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=4d1c8759f0564f28b388106dbf615cb5) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:19.796431 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbcb3c-7d53-46b4-9937-5fd9c7cccfd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:19.797868 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbcb3c-7d53-46b4-9937-5fd9c7cccfd2 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:19.801099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-ff7e382d-c28d-4c97-89a1-22748e1230bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:19.801099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:19.801099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:19.801099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:19.801099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:19.801099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:19.801099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:19.801099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:19.801099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:19.801099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:19.801099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:19.801099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:19.801099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:19.801099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:19.801099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:19.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:19.802832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:19.802832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:19.802832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:19.802832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:19.802832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:19.802832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:19.802832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:19.802832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:19.802832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 375/752] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:19.803733 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e382d-c28d-4c97-89a1-22748e1230bd None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:19.806933 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 378/753] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:19 2022] GET /identity/v3/role_assignments?user.id=4d1c8759f0564f28b388106dbf615cb5&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:19.809595 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbcb3c-7d53-46b4-9937-5fd9c7cccfd2 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:19.842653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:19.932379 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e382d-c28d-4c97-89a1-22748e1230bd None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:20.003855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbcb3c-7d53-46b4-9937-5fd9c7cccfd2 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:20.020982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ab8c0-ebfa-4416-b1f6-06a0f31f4157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:20.021799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ab8c0-ebfa-4416-b1f6-06a0f31f4157 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:20.022077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ab8c0-ebfa-4416-b1f6-06a0f31f4157 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:20.022374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ab8c0-ebfa-4416-b1f6-06a0f31f4157 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:20.044772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fbc71-b7fb-4411-b242-4dbc175e075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:20.045785 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fbc71-b7fb-4411-b242-4dbc175e075e None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:20.046086 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fbc71-b7fb-4411-b242-4dbc175e075e None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:20.046354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fbc71-b7fb-4411-b242-4dbc175e075e None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:20.047168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fbc71-b7fb-4411-b242-4dbc175e075e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:20.047478 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fbc71-b7fb-4411-b242-4dbc175e075e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:20.049049 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fbc71-b7fb-4411-b242-4dbc175e075e None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:20.056588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ab8c0-ebfa-4416-b1f6-06a0f31f4157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-ee6ab8c0-ebfa-4416-b1f6-06a0f31f4157 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:20.070656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:20.071641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:20.072467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:20.072467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:20.072467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:20.072467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:20.072467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:20.072467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:20.072467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:20.072467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:20.072467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:20.072467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:20.072467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:37:20.072467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:20.072467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 376/754] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:37:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:20.079286 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 379/755] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:20.079573 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ab8c0-ebfa-4416-b1f6-06a0f31f4157 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:20.086011 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fbc71-b7fb-4411-b242-4dbc175e075e None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:20.117915 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:20.269419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:20.300490 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ab8c0-ebfa-4416-b1f6-06a0f31f4157 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:20.320117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fbc71-b7fb-4411-b242-4dbc175e075e None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:20.371737 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf623c0d-b991-4b54-a450-7fba024cca21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:20.373463 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf623c0d-b991-4b54-a450-7fba024cca21 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:20.374575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf623c0d-b991-4b54-a450-7fba024cca21 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:20.375054 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf623c0d-b991-4b54-a450-7fba024cca21 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:20.376165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf623c0d-b991-4b54-a450-7fba024cca21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:20.378167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf623c0d-b991-4b54-a450-7fba024cca21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:20.379806 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf623c0d-b991-4b54-a450-7fba024cca21 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:20.389547 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b541d-00f9-40b0-8494-00b502918380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:20.390533 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b541d-00f9-40b0-8494-00b502918380 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:20.392636 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b541d-00f9-40b0-8494-00b502918380 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:20.394313 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b541d-00f9-40b0-8494-00b502918380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:20.395364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b541d-00f9-40b0-8494-00b502918380 None admin] RBAC: Adding query filter params (role.id=cefdc5c73ec347438e5b4687f191382c, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=9e05338c14d642f3b31095f85e1e6ca4) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:20.398070 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b541d-00f9-40b0-8494-00b502918380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:20.399754 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b541d-00f9-40b0-8494-00b502918380 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:20.400238 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 377/756] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:37:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:20.403102 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf623c0d-b991-4b54-a450-7fba024cca21 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:20.416579 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 380/757] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:20 2022] GET /identity/v3/role_assignments?user.id=9e05338c14d642f3b31095f85e1e6ca4&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=cefdc5c73ec347438e5b4687f191382c => generated 259 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:20.550966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:20.588064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf623c0d-b991-4b54-a450-7fba024cca21 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:20.652211 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f23a5-05fd-4d92-ada9-e25ed5304e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:20.652211 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f23a5-05fd-4d92-ada9-e25ed5304e80 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:20.652550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f23a5-05fd-4d92-ada9-e25ed5304e80 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:20.652550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f23a5-05fd-4d92-ada9-e25ed5304e80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:20.671519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f23a5-05fd-4d92-ada9-e25ed5304e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:20.673034 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f23a5-05fd-4d92-ada9-e25ed5304e80 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:20.693501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-2e2f23a5-05fd-4d92-ada9-e25ed5304e80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:20.693501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:20.693501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:20.693501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:20.693501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:20.693501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:20.693501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:20.693501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:20.693501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:20.693501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:20.693501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:20.693501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:20.693501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:20.693501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:20.693501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:20.694557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:20.695493 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:20.695493 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:20.695493 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:20.695493 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:20.697692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 378/758] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:20.704256 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b541d-00f9-40b0-8494-00b502918380 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:20.909795 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b541d-00f9-40b0-8494-00b502918380 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:20.950333 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd707a29-07b2-4f60-b11e-f0aa9166003d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:20.952503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd707a29-07b2-4f60-b11e-f0aa9166003d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:20.952946 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd707a29-07b2-4f60-b11e-f0aa9166003d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:20.953356 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd707a29-07b2-4f60-b11e-f0aa9166003d None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:20.954786 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd707a29-07b2-4f60-b11e-f0aa9166003d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:20.955588 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd707a29-07b2-4f60-b11e-f0aa9166003d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:20.957269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd707a29-07b2-4f60-b11e-f0aa9166003d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:20.974222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 381/759] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:20.979647 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f23a5-05fd-4d92-ada9-e25ed5304e80 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:21.157117 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f23a5-05fd-4d92-ada9-e25ed5304e80 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:21.214724 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:21.230626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b57c7-a79a-4351-9144-7ed11cb3e76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:21.231437 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b57c7-a79a-4351-9144-7ed11cb3e76b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:21.231723 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b57c7-a79a-4351-9144-7ed11cb3e76b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:21.232002 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b57c7-a79a-4351-9144-7ed11cb3e76b None admin] PATH_INFO: `/v3/users/nova` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:21.240733 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b57c7-a79a-4351-9144-7ed11cb3e76b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:21.242289 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b57c7-a79a-4351-9144-7ed11cb3e76b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-b00b57c7-a79a-4351-9144-7ed11cb3e76b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:21.278064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:21.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:21.280496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:21.280496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:21.280496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:21.280496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:21.280496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:21.280496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:21.280496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:21.280496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:21.280496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:37:21.280496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:21.280496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 379/760] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:37:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:21.285353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd707a29-07b2-4f60-b11e-f0aa9166003d None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:21.291422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:21.427404 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd707a29-07b2-4f60-b11e-f0aa9166003d None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:21.461555 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13037ae1-32d8-4a83-9542-7a4c7e96db04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:21.462622 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13037ae1-32d8-4a83-9542-7a4c7e96db04 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:21.463127 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13037ae1-32d8-4a83-9542-7a4c7e96db04 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:21.463606 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13037ae1-32d8-4a83-9542-7a4c7e96db04 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:21.464639 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13037ae1-32d8-4a83-9542-7a4c7e96db04 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:21.465201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13037ae1-32d8-4a83-9542-7a4c7e96db04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:21.466884 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13037ae1-32d8-4a83-9542-7a4c7e96db04 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:21.494378 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 382/761] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:37:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:21.499489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b57c7-a79a-4351-9144-7ed11cb3e76b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:21.540702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0e918-4f46-4a0f-a415-d41bd02ebefd None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:21.541318 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0e918-4f46-4a0f-a415-d41bd02ebefd None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:21.541819 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0e918-4f46-4a0f-a415-d41bd02ebefd None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:21.543785 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 383/762] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:21.548426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994fd408-5698-4d1f-bc7b-f0e00be0263c None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:21.548941 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994fd408-5698-4d1f-bc7b-f0e00be0263c None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:21.549411 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994fd408-5698-4d1f-bc7b-f0e00be0263c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:21.567450 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-994fd408-5698-4d1f-bc7b-f0e00be0263c None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:21.569061 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-994fd408-5698-4d1f-bc7b-f0e00be0263c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:21.657099 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:21.678157 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 384/763] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:21.681958 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b57c7-a79a-4351-9144-7ed11cb3e76b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:21.715958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994fd408-5698-4d1f-bc7b-f0e00be0263c None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:21.731391 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8752d6e-ed8b-47f8-9445-adce4ec652be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:21.732198 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8752d6e-ed8b-47f8-9445-adce4ec652be None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:21.732477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8752d6e-ed8b-47f8-9445-adce4ec652be None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:21.732749 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8752d6e-ed8b-47f8-9445-adce4ec652be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:21.757725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8752d6e-ed8b-47f8-9445-adce4ec652be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:21.759243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8752d6e-ed8b-47f8-9445-adce4ec652be None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:21.793654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-e8752d6e-ed8b-47f8-9445-adce4ec652be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:21.793654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:21.793654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:21.793654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:21.793654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:21.793654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:21.793654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:21.793654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:21.793654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:21.793654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:21.793654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:21.793654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:21.793654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:21.793654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:21.793654 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:21.794645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:21.795521 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:21.795521 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:21.795521 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:21.795521 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:21.795521 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 380/764] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:21.801112 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8752d6e-ed8b-47f8-9445-adce4ec652be None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:21.873481 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:21.895926 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994fd408-5698-4d1f-bc7b-f0e00be0263c None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:21.936484 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b73f06-bfc9-4986-ba87-284c259574ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:21.937315 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b73f06-bfc9-4986-ba87-284c259574ee None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:21.937599 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b73f06-bfc9-4986-ba87-284c259574ee None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:21.937870 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b73f06-bfc9-4986-ba87-284c259574ee None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:21.938736 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b73f06-bfc9-4986-ba87-284c259574ee None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:21.940562 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b73f06-bfc9-4986-ba87-284c259574ee None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-a5b73f06-bfc9-4986-ba87-284c259574ee None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:21.954546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:21.957176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:21.959479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 385/765] 10.222.0.84 () {56 vars in 1129 bytes} [Thu May 19 19:37:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:21.966347 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b73f06-bfc9-4986-ba87-284c259574ee None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:22.018950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8752d6e-ed8b-47f8-9445-adce4ec652be None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:22.062420 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:22.106981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b709b8f-b7da-4565-9bfc-477f3c8596c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:22.107665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b709b8f-b7da-4565-9bfc-477f3c8596c6 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:22.107942 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b709b8f-b7da-4565-9bfc-477f3c8596c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:22.108206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b709b8f-b7da-4565-9bfc-477f3c8596c6 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:22.108936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b709b8f-b7da-4565-9bfc-477f3c8596c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:22.109233 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b709b8f-b7da-4565-9bfc-477f3c8596c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:22.109466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b73f06-bfc9-4986-ba87-284c259574ee None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:22.110753 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b709b8f-b7da-4565-9bfc-477f3c8596c6 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:22.123190 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 381/766] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:22.129204 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b709b8f-b7da-4565-9bfc-477f3c8596c6 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:22.180066 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c8cb7-d288-4597-abeb-b589f5217937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:22.181164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c8cb7-d288-4597-abeb-b589f5217937 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:22.181658 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c8cb7-d288-4597-abeb-b589f5217937 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:22.182121 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c8cb7-d288-4597-abeb-b589f5217937 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:22.183056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c8cb7-d288-4597-abeb-b589f5217937 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:22.183613 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c8cb7-d288-4597-abeb-b589f5217937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:22.185196 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c8cb7-d288-4597-abeb-b589f5217937 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:22.194643 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 386/767] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 19:37:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:22.199999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c8cb7-d288-4597-abeb-b589f5217937 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:22.303807 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:22.308914 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b709b8f-b7da-4565-9bfc-477f3c8596c6 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:22.363273 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc970d-822d-4c0e-80b6-f87f14b34dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:22.365011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc970d-822d-4c0e-80b6-f87f14b34dbe None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:22.365487 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc970d-822d-4c0e-80b6-f87f14b34dbe None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:22.365985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc970d-822d-4c0e-80b6-f87f14b34dbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:22.367032 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c8cb7-d288-4597-abeb-b589f5217937 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:22.374868 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc970d-822d-4c0e-80b6-f87f14b34dbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:22.377277 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc970d-822d-4c0e-80b6-f87f14b34dbe None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:22.385360 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:22.395173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-67bc970d-822d-4c0e-80b6-f87f14b34dbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:22.395173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:22.395173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:22.395173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:22.395173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:22.395173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:22.395173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:22.395173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:22.395173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:22.395173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:22.395173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:22.395173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:22.395173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:22.395173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:22.395173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:22.396056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:22.396921 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:22.396921 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:22.396921 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:22.396921 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:22.396921 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:22.396921 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:22.396921 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:22.396921 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:22.399758 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 382/768] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:22.405306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7dfc6-5177-4e52-a048-a0017a2ba799 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:22.405306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7dfc6-5177-4e52-a048-a0017a2ba799 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:22.405306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7dfc6-5177-4e52-a048-a0017a2ba799 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:22.406347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 383/769] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:22.407255 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7dfc6-5177-4e52-a048-a0017a2ba799 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:22.415164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4c44d-d0c0-48d1-90c7-678d9f6aae92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:22.415842 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4c44d-d0c0-48d1-90c7-678d9f6aae92 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:22.416182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4c44d-d0c0-48d1-90c7-678d9f6aae92 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:22.416455 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4c44d-d0c0-48d1-90c7-678d9f6aae92 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:22.417222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4c44d-d0c0-48d1-90c7-678d9f6aae92 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:22.417530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4c44d-d0c0-48d1-90c7-678d9f6aae92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:22.419072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4c44d-d0c0-48d1-90c7-678d9f6aae92 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:22.430668 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 387/770] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 19:37:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:22.432337 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e5528-9575-48c9-8c00-b67912153d49 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:22.432627 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e5528-9575-48c9-8c00-b67912153d49 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:22.432908 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e5528-9575-48c9-8c00-b67912153d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:22.454438 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-6b7e5528-9575-48c9-8c00-b67912153d49 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:22.456085 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-6b7e5528-9575-48c9-8c00-b67912153d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:22.540762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7dfc6-5177-4e52-a048-a0017a2ba799 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:22.542508 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 388/771] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:22.543667 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e5528-9575-48c9-8c00-b67912153d49 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:22.585540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f20390-cf53-4e1d-999e-48019784f6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:22.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f20390-cf53-4e1d-999e-48019784f6a3 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:22.587817 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f20390-cf53-4e1d-999e-48019784f6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:22.588355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f20390-cf53-4e1d-999e-48019784f6a3 None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:22.589331 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f20390-cf53-4e1d-999e-48019784f6a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:22.589806 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f20390-cf53-4e1d-999e-48019784f6a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:22.592582 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f20390-cf53-4e1d-999e-48019784f6a3 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:22.614231 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 384/772] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:22.615455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f20390-cf53-4e1d-999e-48019784f6a3 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:22.644648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:22.676286 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e5528-9575-48c9-8c00-b67912153d49 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:22.689778 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:22.711258 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d08971-b261-4e44-96de-ed1c994fe030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:22.711916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d08971-b261-4e44-96de-ed1c994fe030 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:22.712190 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d08971-b261-4e44-96de-ed1c994fe030 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:22.712456 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d08971-b261-4e44-96de-ed1c994fe030 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:22.713239 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d08971-b261-4e44-96de-ed1c994fe030 None admin] RBAC: Adding query filter params (interface=public, service_id=6d418feee4c14daeb497934e4ae01882, region_id=RegionOne) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:22.713540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d08971-b261-4e44-96de-ed1c994fe030 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:22.714937 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d08971-b261-4e44-96de-ed1c994fe030 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:22.727108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 389/773] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:37:22 2022] GET /identity/v3/endpoints?service_id=6d418feee4c14daeb497934e4ae01882&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:22.727108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d08971-b261-4e44-96de-ed1c994fe030 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:22.748882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f20390-cf53-4e1d-999e-48019784f6a3 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:22.784783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032b4a1-2474-4848-acd8-e3618c5ee670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:22.785689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032b4a1-2474-4848-acd8-e3618c5ee670 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:22.786099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032b4a1-2474-4848-acd8-e3618c5ee670 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:22.786385 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032b4a1-2474-4848-acd8-e3618c5ee670 None admin] PATH_INFO: `/v3/services/network` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:22.787124 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032b4a1-2474-4848-acd8-e3618c5ee670 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:22.788575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032b4a1-2474-4848-acd8-e3618c5ee670 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-b032b4a1-2474-4848-acd8-e3618c5ee670 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:22.800966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:22.801956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:22.802825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:22.802825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:22.802825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:22.802825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:22.802825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:22.802825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:22.802825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:22.802825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:22.802825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:22.802825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:22.802825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:22.802825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:22.802825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:37:22.802825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:22.802825 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 385/774] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:22 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:22.803592 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cb156-ba46-4af0-837b-fd46b6b5234a None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:22.803659 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cb156-ba46-4af0-837b-fd46b6b5234a None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:22.804718 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cb156-ba46-4af0-837b-fd46b6b5234a None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:22.809604 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 386/775] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:22.809604 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cb156-ba46-4af0-837b-fd46b6b5234a None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:22.835456 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d08971-b261-4e44-96de-ed1c994fe030 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:22.872757 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2926ac-af68-4248-bc09-c39e2a1664bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:22.873619 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2926ac-af68-4248-bc09-c39e2a1664bb None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:22.874098 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2926ac-af68-4248-bc09-c39e2a1664bb None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:22.874208 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2926ac-af68-4248-bc09-c39e2a1664bb None admin] PATH_INFO: `/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/abde7cc02fae4d9bb561a6642c5252a6/roles/a33caf2301fb44db90304eb3d9afc3a5` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:22.876181 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2926ac-af68-4248-bc09-c39e2a1664bb None admin] RBAC: Authorizing `identity:create_grant(project_id=ecf87429758641eeb462b3e409cadc9e, user_id=abde7cc02fae4d9bb561a6642c5252a6, role_id=a33caf2301fb44db90304eb3d9afc3a5)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:22.877805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2926ac-af68-4248-bc09-c39e2a1664bb None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:22.893329 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 390/776] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:37:22 2022] PUT /identity/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/abde7cc02fae4d9bb561a6642c5252a6/roles/a33caf2301fb44db90304eb3d9afc3a5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:37:22.894162 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2926ac-af68-4248-bc09-c39e2a1664bb None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:22.906106 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:22.929517 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cb156-ba46-4af0-837b-fd46b6b5234a None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:22.946541 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:22.976192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75ac98-f197-4341-8e4c-fc48271a6a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:22.977255 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75ac98-f197-4341-8e4c-fc48271a6a46 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:22.977623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75ac98-f197-4341-8e4c-fc48271a6a46 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:22.977986 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75ac98-f197-4341-8e4c-fc48271a6a46 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:22.978858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75ac98-f197-4341-8e4c-fc48271a6a46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:22.980720 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75ac98-f197-4341-8e4c-fc48271a6a46 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:22.987047 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 387/777] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:37:22 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:22.988515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef102a1-c858-4cf0-93e9-f5e705f44fc9 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:22.988924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef102a1-c858-4cf0-93e9-f5e705f44fc9 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:22.989330 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef102a1-c858-4cf0-93e9-f5e705f44fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:23.008772 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-9ef102a1-c858-4cf0-93e9-f5e705f44fc9 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:23.010179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-9ef102a1-c858-4cf0-93e9-f5e705f44fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:23.041692 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2926ac-af68-4248-bc09-c39e2a1664bb None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:23.088219 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea4ef1-26e3-444b-9d53-700bb8e83545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:23.089318 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea4ef1-26e3-444b-9d53-700bb8e83545 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:23.089777 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea4ef1-26e3-444b-9d53-700bb8e83545 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:23.090208 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea4ef1-26e3-444b-9d53-700bb8e83545 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:23.091131 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea4ef1-26e3-444b-9d53-700bb8e83545 None admin] RBAC: Adding query filter params (name=network) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:23.091597 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea4ef1-26e3-444b-9d53-700bb8e83545 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:23.093419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea4ef1-26e3-444b-9d53-700bb8e83545 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:23.100972 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 391/778] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:37:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:23.107665 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea4ef1-26e3-444b-9d53-700bb8e83545 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:23.159183 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 388/779] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:23.164546 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d52acf-dea0-4aef-aade-d8fdc0636379 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:23.164832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d52acf-dea0-4aef-aade-d8fdc0636379 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:23.165102 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d52acf-dea0-4aef-aade-d8fdc0636379 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:23.166797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 389/780] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:23.183771 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d52acf-dea0-4aef-aade-d8fdc0636379 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:23.257537 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:23.261158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea4ef1-26e3-444b-9d53-700bb8e83545 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:23.287533 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:23.302004 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813fd7f-18fe-4cb9-a07f-c5bdc7e0a2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:23.302796 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813fd7f-18fe-4cb9-a07f-c5bdc7e0a2bb None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:23.303164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813fd7f-18fe-4cb9-a07f-c5bdc7e0a2bb None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:23.303440 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813fd7f-18fe-4cb9-a07f-c5bdc7e0a2bb None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:23.304149 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813fd7f-18fe-4cb9-a07f-c5bdc7e0a2bb None admin] RBAC: Adding query filter params (type=network) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:23.304448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813fd7f-18fe-4cb9-a07f-c5bdc7e0a2bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:23.306105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813fd7f-18fe-4cb9-a07f-c5bdc7e0a2bb None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:23.311768 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 392/781] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:37:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:23.315076 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca5e98-f91f-48f0-8d16-65c73265ded1 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:23.315369 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca5e98-f91f-48f0-8d16-65c73265ded1 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:23.315698 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca5e98-f91f-48f0-8d16-65c73265ded1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:23.318178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d52acf-dea0-4aef-aade-d8fdc0636379 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:23.341262 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-e7ca5e98-f91f-48f0-8d16-65c73265ded1 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:23.342904 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-e7ca5e98-f91f-48f0-8d16-65c73265ded1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:23.357035 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3574a25-1a74-4d7d-be37-07a7e6d7929c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:23.357845 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3574a25-1a74-4d7d-be37-07a7e6d7929c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:23.358128 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3574a25-1a74-4d7d-be37-07a7e6d7929c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:23.358449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3574a25-1a74-4d7d-be37-07a7e6d7929c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:23.371000 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3574a25-1a74-4d7d-be37-07a7e6d7929c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-d3574a25-1a74-4d7d-be37-07a7e6d7929c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:23.379696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:23.380634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:23.381445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:23.381445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:23.381445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:23.381445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:23.381445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:23.381445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:23.381445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:23.381445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:23.381445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:23.381445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:23.381445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:37:23.381445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:23.381445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 390/782] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:37:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:23.386479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3574a25-1a74-4d7d-be37-07a7e6d7929c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:23.430302 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 393/783] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:23.432938 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca5e98-f91f-48f0-8d16-65c73265ded1 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:23.501815 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3574a25-1a74-4d7d-be37-07a7e6d7929c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:23.545639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdeaa0b-6719-40dd-a19a-af8f0c95a8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:23.546813 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdeaa0b-6719-40dd-a19a-af8f0c95a8ca None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:23.547597 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdeaa0b-6719-40dd-a19a-af8f0c95a8ca None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:23.548212 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdeaa0b-6719-40dd-a19a-af8f0c95a8ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:23.549256 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdeaa0b-6719-40dd-a19a-af8f0c95a8ca None admin] RBAC: Adding query filter params (interface=public, service_id=01ff1580583b4754822252f075f54b30, region_id=RegionOne) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:23.549740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdeaa0b-6719-40dd-a19a-af8f0c95a8ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:23.551821 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdeaa0b-6719-40dd-a19a-af8f0c95a8ca None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:23.556755 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:23.562685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:23.568867 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca5e98-f91f-48f0-8d16-65c73265ded1 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:23.569512 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 391/784] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:37:23 2022] GET /identity/v3/endpoints?service_id=01ff1580583b4754822252f075f54b30&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:23.570997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdeaa0b-6719-40dd-a19a-af8f0c95a8ca None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:23.607951 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81446e-b0fb-40db-a752-6a75d80535f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:23.608689 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81446e-b0fb-40db-a752-6a75d80535f0 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:23.608976 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81446e-b0fb-40db-a752-6a75d80535f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:23.609326 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81446e-b0fb-40db-a752-6a75d80535f0 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:23.610051 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81446e-b0fb-40db-a752-6a75d80535f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:23.610525 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81446e-b0fb-40db-a752-6a75d80535f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:23.612305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81446e-b0fb-40db-a752-6a75d80535f0 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:23.628177 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 394/785] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:37:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:23.629249 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81446e-b0fb-40db-a752-6a75d80535f0 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:23.700974 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdeaa0b-6719-40dd-a19a-af8f0c95a8ca None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:23.745782 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818a5df-9ed2-48c8-83ca-27c0c5468fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:23.746861 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818a5df-9ed2-48c8-83ca-27c0c5468fe6 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:23.747302 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818a5df-9ed2-48c8-83ca-27c0c5468fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:23.747688 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81446e-b0fb-40db-a752-6a75d80535f0 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:23.747853 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818a5df-9ed2-48c8-83ca-27c0c5468fe6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:23.761013 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818a5df-9ed2-48c8-83ca-27c0c5468fe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-9818a5df-9ed2-48c8-83ca-27c0c5468fe6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:23.771409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:23.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:23.773407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:23.773407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:23.773407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:23.773407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:23.773407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:23.773407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:23.773407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:23.773407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:23.773407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:23.773407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:23.773407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:23.773407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:23.776569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 392/786] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:37:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:23.777520 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818a5df-9ed2-48c8-83ca-27c0c5468fe6 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:23.787779 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc354191-c5db-4cf7-8be3-c742930f4c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:23.788611 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc354191-c5db-4cf7-8be3-c742930f4c7a None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:23.788990 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc354191-c5db-4cf7-8be3-c742930f4c7a None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:23.789360 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc354191-c5db-4cf7-8be3-c742930f4c7a None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:23.790558 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc354191-c5db-4cf7-8be3-c742930f4c7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:23.791857 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc354191-c5db-4cf7-8be3-c742930f4c7a None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:23.802299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 395/787] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:37:23 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:23.802299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc354191-c5db-4cf7-8be3-c742930f4c7a None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:23.839896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:23.897458 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:23.971941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818a5df-9ed2-48c8-83ca-27c0c5468fe6 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:24.013583 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc354191-c5db-4cf7-8be3-c742930f4c7a None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:24.039945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089ad17-3baa-49fd-879b-c69a68d83608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:24.040756 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089ad17-3baa-49fd-879b-c69a68d83608 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:24.041032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089ad17-3baa-49fd-879b-c69a68d83608 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:24.041302 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089ad17-3baa-49fd-879b-c69a68d83608 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:24.061982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089ad17-3baa-49fd-879b-c69a68d83608 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:24.064542 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089ad17-3baa-49fd-879b-c69a68d83608 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:24.080230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4789c-7a09-431b-a6a0-44ff620961a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:24.081231 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4789c-7a09-431b-a6a0-44ff620961a1 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:24.081729 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4789c-7a09-431b-a6a0-44ff620961a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:24.082191 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4789c-7a09-431b-a6a0-44ff620961a1 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:24.083171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4789c-7a09-431b-a6a0-44ff620961a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:24.083689 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4789c-7a09-431b-a6a0-44ff620961a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:24.086007 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4789c-7a09-431b-a6a0-44ff620961a1 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:24.087043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-2089ad17-3baa-49fd-879b-c69a68d83608 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:24.087043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:24.087043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:24.087043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:24.087043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:24.087043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:24.087043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:24.087043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:24.087043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:24.087043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:24.087043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:24.087043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:24.087043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:24.087043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:24.087043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:24.087888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:24.088785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:24.088785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:24.088785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:24.088785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:24.088785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 393/788] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:24.095162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089ad17-3baa-49fd-879b-c69a68d83608 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:24.104898 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 396/789] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:37:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:24.110552 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4789c-7a09-431b-a6a0-44ff620961a1 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:24.219739 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:24.228237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089ad17-3baa-49fd-879b-c69a68d83608 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:24.256646 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:24.259582 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4789c-7a09-431b-a6a0-44ff620961a1 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:24.302202 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a4fea-aac5-4af8-95a3-c31b908452d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:24.302941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a4fea-aac5-4af8-95a3-c31b908452d1 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:24.303220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a4fea-aac5-4af8-95a3-c31b908452d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:24.303488 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a4fea-aac5-4af8-95a3-c31b908452d1 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:24.304221 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a4fea-aac5-4af8-95a3-c31b908452d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:24.304528 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a4fea-aac5-4af8-95a3-c31b908452d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:24.305980 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a4fea-aac5-4af8-95a3-c31b908452d1 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:24.319591 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 394/790] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:24.325155 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a4fea-aac5-4af8-95a3-c31b908452d1 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:24.365254 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361bf26-2430-46c0-a142-855340391060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:24.366075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361bf26-2430-46c0-a142-855340391060 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:24.366383 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361bf26-2430-46c0-a142-855340391060 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:24.366662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361bf26-2430-46c0-a142-855340391060 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:24.391685 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361bf26-2430-46c0-a142-855340391060 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:24.393296 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361bf26-2430-46c0-a142-855340391060 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:24.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-2361bf26-2430-46c0-a142-855340391060 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:24.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:24.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:24.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:24.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:24.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:24.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:24.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:24.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:24.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:24.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:24.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:24.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:24.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:24.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:24.413119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:24.413967 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:24.413967 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:24.413967 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:24.413967 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:24.413967 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 397/791] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:24.421048 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361bf26-2430-46c0-a142-855340391060 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:24.466068 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a4fea-aac5-4af8-95a3-c31b908452d1 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:24.504392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f9ccf-b362-42f3-8dad-9b6b67667b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:24.506115 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f9ccf-b362-42f3-8dad-9b6b67667b9e None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:24.506873 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f9ccf-b362-42f3-8dad-9b6b67667b9e None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:24.509228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f9ccf-b362-42f3-8dad-9b6b67667b9e None admin] PATH_INFO: `/v3/users/swift` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:24.521771 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f9ccf-b362-42f3-8dad-9b6b67667b9e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:24.523584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f9ccf-b362-42f3-8dad-9b6b67667b9e None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-763f9ccf-b362-42f3-8dad-9b6b67667b9e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:24.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:24.546665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:24.547583 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:24.547583 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:24.547583 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:24.547583 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:24.547583 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:24.547583 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:24.547583 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:24.547583 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:24.547583 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:37:24.547583 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:24.550195 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 395/792] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:37:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:24.559233 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f9ccf-b362-42f3-8dad-9b6b67667b9e None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:24.610525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:24.644986 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361bf26-2430-46c0-a142-855340391060 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:24.705776 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f9ccf-b362-42f3-8dad-9b6b67667b9e None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:24.717805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72fd15-49cc-4f56-a8d0-f355c83c4146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:24.718505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72fd15-49cc-4f56-a8d0-f355c83c4146 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:24.718785 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72fd15-49cc-4f56-a8d0-f355c83c4146 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:24.719054 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72fd15-49cc-4f56-a8d0-f355c83c4146 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:24.719802 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72fd15-49cc-4f56-a8d0-f355c83c4146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:24.720367 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72fd15-49cc-4f56-a8d0-f355c83c4146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:24.723690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72fd15-49cc-4f56-a8d0-f355c83c4146 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:24.729913 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:24.749780 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 398/793] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:24.763623 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72fd15-49cc-4f56-a8d0-f355c83c4146 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:24.763905 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785e7ac-d112-49b1-a343-68134b9afbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:24.764693 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785e7ac-d112-49b1-a343-68134b9afbbf None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:24.764972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785e7ac-d112-49b1-a343-68134b9afbbf None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:24.765244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785e7ac-d112-49b1-a343-68134b9afbbf None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:24.766091 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785e7ac-d112-49b1-a343-68134b9afbbf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:24.766423 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785e7ac-d112-49b1-a343-68134b9afbbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:24.768270 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785e7ac-d112-49b1-a343-68134b9afbbf None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:24.787598 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 396/794] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 19:37:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:24.792945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785e7ac-d112-49b1-a343-68134b9afbbf None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:24.931770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:24.942189 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785e7ac-d112-49b1-a343-68134b9afbbf None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:24.988550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f2c3f-5650-4230-a786-fb32aa1a410d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:24.989489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f2c3f-5650-4230-a786-fb32aa1a410d None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:24.989768 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f2c3f-5650-4230-a786-fb32aa1a410d None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:24.990037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f2c3f-5650-4230-a786-fb32aa1a410d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:24.992258 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72fd15-49cc-4f56-a8d0-f355c83c4146 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:25.001849 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f2c3f-5650-4230-a786-fb32aa1a410d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:25.003339 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f2c3f-5650-4230-a786-fb32aa1a410d None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:25.022858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-d10f2c3f-5650-4230-a786-fb32aa1a410d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:25.022858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:25.022858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:25.022858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:25.022858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:25.022858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:25.022858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:25.022858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:25.022858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:25.022858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:25.022858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:25.022858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:25.022858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:25.022858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:25.022858 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:25.023778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:25.024651 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:25.024651 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:25.024651 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:25.024651 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:25.024651 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 397/795] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:25.035448 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f2c3f-5650-4230-a786-fb32aa1a410d None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:25.077376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037b27b-3e39-49dc-a2f4-66c78ba185a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:25.080041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037b27b-3e39-49dc-a2f4-66c78ba185a1 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:25.080511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037b27b-3e39-49dc-a2f4-66c78ba185a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:25.080966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037b27b-3e39-49dc-a2f4-66c78ba185a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:25.092991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037b27b-3e39-49dc-a2f4-66c78ba185a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:25.094535 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037b27b-3e39-49dc-a2f4-66c78ba185a1 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-a037b27b-3e39-49dc-a2f4-66c78ba185a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:25.110702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:25.111859 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:25.113559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:25.113559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:25.113559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:25.113559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:25.113559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:25.113559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:25.113559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:25.113559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:25.113559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:37:25.113559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:25.113559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 399/796] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:37:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:25.119347 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037b27b-3e39-49dc-a2f4-66c78ba185a1 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:25.197524 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f2c3f-5650-4230-a786-fb32aa1a410d None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:25.245957 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:25.252047 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3327bcd-2bb4-432f-a43c-42f5e9565118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:25.252687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3327bcd-2bb4-432f-a43c-42f5e9565118 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:25.253030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3327bcd-2bb4-432f-a43c-42f5e9565118 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:25.253234 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3327bcd-2bb4-432f-a43c-42f5e9565118 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:25.253969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3327bcd-2bb4-432f-a43c-42f5e9565118 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:25.254293 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3327bcd-2bb4-432f-a43c-42f5e9565118 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:25.255674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3327bcd-2bb4-432f-a43c-42f5e9565118 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:25.285083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 398/797] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:25.291138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3327bcd-2bb4-432f-a43c-42f5e9565118 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:25.298673 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:25.306897 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037b27b-3e39-49dc-a2f4-66c78ba185a1 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:25.347038 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6bbf7-eccb-419c-9008-5bf970dadde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:25.348363 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6bbf7-eccb-419c-9008-5bf970dadde8 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:25.348856 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6bbf7-eccb-419c-9008-5bf970dadde8 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:25.349430 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6bbf7-eccb-419c-9008-5bf970dadde8 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:25.350783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6bbf7-eccb-419c-9008-5bf970dadde8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:25.351977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6bbf7-eccb-419c-9008-5bf970dadde8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:25.353648 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6bbf7-eccb-419c-9008-5bf970dadde8 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:25.375888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 400/798] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 19:37:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:25.390332 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6bbf7-eccb-419c-9008-5bf970dadde8 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:25.503988 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3327bcd-2bb4-432f-a43c-42f5e9565118 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:25.509426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6bbf7-eccb-419c-9008-5bf970dadde8 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:25.524383 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:25.538135 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db07c7d-d5ef-48a2-a775-c7027ce92233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:25.538909 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db07c7d-d5ef-48a2-a775-c7027ce92233 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:25.539185 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db07c7d-d5ef-48a2-a775-c7027ce92233 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:25.539456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db07c7d-d5ef-48a2-a775-c7027ce92233 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:25.549231 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6baaa-51f3-4b42-a94d-66e8bd279137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:25.550327 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6baaa-51f3-4b42-a94d-66e8bd279137 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:25.550834 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6baaa-51f3-4b42-a94d-66e8bd279137 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:25.551355 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6baaa-51f3-4b42-a94d-66e8bd279137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:25.551657 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db07c7d-d5ef-48a2-a775-c7027ce92233 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:25.553354 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db07c7d-d5ef-48a2-a775-c7027ce92233 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:25.562873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6baaa-51f3-4b42-a94d-66e8bd279137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:25.565249 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6baaa-51f3-4b42-a94d-66e8bd279137 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:25.570639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-3db07c7d-d5ef-48a2-a775-c7027ce92233 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:25.570639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:25.570639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:25.570639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:25.570639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:25.570639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:25.570639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:25.570639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:25.570639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:25.570639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:25.570639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:25.570639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:25.570639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:25.570639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:25.570639 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:25.571497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:25.572368 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:25.572368 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:25.572368 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:25.572368 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:25.572368 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:25.572368 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:25.572368 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:25.572368 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:25.572368 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 399/799] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:25.578589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb9277-9d8a-4df0-b0db-6176ba0c72b6 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:25.578953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb9277-9d8a-4df0-b0db-6176ba0c72b6 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:25.579357 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb9277-9d8a-4df0-b0db-6176ba0c72b6 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:25.581150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 400/800] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:25.582231 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb9277-9d8a-4df0-b0db-6176ba0c72b6 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:25.590812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-aba6baaa-51f3-4b42-a94d-66e8bd279137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:25.590812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:25.590812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:25.590812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:25.590812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:25.590812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:25.590812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:25.590812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:25.590812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:25.590812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:25.590812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:25.590812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:25.590812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:25.590812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:25.590812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:25.591699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:25.592581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:25.592581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:25.592581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:25.592581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:25.595048 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 401/801] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:25.596943 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90826d4-dcdb-42fe-a88d-8eda5bfdcf55 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:25.597408 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90826d4-dcdb-42fe-a88d-8eda5bfdcf55 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:25.597870 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90826d4-dcdb-42fe-a88d-8eda5bfdcf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:25.618016 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-e90826d4-dcdb-42fe-a88d-8eda5bfdcf55 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:25.619568 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-e90826d4-dcdb-42fe-a88d-8eda5bfdcf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:25.653799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:25.698301 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 402/802] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:25.699804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb9277-9d8a-4df0-b0db-6176ba0c72b6 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:25.707746 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90826d4-dcdb-42fe-a88d-8eda5bfdcf55 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:25.735758 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef742a-2114-49a1-b6f2-3c53f64b9c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:25.736422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef742a-2114-49a1-b6f2-3c53f64b9c9e None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:25.736700 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef742a-2114-49a1-b6f2-3c53f64b9c9e None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:25.736970 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef742a-2114-49a1-b6f2-3c53f64b9c9e None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:25.737768 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef742a-2114-49a1-b6f2-3c53f64b9c9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:25.738072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef742a-2114-49a1-b6f2-3c53f64b9c9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:25.739552 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef742a-2114-49a1-b6f2-3c53f64b9c9e None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:25.752263 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 401/803] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:25.753741 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1fbf6-fac4-4259-89ca-b754ee5acdd8 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:25.754170 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1fbf6-fac4-4259-89ca-b754ee5acdd8 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:25.754622 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1fbf6-fac4-4259-89ca-b754ee5acdd8 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:25.756302 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 402/804] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:25.757115 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1fbf6-fac4-4259-89ca-b754ee5acdd8 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:25.819153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:25.827143 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90826d4-dcdb-42fe-a88d-8eda5bfdcf55 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:25.863648 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd0cc8-d6ba-4a42-be01-be853bb50063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:25.864322 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd0cc8-d6ba-4a42-be01-be853bb50063 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:25.864598 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd0cc8-d6ba-4a42-be01-be853bb50063 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:25.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd0cc8-d6ba-4a42-be01-be853bb50063 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:25.865595 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd0cc8-d6ba-4a42-be01-be853bb50063 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:25.865896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd0cc8-d6ba-4a42-be01-be853bb50063 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:25.867326 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd0cc8-d6ba-4a42-be01-be853bb50063 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:25.872648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1fbf6-fac4-4259-89ca-b754ee5acdd8 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:25.879078 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 403/805] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:25.879883 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd0cc8-d6ba-4a42-be01-be853bb50063 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:25.901511 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:25.915893 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7349de-e082-4834-acc9-7abea668b3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:25.916687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7349de-e082-4834-acc9-7abea668b3be None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:25.916960 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7349de-e082-4834-acc9-7abea668b3be None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:25.917225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7349de-e082-4834-acc9-7abea668b3be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:25.929554 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7349de-e082-4834-acc9-7abea668b3be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-4e7349de-e082-4834-acc9-7abea668b3be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:25.937138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:25.938050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:25.939030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:25.939030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:25.939030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:25.939030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:25.939030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:25.939030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:25.939030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:25.939030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:25.939030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:25.939030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:25.939030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:37:25.939030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:25.939030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 403/806] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:37:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:25.940058 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2ac90-1374-4e8b-81ea-09f67db45184 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:25.940153 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2ac90-1374-4e8b-81ea-09f67db45184 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:25.940357 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2ac90-1374-4e8b-81ea-09f67db45184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:25.957803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-56d2ac90-1374-4e8b-81ea-09f67db45184 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:25.959031 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-56d2ac90-1374-4e8b-81ea-09f67db45184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:25.995861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd0cc8-d6ba-4a42-be01-be853bb50063 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:26.038290 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 404/807] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:26.038290 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2ac90-1374-4e8b-81ea-09f67db45184 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:26.041748 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c91d1-cf73-4dc4-935c-252ccbcfb5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:26.042772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c91d1-cf73-4dc4-935c-252ccbcfb5e7 None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:26.043160 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c91d1-cf73-4dc4-935c-252ccbcfb5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:26.043495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c91d1-cf73-4dc4-935c-252ccbcfb5e7 None admin] PATH_INFO: `/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/4d1c8759f0564f28b388106dbf615cb5/roles/a33caf2301fb44db90304eb3d9afc3a5` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:26.045717 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c91d1-cf73-4dc4-935c-252ccbcfb5e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecf87429758641eeb462b3e409cadc9e, user_id=4d1c8759f0564f28b388106dbf615cb5, role_id=a33caf2301fb44db90304eb3d9afc3a5)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:26.047468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c91d1-cf73-4dc4-935c-252ccbcfb5e7 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:26.063758 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 404/808] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:37:25 2022] PUT /identity/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/4d1c8759f0564f28b388106dbf615cb5/roles/a33caf2301fb44db90304eb3d9afc3a5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:37:26.064640 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c91d1-cf73-4dc4-935c-252ccbcfb5e7 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:26.082658 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:26.190608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2ac90-1374-4e8b-81ea-09f67db45184 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:26.210045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c91d1-cf73-4dc4-935c-252ccbcfb5e7 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:26.210792 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:26.244404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fee1c-bb91-496d-bca1-344eb4973607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:26.245538 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fee1c-bb91-496d-bca1-344eb4973607 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:26.245814 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fee1c-bb91-496d-bca1-344eb4973607 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:26.246084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fee1c-bb91-496d-bca1-344eb4973607 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:26.256150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fee1c-bb91-496d-bca1-344eb4973607 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:26.257645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fee1c-bb91-496d-bca1-344eb4973607 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:26.257954 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90239192-0354-420e-bea2-cb517aeb2cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:26.259082 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90239192-0354-420e-bea2-cb517aeb2cdc None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:26.259572 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90239192-0354-420e-bea2-cb517aeb2cdc None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:26.260047 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90239192-0354-420e-bea2-cb517aeb2cdc None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:26.261736 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90239192-0354-420e-bea2-cb517aeb2cdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:26.262425 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90239192-0354-420e-bea2-cb517aeb2cdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:26.264045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90239192-0354-420e-bea2-cb517aeb2cdc None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:26.275085 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 405/809] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:37:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:26.275422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-355fee1c-bb91-496d-bca1-344eb4973607 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:26.275422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:26.275422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:26.275422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:26.275422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:26.275422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:26.275422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:26.275422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:26.275422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:26.275422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:26.275422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:26.275422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:26.275422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:26.275422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:26.275422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:26.276347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:26.277229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:26.277229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:26.277229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:26.277229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:26.277229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:26.277229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:26.277229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:26.277229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:26.277229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 405/810] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:26.279086 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90239192-0354-420e-bea2-cb517aeb2cdc None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:26.280396 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fee1c-bb91-496d-bca1-344eb4973607 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:26.370536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:26.399034 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90239192-0354-420e-bea2-cb517aeb2cdc None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:26.401884 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fee1c-bb91-496d-bca1-344eb4973607 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:26.436086 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56d1b6-aada-4e67-8bfe-15e583cb18b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:26.436903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56d1b6-aada-4e67-8bfe-15e583cb18b7 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:26.437174 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56d1b6-aada-4e67-8bfe-15e583cb18b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:26.437458 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56d1b6-aada-4e67-8bfe-15e583cb18b7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:26.438094 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56d1b6-aada-4e67-8bfe-15e583cb18b7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:26.439724 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56d1b6-aada-4e67-8bfe-15e583cb18b7 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:26.445766 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db45956-345d-457f-a6b5-5f50d14777a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:26.446803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db45956-345d-457f-a6b5-5f50d14777a7 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:26.447279 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db45956-345d-457f-a6b5-5f50d14777a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:26.447738 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db45956-345d-457f-a6b5-5f50d14777a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-0c56d1b6-aada-4e67-8bfe-15e583cb18b7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:26.451916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:26.452841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:26.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 406/811] 10.222.0.84 () {56 vars in 1129 bytes} [Thu May 19 19:37:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:26.455331 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56d1b6-aada-4e67-8bfe-15e583cb18b7 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:26.460580 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db45956-345d-457f-a6b5-5f50d14777a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:26.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db45956-345d-457f-a6b5-5f50d14777a7 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:26.480103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-3db45956-345d-457f-a6b5-5f50d14777a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:26.480103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:26.480103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:26.480103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:26.480103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:26.480103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:26.480103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:26.480103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:26.480103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:26.480103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:26.480103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:26.480103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:26.480103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:26.480103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:26.480103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:26.481206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:26.482100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:26.482100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:26.482100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:26.482100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:26.482100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 406/812] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:26.482634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37527060-5250-40fd-99b4-eec37c591d5a None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:26.482916 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37527060-5250-40fd-99b4-eec37c591d5a None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:26.483186 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37527060-5250-40fd-99b4-eec37c591d5a None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:26.484598 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 407/813] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:26.487040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37527060-5250-40fd-99b4-eec37c591d5a None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:26.517751 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:26.570464 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56d1b6-aada-4e67-8bfe-15e583cb18b7 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:26.599988 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37527060-5250-40fd-99b4-eec37c591d5a None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:26.606645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d993adb-d898-4243-89cf-b3f6fcbaa9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:26.607303 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d993adb-d898-4243-89cf-b3f6fcbaa9bc None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:26.607579 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d993adb-d898-4243-89cf-b3f6fcbaa9bc None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:26.607845 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d993adb-d898-4243-89cf-b3f6fcbaa9bc None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:26.608623 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d993adb-d898-4243-89cf-b3f6fcbaa9bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:26.608921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d993adb-d898-4243-89cf-b3f6fcbaa9bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:26.610480 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d993adb-d898-4243-89cf-b3f6fcbaa9bc None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:26.620677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:26.624176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 407/814] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:26.625362 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d993adb-d898-4243-89cf-b3f6fcbaa9bc None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:26.641217 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012dbc47-372a-490c-acd0-259120df2330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:26.641930 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012dbc47-372a-490c-acd0-259120df2330 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:26.642200 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012dbc47-372a-490c-acd0-259120df2330 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:26.642586 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012dbc47-372a-490c-acd0-259120df2330 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:26.643213 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012dbc47-372a-490c-acd0-259120df2330 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:26.643514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012dbc47-372a-490c-acd0-259120df2330 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:26.644900 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012dbc47-372a-490c-acd0-259120df2330 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:26.649549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 408/815] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 19:37:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:26.650828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ea260-23ae-4d92-a133-ff8c3e2e220b None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:26.651109 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ea260-23ae-4d92-a133-ff8c3e2e220b None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:26.651376 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ea260-23ae-4d92-a133-ff8c3e2e220b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:26.670522 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-845ea260-23ae-4d92-a133-ff8c3e2e220b None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:26.671695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-845ea260-23ae-4d92-a133-ff8c3e2e220b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:26.746591 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 409/816] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:26.747409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ea260-23ae-4d92-a133-ff8c3e2e220b None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:26.762652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d993adb-d898-4243-89cf-b3f6fcbaa9bc None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:26.766036 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:26.804392 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1dbca-7cdb-4cf4-a824-927b3c94cdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:26.805059 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1dbca-7cdb-4cf4-a824-927b3c94cdcd None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:26.805337 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1dbca-7cdb-4cf4-a824-927b3c94cdcd None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:26.805610 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1dbca-7cdb-4cf4-a824-927b3c94cdcd None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:26.806355 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1dbca-7cdb-4cf4-a824-927b3c94cdcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:26.806661 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1dbca-7cdb-4cf4-a824-927b3c94cdcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:26.808030 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1dbca-7cdb-4cf4-a824-927b3c94cdcd None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:26.824513 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 408/817] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:26.824513 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1dbca-7cdb-4cf4-a824-927b3c94cdcd None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:26.861430 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ea260-23ae-4d92-a133-ff8c3e2e220b None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:26.887336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:26.902007 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c38204-e45e-41de-b898-1e5a27776cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:26.902931 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c38204-e45e-41de-b898-1e5a27776cf5 None admin] REQUEST_METHOD: `PUT` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:26.903208 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c38204-e45e-41de-b898-1e5a27776cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:26.903475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c38204-e45e-41de-b898-1e5a27776cf5 None admin] PATH_INFO: `/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/9e05338c14d642f3b31095f85e1e6ca4/roles/cefdc5c73ec347438e5b4687f191382c` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:26.923071 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c38204-e45e-41de-b898-1e5a27776cf5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecf87429758641eeb462b3e409cadc9e, user_id=9e05338c14d642f3b31095f85e1e6ca4, role_id=cefdc5c73ec347438e5b4687f191382c)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:26.925050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c38204-e45e-41de-b898-1e5a27776cf5 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:26.931232 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1dbca-7cdb-4cf4-a824-927b3c94cdcd None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:26.939418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 410/818] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:37:26 2022] PUT /identity/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/9e05338c14d642f3b31095f85e1e6ca4/roles/cefdc5c73ec347438e5b4687f191382c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:37:26.940468 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c38204-e45e-41de-b898-1e5a27776cf5 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:26.984185 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8a7cf-5a07-4cd7-9b89-01eeeeeedf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:26.984878 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8a7cf-5a07-4cd7-9b89-01eeeeeedf22 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:26.985156 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8a7cf-5a07-4cd7-9b89-01eeeeeedf22 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:26.985506 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8a7cf-5a07-4cd7-9b89-01eeeeeedf22 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:26.986448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8a7cf-5a07-4cd7-9b89-01eeeeeedf22 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:26.986622 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8a7cf-5a07-4cd7-9b89-01eeeeeedf22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:26.988233 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8a7cf-5a07-4cd7-9b89-01eeeeeedf22 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:26.998924 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 409/819] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 19:37:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:26.998924 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8a7cf-5a07-4cd7-9b89-01eeeeeedf22 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:27.063806 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:27.115056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c38204-e45e-41de-b898-1e5a27776cf5 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:27.123765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8a7cf-5a07-4cd7-9b89-01eeeeeedf22 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:27.151911 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:27.163536 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f51ad3-fd0c-496c-98e0-92c0233b6bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:27.164433 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f51ad3-fd0c-496c-98e0-92c0233b6bc6 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:27.164711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f51ad3-fd0c-496c-98e0-92c0233b6bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:27.164976 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f51ad3-fd0c-496c-98e0-92c0233b6bc6 None admin] PATH_INFO: `/v3/services/network` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:27.165633 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f51ad3-fd0c-496c-98e0-92c0233b6bc6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:27.165801 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52991bad-bc35-4807-b03c-0aa72d18cb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:27.167068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52991bad-bc35-4807-b03c-0aa72d18cb79 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:27.167068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52991bad-bc35-4807-b03c-0aa72d18cb79 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:27.167227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f51ad3-fd0c-496c-98e0-92c0233b6bc6 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:27.167298 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52991bad-bc35-4807-b03c-0aa72d18cb79 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:27.176269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52991bad-bc35-4807-b03c-0aa72d18cb79 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:27.177718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52991bad-bc35-4807-b03c-0aa72d18cb79 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-71f51ad3-fd0c-496c-98e0-92c0233b6bc6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:27.180695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:27.181919 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:27.182901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:27.182901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:27.182901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:27.182901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:27.182901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:27.182901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:27.182901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:27.182901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:27.182901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:27.182901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:27.182901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:27.182901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:27.182901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:37:27.182901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:27.182901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 411/820] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:26 2022] GET /identity/v3/services/network => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:27.183671 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f51ad3-fd0c-496c-98e0-92c0233b6bc6 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-52991bad-bc35-4807-b03c-0aa72d18cb79 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:27.195024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:27.196495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:27.199540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:27.199540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:27.199540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:27.199540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:27.199540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:27.199540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:27.199540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:27.199540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:27.199540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:37:27.199540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:27.199540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 410/821] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:37:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:27.199540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52991bad-bc35-4807-b03c-0aa72d18cb79 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:27.316235 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52991bad-bc35-4807-b03c-0aa72d18cb79 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:27.334765 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f51ad3-fd0c-496c-98e0-92c0233b6bc6 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:27.357120 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5d4da-f76a-4a20-9ec9-229959dfc76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:27.358176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5d4da-f76a-4a20-9ec9-229959dfc76c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:27.358770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5d4da-f76a-4a20-9ec9-229959dfc76c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:27.359250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5d4da-f76a-4a20-9ec9-229959dfc76c None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:27.360251 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5d4da-f76a-4a20-9ec9-229959dfc76c None admin] RBAC: Adding query filter params (name=network) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:27.361553 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5d4da-f76a-4a20-9ec9-229959dfc76c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:27.363395 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5d4da-f76a-4a20-9ec9-229959dfc76c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:27.365784 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:27.370412 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 411/822] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:37:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:27.371457 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5d4da-f76a-4a20-9ec9-229959dfc76c None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:27.375405 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722afbe-6613-4405-ba36-8af7c783906a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:27.377678 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722afbe-6613-4405-ba36-8af7c783906a None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:27.377678 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722afbe-6613-4405-ba36-8af7c783906a None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:27.377678 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722afbe-6613-4405-ba36-8af7c783906a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:27.377678 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722afbe-6613-4405-ba36-8af7c783906a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:27.379084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722afbe-6613-4405-ba36-8af7c783906a None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:27.414725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 412/823] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:37:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:37:27.415682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722afbe-6613-4405-ba36-8af7c783906a None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:27.430183 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:27.556344 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722afbe-6613-4405-ba36-8af7c783906a None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:27.579537 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5d4da-f76a-4a20-9ec9-229959dfc76c None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:27.622444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3facda-8fe1-44a7-9074-f6f84b52167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:27.623167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3facda-8fe1-44a7-9074-f6f84b52167d None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:27.623454 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3facda-8fe1-44a7-9074-f6f84b52167d None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:27.623724 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3facda-8fe1-44a7-9074-f6f84b52167d None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:27.624448 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3facda-8fe1-44a7-9074-f6f84b52167d None admin] RBAC: Adding query filter params (type=network) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:27.625184 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3facda-8fe1-44a7-9074-f6f84b52167d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:27.627149 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3facda-8fe1-44a7-9074-f6f84b52167d None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:27.635352 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 413/824] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:37:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:27.641394 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3facda-8fe1-44a7-9074-f6f84b52167d None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:27.653584 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd3e0e-11aa-4ab8-a82e-9691cec0285a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:27.654298 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd3e0e-11aa-4ab8-a82e-9691cec0285a None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:27.654626 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd3e0e-11aa-4ab8-a82e-9691cec0285a None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:27.654897 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd3e0e-11aa-4ab8-a82e-9691cec0285a None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:27.655747 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd3e0e-11aa-4ab8-a82e-9691cec0285a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:27.656050 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd3e0e-11aa-4ab8-a82e-9691cec0285a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:27.657659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd3e0e-11aa-4ab8-a82e-9691cec0285a None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:27.695997 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 412/825] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:37:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:27.703676 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd3e0e-11aa-4ab8-a82e-9691cec0285a None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:27.736398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:27.806306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3facda-8fe1-44a7-9074-f6f84b52167d None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:27.831866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd3e0e-11aa-4ab8-a82e-9691cec0285a None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:27.842022 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:27.850907 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da122e55-11c2-47b7-9e50-bcd91e080413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:27.852286 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da122e55-11c2-47b7-9e50-bcd91e080413 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:27.852572 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da122e55-11c2-47b7-9e50-bcd91e080413 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:27.852983 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da122e55-11c2-47b7-9e50-bcd91e080413 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:27.853845 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da122e55-11c2-47b7-9e50-bcd91e080413 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:27.855719 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da122e55-11c2-47b7-9e50-bcd91e080413 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:27.879915 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 414/826] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:37:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:37:27.893959 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1446bb2-b824-40df-8738-456b14797072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:27.894671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1446bb2-b824-40df-8738-456b14797072 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:27.894952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1446bb2-b824-40df-8738-456b14797072 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:27.895229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1446bb2-b824-40df-8738-456b14797072 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:27.986528 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1446bb2-b824-40df-8738-456b14797072 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:27.987650 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1446bb2-b824-40df-8738-456b14797072 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:28.100325 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 413/827] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:37:27 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:37:28.105779 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da122e55-11c2-47b7-9e50-bcd91e080413 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:28.210953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da122e55-11c2-47b7-9e50-bcd91e080413 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:28.213362 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:28.250700 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75bf8d-1870-4dfc-996b-597cc7c3d2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:28.251894 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75bf8d-1870-4dfc-996b-597cc7c3d2a2 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:28.251894 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75bf8d-1870-4dfc-996b-597cc7c3d2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:28.252239 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75bf8d-1870-4dfc-996b-597cc7c3d2a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:28.263431 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75bf8d-1870-4dfc-996b-597cc7c3d2a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:28.264896 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75bf8d-1870-4dfc-996b-597cc7c3d2a2 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:28.283856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-bd75bf8d-1870-4dfc-996b-597cc7c3d2a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:28.283856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:28.283856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:28.283856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:28.283856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:28.283856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:28.283856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:28.283856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:28.283856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:28.283856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:28.283856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:28.283856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:28.283856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:28.283856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:28.283856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:28.285985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:28.288148 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:28.288148 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:28.288148 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:28.288148 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:28.288148 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 415/828] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:28.291573 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1446bb2-b824-40df-8738-456b14797072 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:28.293862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:28.408758 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1446bb2-b824-40df-8738-456b14797072 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:28.446641 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1cb30-ea51-4649-b504-bda8ffc699c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:28.447317 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1cb30-ea51-4649-b504-bda8ffc699c4 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:28.447599 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1cb30-ea51-4649-b504-bda8ffc699c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:28.447874 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1cb30-ea51-4649-b504-bda8ffc699c4 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:28.448600 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1cb30-ea51-4649-b504-bda8ffc699c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:28.448903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1cb30-ea51-4649-b504-bda8ffc699c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:28.450297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1cb30-ea51-4649-b504-bda8ffc699c4 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:28.464564 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 414/829] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:28.470035 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75bf8d-1870-4dfc-996b-597cc7c3d2a2 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:28.507140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6482f9e-7c0f-486d-b9ba-198de48145fd None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:28.507462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6482f9e-7c0f-486d-b9ba-198de48145fd None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:28.507625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6482f9e-7c0f-486d-b9ba-198de48145fd None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:28.509229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 415/830] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:37:28.514694 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617554f7-964d-44e1-bf4f-c4122940fc28 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:28.515078 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617554f7-964d-44e1-bf4f-c4122940fc28 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:28.515353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617554f7-964d-44e1-bf4f-c4122940fc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:28.536039 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-617554f7-964d-44e1-bf4f-c4122940fc28 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:28.537225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-617554f7-964d-44e1-bf4f-c4122940fc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:28.585194 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75bf8d-1870-4dfc-996b-597cc7c3d2a2 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:28.602637 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:28.617448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 416/831] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:37:28.639538 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617554f7-964d-44e1-bf4f-c4122940fc28 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:28.642703 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae03e3-3f84-4276-9c69-6cc6d6e0163d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:28.643758 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae03e3-3f84-4276-9c69-6cc6d6e0163d None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:28.644235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae03e3-3f84-4276-9c69-6cc6d6e0163d None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:28.644809 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae03e3-3f84-4276-9c69-6cc6d6e0163d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:28.652254 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae03e3-3f84-4276-9c69-6cc6d6e0163d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:28.653999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae03e3-3f84-4276-9c69-6cc6d6e0163d None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:28.668620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-8dae03e3-3f84-4276-9c69-6cc6d6e0163d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:28.668620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:28.668620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:28.668620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:28.668620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:28.668620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:28.668620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:28.668620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:28.668620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:28.668620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:28.668620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:28.668620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:28.668620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:28.668620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:28.668620 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:28.669513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:28.670406 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:28.670406 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:28.670406 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:28.670406 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:28.670406 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:28.670406 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:28.670406 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:28.670406 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:28.672855 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 416/832] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:28.682293 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae03e3-3f84-4276-9c69-6cc6d6e0163d None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:28.702706 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:28.744678 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617554f7-964d-44e1-bf4f-c4122940fc28 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:28.779414 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0732a-bfbd-4824-8b48-1d2a0fd50663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:28.780120 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0732a-bfbd-4824-8b48-1d2a0fd50663 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:28.780406 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0732a-bfbd-4824-8b48-1d2a0fd50663 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:28.780809 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0732a-bfbd-4824-8b48-1d2a0fd50663 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:28.791022 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0732a-bfbd-4824-8b48-1d2a0fd50663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-2cb0732a-bfbd-4824-8b48-1d2a0fd50663 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:28.798662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:28.799602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:28.800426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:28.800426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:28.800426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:28.800426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:28.800426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:28.800426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:28.800426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:28.800426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:28.800426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:28.800426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:28.800426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:37:28.800426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:28.800426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 417/833] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:37:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:28.802236 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae03e3-3f84-4276-9c69-6cc6d6e0163d None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:28.805423 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0732a-bfbd-4824-8b48-1d2a0fd50663 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:28.840005 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65014abd-e5c3-44a8-a881-263d0fb4570f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:28.840667 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65014abd-e5c3-44a8-a881-263d0fb4570f None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:28.840944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65014abd-e5c3-44a8-a881-263d0fb4570f None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:28.841214 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65014abd-e5c3-44a8-a881-263d0fb4570f None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:28.842005 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65014abd-e5c3-44a8-a881-263d0fb4570f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:28.842334 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65014abd-e5c3-44a8-a881-263d0fb4570f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:28.843913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65014abd-e5c3-44a8-a881-263d0fb4570f None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:28.861970 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 417/834] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:28.863720 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65014abd-e5c3-44a8-a881-263d0fb4570f None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:28.871004 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:28.932692 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0732a-bfbd-4824-8b48-1d2a0fd50663 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:28.966187 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:28.976490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244a38d-3222-4a05-bea6-cbe42e095218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:28.977201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244a38d-3222-4a05-bea6-cbe42e095218 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:28.977609 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244a38d-3222-4a05-bea6-cbe42e095218 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:28.977842 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244a38d-3222-4a05-bea6-cbe42e095218 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:28.978647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244a38d-3222-4a05-bea6-cbe42e095218 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:28.979189 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244a38d-3222-4a05-bea6-cbe42e095218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:28.980455 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244a38d-3222-4a05-bea6-cbe42e095218 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:28.990202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 418/835] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:37:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:28.995167 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244a38d-3222-4a05-bea6-cbe42e095218 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:28.996311 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65014abd-e5c3-44a8-a881-263d0fb4570f None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:29.033486 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d72f1-b25a-4cbc-8bc2-80675eea72d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:29.034339 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d72f1-b25a-4cbc-8bc2-80675eea72d3 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:29.034719 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d72f1-b25a-4cbc-8bc2-80675eea72d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:29.035098 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d72f1-b25a-4cbc-8bc2-80675eea72d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:29.036194 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d72f1-b25a-4cbc-8bc2-80675eea72d3 None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=abde7cc02fae4d9bb561a6642c5252a6) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:29.036489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d72f1-b25a-4cbc-8bc2-80675eea72d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:29.038085 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d72f1-b25a-4cbc-8bc2-80675eea72d3 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:29.046573 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 418/836] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:28 2022] GET /identity/v3/role_assignments?user.id=abde7cc02fae4d9bb561a6642c5252a6&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:29.123267 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244a38d-3222-4a05-bea6-cbe42e095218 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:29.138870 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:29.161833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136cfe0-ba47-4a00-8412-3ff2d4e61a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:29.162681 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136cfe0-ba47-4a00-8412-3ff2d4e61a72 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:29.162962 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136cfe0-ba47-4a00-8412-3ff2d4e61a72 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:29.163235 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136cfe0-ba47-4a00-8412-3ff2d4e61a72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:29.175851 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136cfe0-ba47-4a00-8412-3ff2d4e61a72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:29.177344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136cfe0-ba47-4a00-8412-3ff2d4e61a72 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:29.196102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-5136cfe0-ba47-4a00-8412-3ff2d4e61a72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:29.196102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:29.196102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:29.196102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:29.196102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:29.196102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:29.196102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:29.196102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:29.196102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:29.196102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:29.196102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:29.196102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:29.196102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:29.196102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:29.196102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:29.197056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:29.197920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:29.197920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:29.197920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:29.197920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:29.197920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 419/837] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:29.203160 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d72f1-b25a-4cbc-8bc2-80675eea72d3 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:29.324599 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d72f1-b25a-4cbc-8bc2-80675eea72d3 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:29.362360 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ec701-cf56-4fa5-ad80-ff76f52b506c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:29.363036 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ec701-cf56-4fa5-ad80-ff76f52b506c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:29.363301 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ec701-cf56-4fa5-ad80-ff76f52b506c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:29.363563 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ec701-cf56-4fa5-ad80-ff76f52b506c None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:29.364290 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ec701-cf56-4fa5-ad80-ff76f52b506c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:29.364587 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ec701-cf56-4fa5-ad80-ff76f52b506c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:29.366241 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ec701-cf56-4fa5-ad80-ff76f52b506c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:29.376361 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:29.380098 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 419/838] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:29.385525 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136cfe0-ba47-4a00-8412-3ff2d4e61a72 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:29.487743 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3bd72-7095-4791-b981-69c2cfd9b2b1 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:29.488730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3bd72-7095-4791-b981-69c2cfd9b2b1 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:29.489239 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3bd72-7095-4791-b981-69c2cfd9b2b1 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:29.491063 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 420/839] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:37:29.498865 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29ef1f-6de4-4e37-8954-bda5eee7ebd3 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:29.499384 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29ef1f-6de4-4e37-8954-bda5eee7ebd3 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:29.499880 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29ef1f-6de4-4e37-8954-bda5eee7ebd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:29.502233 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136cfe0-ba47-4a00-8412-3ff2d4e61a72 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:29.517486 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-cd29ef1f-6de4-4e37-8954-bda5eee7ebd3 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:29.519074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-cd29ef1f-6de4-4e37-8954-bda5eee7ebd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:29.542040 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea92e37-6da6-43b3-9cee-ed63b16ba7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:29.542928 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea92e37-6da6-43b3-9cee-ed63b16ba7c8 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:29.543210 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea92e37-6da6-43b3-9cee-ed63b16ba7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:29.543486 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea92e37-6da6-43b3-9cee-ed63b16ba7c8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:29.553062 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea92e37-6da6-43b3-9cee-ed63b16ba7c8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:29.554557 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea92e37-6da6-43b3-9cee-ed63b16ba7c8 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-bea92e37-6da6-43b3-9cee-ed63b16ba7c8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:29.570504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:29.571466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:29.572372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:29.572372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:29.572372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:29.572372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:29.572372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:29.572372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:29.572372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:29.572372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:29.572372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:37:29.572372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:29.572372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 420/840] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:37:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:29.577319 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea92e37-6da6-43b3-9cee-ed63b16ba7c8 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:29.606586 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:29.618399 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 421/841] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:37:29.640235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29ef1f-6de4-4e37-8954-bda5eee7ebd3 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:29.705672 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea92e37-6da6-43b3-9cee-ed63b16ba7c8 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:29.742938 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29ef1f-6de4-4e37-8954-bda5eee7ebd3 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:29.743137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46c95d-80cd-4dd7-9654-cc94bd1fca03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:29.743663 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46c95d-80cd-4dd7-9654-cc94bd1fca03 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:29.743943 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46c95d-80cd-4dd7-9654-cc94bd1fca03 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:29.744219 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46c95d-80cd-4dd7-9654-cc94bd1fca03 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:29.745062 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46c95d-80cd-4dd7-9654-cc94bd1fca03 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:29.745374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46c95d-80cd-4dd7-9654-cc94bd1fca03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:29.746901 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46c95d-80cd-4dd7-9654-cc94bd1fca03 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:29.770283 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 421/842] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 19:37:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:29.771910 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46c95d-80cd-4dd7-9654-cc94bd1fca03 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:29.779243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:29.788368 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657b42e-7526-47c5-99e4-ed985fb6f8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:29.789633 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657b42e-7526-47c5-99e4-ed985fb6f8b6 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:29.790222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657b42e-7526-47c5-99e4-ed985fb6f8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:29.790717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657b42e-7526-47c5-99e4-ed985fb6f8b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:29.802802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657b42e-7526-47c5-99e4-ed985fb6f8b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-c657b42e-7526-47c5-99e4-ed985fb6f8b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:29.811946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:29.812925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:29.813999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:29.813999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:29.813999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:29.813999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:29.813999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:29.813999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:29.813999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:29.813999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:29.813999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:29.813999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:29.813999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:37:29.813999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:29.821906 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 422/843] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:37:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:29.823373 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657b42e-7526-47c5-99e4-ed985fb6f8b6 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:29.872256 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:29.893041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46c95d-80cd-4dd7-9654-cc94bd1fca03 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:29.929978 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c8918-00bf-4405-ba5e-75307c8056bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:29.930665 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c8918-00bf-4405-ba5e-75307c8056bd None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:29.930949 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c8918-00bf-4405-ba5e-75307c8056bd None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:29.931224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c8918-00bf-4405-ba5e-75307c8056bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:29.966508 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657b42e-7526-47c5-99e4-ed985fb6f8b6 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:30.004642 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c9023-417a-46a9-856a-90f1186c2158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:30.005318 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c9023-417a-46a9-856a-90f1186c2158 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:30.005598 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c9023-417a-46a9-856a-90f1186c2158 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:30.005868 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c9023-417a-46a9-856a-90f1186c2158 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:30.006659 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c9023-417a-46a9-856a-90f1186c2158 None admin] RBAC: Adding query filter params (name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:30.007236 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c9023-417a-46a9-856a-90f1186c2158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:30.008370 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c9023-417a-46a9-856a-90f1186c2158 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:30.016918 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 423/844] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:37:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:30.018722 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d929d-50d1-44cf-abdd-740c072d4e80 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:30.019011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d929d-50d1-44cf-abdd-740c072d4e80 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:30.019285 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d929d-50d1-44cf-abdd-740c072d4e80 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:30.020710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 424/845] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:30.024365 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c8918-00bf-4405-ba5e-75307c8056bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:30.025948 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88a3ee-394f-4e90-a957-3bb5c8633de5 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:30.026135 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c8918-00bf-4405-ba5e-75307c8056bd None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:30.026358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88a3ee-394f-4e90-a957-3bb5c8633de5 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:30.026601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88a3ee-394f-4e90-a957-3bb5c8633de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:30.045941 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-4b88a3ee-394f-4e90-a957-3bb5c8633de5 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:30.047150 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-4b88a3ee-394f-4e90-a957-3bb5c8633de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:30.097823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:30.120437 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 425/846] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:37:30.121746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88a3ee-394f-4e90-a957-3bb5c8633de5 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:30.131171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:30.152938 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 422/847] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:37:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:37:30.154208 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c8918-00bf-4405-ba5e-75307c8056bd None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:30.232120 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88a3ee-394f-4e90-a957-3bb5c8633de5 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:30.256628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c8918-00bf-4405-ba5e-75307c8056bd None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:30.266852 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ece372-760e-4224-b9cc-accee9a8912a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:30.267732 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ece372-760e-4224-b9cc-accee9a8912a None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:30.268009 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ece372-760e-4224-b9cc-accee9a8912a None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:30.268278 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ece372-760e-4224-b9cc-accee9a8912a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:30.278287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ece372-760e-4224-b9cc-accee9a8912a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:30.279748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ece372-760e-4224-b9cc-accee9a8912a None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:30.292647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b0058-a774-4c5c-a170-b0190f3cd688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:30.294208 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b0058-a774-4c5c-a170-b0190f3cd688 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:30.294551 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b0058-a774-4c5c-a170-b0190f3cd688 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:30.294828 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b0058-a774-4c5c-a170-b0190f3cd688 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:30.295562 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b0058-a774-4c5c-a170-b0190f3cd688 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:30.296926 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b0058-a774-4c5c-a170-b0190f3cd688 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:30.297864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-b8ece372-760e-4224-b9cc-accee9a8912a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:30.297864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:30.297864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:30.297864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:30.297864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:30.297864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:30.297864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:30.297864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:30.297864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:30.297864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:30.297864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:30.297864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:30.297864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:30.297864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:30.297864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:30.298762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:30.299605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:30.299605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:30.299605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:30.299605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:30.300346 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 426/848] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:30.300631 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ece372-760e-4224-b9cc-accee9a8912a None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-ed1b0058-a774-4c5c-a170-b0190f3cd688 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:30.313003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:30.313993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:30.314831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:30.314831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:30.314831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:30.314831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:30.314831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:30.314831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:30.314831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:30.314831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:30.314831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:30.314831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:30.314831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:30.314831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:30.314831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:37:30.314831 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:30.319212 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 423/849] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:37:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:30.320040 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b0058-a774-4c5c-a170-b0190f3cd688 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:30.338102 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:30.372753 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:30.412650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ece372-760e-4224-b9cc-accee9a8912a None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:30.433524 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b0058-a774-4c5c-a170-b0190f3cd688 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:30.449430 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6edcd3-5e75-4c6a-93b2-6203203e0f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:30.450230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6edcd3-5e75-4c6a-93b2-6203203e0f78 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:30.450534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6edcd3-5e75-4c6a-93b2-6203203e0f78 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:30.450809 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6edcd3-5e75-4c6a-93b2-6203203e0f78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:30.461341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6edcd3-5e75-4c6a-93b2-6203203e0f78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:30.462831 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6edcd3-5e75-4c6a-93b2-6203203e0f78 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:30.471143 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35710ddc-30f1-4421-89bd-3a1e48c9f587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:30.471785 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35710ddc-30f1-4421-89bd-3a1e48c9f587 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:30.472062 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35710ddc-30f1-4421-89bd-3a1e48c9f587 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:30.472354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35710ddc-30f1-4421-89bd-3a1e48c9f587 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:30.473060 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35710ddc-30f1-4421-89bd-3a1e48c9f587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:30.473366 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35710ddc-30f1-4421-89bd-3a1e48c9f587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:30.474781 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35710ddc-30f1-4421-89bd-3a1e48c9f587 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:30.481625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-2e6edcd3-5e75-4c6a-93b2-6203203e0f78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:30.481625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:30.481625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:30.481625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:30.481625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:30.481625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:30.481625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:30.481625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:30.481625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:30.481625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:30.481625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:30.481625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:30.481625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:30.481625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:30.481625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:30.482608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:30.483504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:30.483504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:30.483504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:30.483504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:30.483504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 427/850] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:30.483776 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6edcd3-5e75-4c6a-93b2-6203203e0f78 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:30.492604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 424/851] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:30.493722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35710ddc-30f1-4421-89bd-3a1e48c9f587 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:30.571903 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:30.587152 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6edcd3-5e75-4c6a-93b2-6203203e0f78 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:30.605239 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35710ddc-30f1-4421-89bd-3a1e48c9f587 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:30.619998 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:30.623120 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14944dd-e2d9-48b7-995a-8550d8be92f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:30.623787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14944dd-e2d9-48b7-995a-8550d8be92f9 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:30.624064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14944dd-e2d9-48b7-995a-8550d8be92f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:30.624389 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14944dd-e2d9-48b7-995a-8550d8be92f9 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:30.627909 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14944dd-e2d9-48b7-995a-8550d8be92f9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:30.627909 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14944dd-e2d9-48b7-995a-8550d8be92f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:30.627909 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14944dd-e2d9-48b7-995a-8550d8be92f9 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:30.636517 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 428/852] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:37:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:30.636517 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14944dd-e2d9-48b7-995a-8550d8be92f9 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:30.656085 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ad2ce-6ab7-463c-bb9b-30f96334f3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:30.656752 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ad2ce-6ab7-463c-bb9b-30f96334f3c5 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:30.657102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ad2ce-6ab7-463c-bb9b-30f96334f3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:30.657701 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ad2ce-6ab7-463c-bb9b-30f96334f3c5 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:30.658058 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ad2ce-6ab7-463c-bb9b-30f96334f3c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:30.658397 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ad2ce-6ab7-463c-bb9b-30f96334f3c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:30.659765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ad2ce-6ab7-463c-bb9b-30f96334f3c5 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:30.673785 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 425/853] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:30.674540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ad2ce-6ab7-463c-bb9b-30f96334f3c5 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:30.737577 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14944dd-e2d9-48b7-995a-8550d8be92f9 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:30.776951 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ad2ce-6ab7-463c-bb9b-30f96334f3c5 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:30.777915 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64901e21-c060-41da-a545-4c101f158a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:30.778922 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64901e21-c060-41da-a545-4c101f158a88 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:30.779954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64901e21-c060-41da-a545-4c101f158a88 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:30.780234 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64901e21-c060-41da-a545-4c101f158a88 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:30.789036 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64901e21-c060-41da-a545-4c101f158a88 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:30.791067 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64901e21-c060-41da-a545-4c101f158a88 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-64901e21-c060-41da-a545-4c101f158a88 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:30.808770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:30.809729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:30.810704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:30.810704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:30.810704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:30.810704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:30.810704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:30.810704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:30.810704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:30.810704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:30.810704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:37:30.810704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:30.810704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 429/854] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:37:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:30.811286 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64901e21-c060-41da-a545-4c101f158a88 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:30.816239 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616368d-d880-42c3-aae1-1e14ecdac258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:30.816907 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616368d-d880-42c3-aae1-1e14ecdac258 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:30.817213 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616368d-d880-42c3-aae1-1e14ecdac258 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:30.817444 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616368d-d880-42c3-aae1-1e14ecdac258 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:30.818181 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616368d-d880-42c3-aae1-1e14ecdac258 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:30.818531 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616368d-d880-42c3-aae1-1e14ecdac258 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:30.819894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616368d-d880-42c3-aae1-1e14ecdac258 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:30.824073 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:30.824849 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 426/855] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:37:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:30.825628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616368d-d880-42c3-aae1-1e14ecdac258 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:30.862210 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:30.947360 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616368d-d880-42c3-aae1-1e14ecdac258 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:30.966603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64901e21-c060-41da-a545-4c101f158a88 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:30.988645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7414342-792a-4266-8bd4-7991b3d844f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:30.989340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7414342-792a-4266-8bd4-7991b3d844f9 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:30.989619 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7414342-792a-4266-8bd4-7991b3d844f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:30.989892 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7414342-792a-4266-8bd4-7991b3d844f9 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:30.990875 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7414342-792a-4266-8bd4-7991b3d844f9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:30.991091 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7414342-792a-4266-8bd4-7991b3d844f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:30.992568 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7414342-792a-4266-8bd4-7991b3d844f9 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:31.017468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 427/856] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 19:37:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:31.019119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7414342-792a-4266-8bd4-7991b3d844f9 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:31.025645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d78571-fd4a-4339-9dbc-a3fa907371b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:31.026804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d78571-fd4a-4339-9dbc-a3fa907371b1 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:31.027289 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d78571-fd4a-4339-9dbc-a3fa907371b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:31.027775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d78571-fd4a-4339-9dbc-a3fa907371b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:31.037962 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d78571-fd4a-4339-9dbc-a3fa907371b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:31.040268 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d78571-fd4a-4339-9dbc-a3fa907371b1 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:31.059248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-89d78571-fd4a-4339-9dbc-a3fa907371b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:31.059248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:31.059248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:31.059248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:31.059248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:31.059248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:31.059248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:31.059248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:31.059248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:31.059248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:31.059248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:31.059248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:31.059248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:31.059248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:31.059248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:31.060220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:31.061106 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:31.061106 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:31.061106 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:31.061106 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:31.061106 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:31.061106 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:31.061106 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:31.061106 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:31.068545 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 430/857] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:31.070624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d78571-fd4a-4339-9dbc-a3fa907371b1 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:31.136990 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:31.149109 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:31.151794 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7414342-792a-4266-8bd4-7991b3d844f9 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:31.183356 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d78571-fd4a-4339-9dbc-a3fa907371b1 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:31.201634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab61c7-167a-4351-9c60-6d34b7df8694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:31.202796 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab61c7-167a-4351-9c60-6d34b7df8694 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:31.203178 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab61c7-167a-4351-9c60-6d34b7df8694 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:31.203547 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab61c7-167a-4351-9c60-6d34b7df8694 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:31.220871 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e799b-dd8c-45d8-9c92-478a9e409d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:31.221881 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e799b-dd8c-45d8-9c92-478a9e409d7c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:31.222452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e799b-dd8c-45d8-9c92-478a9e409d7c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:31.222923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e799b-dd8c-45d8-9c92-478a9e409d7c None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:31.223893 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e799b-dd8c-45d8-9c92-478a9e409d7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:31.224416 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e799b-dd8c-45d8-9c92-478a9e409d7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:31.226439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e799b-dd8c-45d8-9c92-478a9e409d7c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:31.245156 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 431/858] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:31.246880 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e799b-dd8c-45d8-9c92-478a9e409d7c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:31.290825 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab61c7-167a-4351-9c60-6d34b7df8694 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:31.292711 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab61c7-167a-4351-9c60-6d34b7df8694 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:31.380215 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e799b-dd8c-45d8-9c92-478a9e409d7c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:31.402498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:31.410963 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:31.417693 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624b735-fb0f-4ad9-87b7-62680b115843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:31.418458 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624b735-fb0f-4ad9-87b7-62680b115843 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:31.418745 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624b735-fb0f-4ad9-87b7-62680b115843 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:31.419024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624b735-fb0f-4ad9-87b7-62680b115843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:31.419178 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 428/859] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:37:31 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:37:31.426503 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624b735-fb0f-4ad9-87b7-62680b115843 None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=4d1c8759f0564f28b388106dbf615cb5) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:31.426503 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624b735-fb0f-4ad9-87b7-62680b115843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:31.426503 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624b735-fb0f-4ad9-87b7-62680b115843 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:31.426824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab61c7-167a-4351-9c60-6d34b7df8694 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:31.431864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 432/860] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:31 2022] GET /identity/v3/role_assignments?user.id=4d1c8759f0564f28b388106dbf615cb5&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:31.460050 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1771f-b635-4b79-b323-4123b336873c None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:31.460710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1771f-b635-4b79-b323-4123b336873c None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:31.461136 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1771f-b635-4b79-b323-4123b336873c None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:31.464007 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 433/861] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:31.470072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77098438-345c-48b9-94b2-d4fd724f75eb None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:31.470450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77098438-345c-48b9-94b2-d4fd724f75eb None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:31.471663 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77098438-345c-48b9-94b2-d4fd724f75eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:31.490608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-77098438-345c-48b9-94b2-d4fd724f75eb None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:31.491817 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-77098438-345c-48b9-94b2-d4fd724f75eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:31.548970 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab61c7-167a-4351-9c60-6d34b7df8694 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:31.584663 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 434/862] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:37:31.592357 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9a4f5-3de7-47ff-8e2b-c5344f86deef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:31.593431 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9a4f5-3de7-47ff-8e2b-c5344f86deef None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:31.593851 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9a4f5-3de7-47ff-8e2b-c5344f86deef None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:31.594282 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9a4f5-3de7-47ff-8e2b-c5344f86deef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:31.607217 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9a4f5-3de7-47ff-8e2b-c5344f86deef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:31.608259 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77098438-345c-48b9-94b2-d4fd724f75eb None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:31.617570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9a4f5-3de7-47ff-8e2b-c5344f86deef None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:31.647027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-a8a9a4f5-3de7-47ff-8e2b-c5344f86deef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:31.647027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:31.647027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:31.647027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:31.647027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:31.647027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:31.647027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:31.647027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:31.647027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:31.647027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:31.647027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:31.647027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:31.647027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:31.647027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:31.647027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:31.649999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:31.650897 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:31.650897 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:31.650897 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:31.650897 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:31.652821 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 429/863] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:31.659991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9a4f5-3de7-47ff-8e2b-c5344f86deef None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:31.716629 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:31.725536 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77098438-345c-48b9-94b2-d4fd724f75eb None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:31.761224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:31.773222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44474bf7-55b1-4403-8b31-a73860661fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:31.774042 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44474bf7-55b1-4403-8b31-a73860661fc7 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:31.774343 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44474bf7-55b1-4403-8b31-a73860661fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:31.774672 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44474bf7-55b1-4403-8b31-a73860661fc7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:31.775329 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44474bf7-55b1-4403-8b31-a73860661fc7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:31.776756 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44474bf7-55b1-4403-8b31-a73860661fc7 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-44474bf7-55b1-4403-8b31-a73860661fc7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:31.789037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:31.790044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:31.790956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 435/864] 10.222.0.84 () {56 vars in 1132 bytes} [Thu May 19 19:37:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:31.793608 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9a4f5-3de7-47ff-8e2b-c5344f86deef None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:31.795938 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44474bf7-55b1-4403-8b31-a73860661fc7 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:31.828689 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422edce-0db0-42da-b0a3-78cc726c288b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:31.829401 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422edce-0db0-42da-b0a3-78cc726c288b None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:31.829678 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422edce-0db0-42da-b0a3-78cc726c288b None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:31.829947 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422edce-0db0-42da-b0a3-78cc726c288b None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:31.830692 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422edce-0db0-42da-b0a3-78cc726c288b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:31.830991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422edce-0db0-42da-b0a3-78cc726c288b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:31.832366 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422edce-0db0-42da-b0a3-78cc726c288b None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:31.844357 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 430/865] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:31.853729 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422edce-0db0-42da-b0a3-78cc726c288b None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:31.920931 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44474bf7-55b1-4403-8b31-a73860661fc7 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:31.972232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9f8ca-7be9-4010-a650-051e1a594147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:31.973388 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9f8ca-7be9-4010-a650-051e1a594147 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:31.973823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9f8ca-7be9-4010-a650-051e1a594147 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:31.974249 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9f8ca-7be9-4010-a650-051e1a594147 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:31.974884 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422edce-0db0-42da-b0a3-78cc726c288b None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:31.975632 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9f8ca-7be9-4010-a650-051e1a594147 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:31.976423 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9f8ca-7be9-4010-a650-051e1a594147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:31.977158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:31.979765 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9f8ca-7be9-4010-a650-051e1a594147 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:31.985759 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 436/866] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:37:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:31.991175 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9f8ca-7be9-4010-a650-051e1a594147 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:32.014545 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37743a-7417-4dcc-bdeb-d98f6c212ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:32.015682 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37743a-7417-4dcc-bdeb-d98f6c212ed9 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:32.015893 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37743a-7417-4dcc-bdeb-d98f6c212ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:32.016251 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37743a-7417-4dcc-bdeb-d98f6c212ed9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:32.024810 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37743a-7417-4dcc-bdeb-d98f6c212ed9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:32.026548 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37743a-7417-4dcc-bdeb-d98f6c212ed9 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:32.041041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-4d37743a-7417-4dcc-bdeb-d98f6c212ed9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:32.041041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:32.041041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:32.041041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:32.041041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:32.041041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:32.041041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:32.041041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:32.041041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:32.041041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:32.041041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:32.041041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:32.041041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:32.041041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:32.041041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:32.042150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:32.043265 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:32.043265 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:32.043265 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:32.043265 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:32.043265 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:32.043265 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:32.043265 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:32.043265 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:32.044692 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 431/867] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:32.051275 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37743a-7417-4dcc-bdeb-d98f6c212ed9 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:32.062871 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:32.116801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9f8ca-7be9-4010-a650-051e1a594147 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:32.161989 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37743a-7417-4dcc-bdeb-d98f6c212ed9 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:32.165500 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69ea95-9f65-41c2-bb8e-7e76c92f6218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:32.166397 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69ea95-9f65-41c2-bb8e-7e76c92f6218 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:32.166837 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69ea95-9f65-41c2-bb8e-7e76c92f6218 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:32.167262 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69ea95-9f65-41c2-bb8e-7e76c92f6218 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:32.168121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69ea95-9f65-41c2-bb8e-7e76c92f6218 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:32.168583 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69ea95-9f65-41c2-bb8e-7e76c92f6218 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:32.170644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69ea95-9f65-41c2-bb8e-7e76c92f6218 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:32.177921 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 437/868] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:37:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:32.204473 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6880a01-a51d-4839-8053-4cb4e2234d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:32.205263 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6880a01-a51d-4839-8053-4cb4e2234d7f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:32.205545 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6880a01-a51d-4839-8053-4cb4e2234d7f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:32.205822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6880a01-a51d-4839-8053-4cb4e2234d7f None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:32.206804 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6880a01-a51d-4839-8053-4cb4e2234d7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:32.207116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6880a01-a51d-4839-8053-4cb4e2234d7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:32.208674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6880a01-a51d-4839-8053-4cb4e2234d7f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:32.229411 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 432/869] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:32.230868 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69ea95-9f65-41c2-bb8e-7e76c92f6218 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:32.349619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69ea95-9f65-41c2-bb8e-7e76c92f6218 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:32.368606 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:32.389644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d7428-e165-4a4d-b25e-c4d9031e9a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:32.390255 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d7428-e165-4a4d-b25e-c4d9031e9a42 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:32.390557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d7428-e165-4a4d-b25e-c4d9031e9a42 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:32.390827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d7428-e165-4a4d-b25e-c4d9031e9a42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:32.391737 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d7428-e165-4a4d-b25e-c4d9031e9a42 None admin] RBAC: Adding query filter params (role.id=cefdc5c73ec347438e5b4687f191382c, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=9e05338c14d642f3b31095f85e1e6ca4) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:32.392043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d7428-e165-4a4d-b25e-c4d9031e9a42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:32.393489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d7428-e165-4a4d-b25e-c4d9031e9a42 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:32.400143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 438/870] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:32 2022] GET /identity/v3/role_assignments?user.id=9e05338c14d642f3b31095f85e1e6ca4&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=cefdc5c73ec347438e5b4687f191382c => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:33.171751 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d5d51-3bb4-4f00-9bc0-e49517561836 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:33.172281 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d5d51-3bb4-4f00-9bc0-e49517561836 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:33.172281 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d5d51-3bb4-4f00-9bc0-e49517561836 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:33.174502 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 433/871] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:33.180080 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03192c18-1fb1-4e6c-9d6c-ae9eb36c8c17 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:33.180912 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03192c18-1fb1-4e6c-9d6c-ae9eb36c8c17 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:33.181383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03192c18-1fb1-4e6c-9d6c-ae9eb36c8c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:33.198893 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-03192c18-1fb1-4e6c-9d6c-ae9eb36c8c17 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:33.200402 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-03192c18-1fb1-4e6c-9d6c-ae9eb36c8c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:33.268291 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 439/872] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:37:33.290903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d5d51-3bb4-4f00-9bc0-e49517561836 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:33.319765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:33.404362 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d5d51-3bb4-4f00-9bc0-e49517561836 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:33.442870 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebab755-16b2-4a5d-a245-bfe282d5fc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:33.443897 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebab755-16b2-4a5d-a245-bfe282d5fc1d None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:33.444390 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebab755-16b2-4a5d-a245-bfe282d5fc1d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:33.444874 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebab755-16b2-4a5d-a245-bfe282d5fc1d None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:33.445807 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebab755-16b2-4a5d-a245-bfe282d5fc1d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:33.447529 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebab755-16b2-4a5d-a245-bfe282d5fc1d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:33.465027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 434/873] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:37:33 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:37:33.796852 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46c150-14af-40ae-99ec-ac45c03a6d7e None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:33.797228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46c150-14af-40ae-99ec-ac45c03a6d7e None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:33.797430 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46c150-14af-40ae-99ec-ac45c03a6d7e None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:33.799311 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 440/874] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:33.804781 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9806c-75fc-422d-912f-0113a22372cd None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:33.805068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9806c-75fc-422d-912f-0113a22372cd None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:33.805342 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9806c-75fc-422d-912f-0113a22372cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:33.822475 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-c7e9806c-75fc-422d-912f-0113a22372cd None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:33.823646 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-c7e9806c-75fc-422d-912f-0113a22372cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:33.902639 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 435/875] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:33.924826 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46c150-14af-40ae-99ec-ac45c03a6d7e None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:34.027907 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46c150-14af-40ae-99ec-ac45c03a6d7e None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:34.054823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:34.062839 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff78cf-d58a-4a3d-aef9-52af4d32ab0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:34.064081 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff78cf-d58a-4a3d-aef9-52af4d32ab0a None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:34.064402 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff78cf-d58a-4a3d-aef9-52af4d32ab0a None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:34.064844 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff78cf-d58a-4a3d-aef9-52af4d32ab0a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:34.065774 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff78cf-d58a-4a3d-aef9-52af4d32ab0a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:34.067191 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff78cf-d58a-4a3d-aef9-52af4d32ab0a None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-e3ff78cf-d58a-4a3d-aef9-52af4d32ab0a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:34.079874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:34.080886 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:34.082146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:34.082146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:34.082146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:34.082146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:34.082146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:34.082146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:34.082146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:34.082146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:34.082146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:34.082146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:34.082146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:34.082146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:34.082146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:37:34.082146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:34.082146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 441/876] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:37:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:34.087338 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9806c-75fc-422d-912f-0113a22372cd None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:34.098021 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:34.189867 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9806c-75fc-422d-912f-0113a22372cd None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:34.224329 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5fc07-e584-4773-b9a5-4a783015baea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:34.224998 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5fc07-e584-4773-b9a5-4a783015baea None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:34.225270 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5fc07-e584-4773-b9a5-4a783015baea None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:34.225534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5fc07-e584-4773-b9a5-4a783015baea None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:34.226584 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5fc07-e584-4773-b9a5-4a783015baea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:34.226884 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5fc07-e584-4773-b9a5-4a783015baea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:34.228259 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5fc07-e584-4773-b9a5-4a783015baea None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:34.233652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 436/877] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:37:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:34.239697 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff78cf-d58a-4a3d-aef9-52af4d32ab0a None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:34.345480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff78cf-d58a-4a3d-aef9-52af4d32ab0a None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:34.380767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fff6c-6417-47c2-ae13-7e5453903199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:34.381427 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fff6c-6417-47c2-ae13-7e5453903199 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:34.381952 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fff6c-6417-47c2-ae13-7e5453903199 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:34.382227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fff6c-6417-47c2-ae13-7e5453903199 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:34.383052 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fff6c-6417-47c2-ae13-7e5453903199 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:34.383353 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fff6c-6417-47c2-ae13-7e5453903199 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:34.384731 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fff6c-6417-47c2-ae13-7e5453903199 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:34.389633 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 442/878] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:37:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:34.435960 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe835ce5-f960-4bf4-b6e9-a868b7129a27 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:34.436306 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe835ce5-f960-4bf4-b6e9-a868b7129a27 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:34.436505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe835ce5-f960-4bf4-b6e9-a868b7129a27 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:34.438285 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 437/879] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:34.443893 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ba206-22df-4dd4-b81c-952f819dbd12 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:34.444165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ba206-22df-4dd4-b81c-952f819dbd12 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:34.444438 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ba206-22df-4dd4-b81c-952f819dbd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:34.462951 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-f22ba206-22df-4dd4-b81c-952f819dbd12 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:34.464145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-f22ba206-22df-4dd4-b81c-952f819dbd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:34.498249 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:34.539807 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 443/880] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:37:34.562154 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe835ce5-f960-4bf4-b6e9-a868b7129a27 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:34.650239 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:34.682737 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe835ce5-f960-4bf4-b6e9-a868b7129a27 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:34.721866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec38a4-6774-4a90-94ca-e73bd0084364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:34.722627 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec38a4-6774-4a90-94ca-e73bd0084364 None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:34.722908 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec38a4-6774-4a90-94ca-e73bd0084364 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:34.723181 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec38a4-6774-4a90-94ca-e73bd0084364 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:34.723923 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec38a4-6774-4a90-94ca-e73bd0084364 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:34.725312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec38a4-6774-4a90-94ca-e73bd0084364 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:34.743525 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 438/881] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:37:34 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:37:34.909051 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205d59b-9784-47c5-b820-c5798e6d6b3a None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:34.909712 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205d59b-9784-47c5-b820-c5798e6d6b3a None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:34.910206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205d59b-9784-47c5-b820-c5798e6d6b3a None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:34.912231 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 444/882] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:34.918248 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b91cae-0ac3-409e-8b04-b0ba4083b549 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:34.918854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b91cae-0ac3-409e-8b04-b0ba4083b549 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:34.919351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b91cae-0ac3-409e-8b04-b0ba4083b549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:34.937784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-48b91cae-0ac3-409e-8b04-b0ba4083b549 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:34.939299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-48b91cae-0ac3-409e-8b04-b0ba4083b549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:35.052692 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 439/883] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:35.074075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205d59b-9784-47c5-b820-c5798e6d6b3a None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:35.183992 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205d59b-9784-47c5-b820-c5798e6d6b3a None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:35.219035 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503866a-2c0d-4360-9dd0-8023d4822d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:35.219975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503866a-2c0d-4360-9dd0-8023d4822d39 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:35.220287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503866a-2c0d-4360-9dd0-8023d4822d39 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:35.220603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503866a-2c0d-4360-9dd0-8023d4822d39 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:35.231323 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503866a-2c0d-4360-9dd0-8023d4822d39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-3503866a-2c0d-4360-9dd0-8023d4822d39 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:35.239730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:35.241092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:35.241933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:35.241933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:35.241933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:35.241933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:35.241933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:35.241933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:35.241933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:35.241933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:35.241933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:35.241933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:35.241933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:37:35.241933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:35.241933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 445/884] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 19:37:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:35.247073 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b91cae-0ac3-409e-8b04-b0ba4083b549 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:35.311846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:35.358492 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b91cae-0ac3-409e-8b04-b0ba4083b549 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:35.395567 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a96a6b-b345-471e-acaa-afebada74932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:35.396245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a96a6b-b345-471e-acaa-afebada74932 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:35.396526 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a96a6b-b345-471e-acaa-afebada74932 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:35.396800 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a96a6b-b345-471e-acaa-afebada74932 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:35.397550 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a96a6b-b345-471e-acaa-afebada74932 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:35.397857 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a96a6b-b345-471e-acaa-afebada74932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:35.399275 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a96a6b-b345-471e-acaa-afebada74932 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:35.413396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 440/885] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 19:37:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:35.415347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503866a-2c0d-4360-9dd0-8023d4822d39 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:35.446895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:35.521832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503866a-2c0d-4360-9dd0-8023d4822d39 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:35.558793 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9bbe3-3325-40de-9701-3897841a7131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:35.559625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9bbe3-3325-40de-9701-3897841a7131 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:35.559905 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9bbe3-3325-40de-9701-3897841a7131 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:35.560182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9bbe3-3325-40de-9701-3897841a7131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:35.573634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9bbe3-3325-40de-9701-3897841a7131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:35.575167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9bbe3-3325-40de-9701-3897841a7131 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:35.594709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-27f9bbe3-3325-40de-9701-3897841a7131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:35.594709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:35.594709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:35.594709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:35.594709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:35.594709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:35.594709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:35.594709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:35.594709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:35.594709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:35.594709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:35.594709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:35.594709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:35.594709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:35.594709 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:35.596803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:35.598413 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:35.598413 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:35.598413 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:35.598413 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:35.598413 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 446/886] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:35.602056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a96a6b-b345-471e-acaa-afebada74932 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:35.723713 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a96a6b-b345-471e-acaa-afebada74932 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:35.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:35.747972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c597564-db02-437d-92e2-cc3a508173e4 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:35.748257 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c597564-db02-437d-92e2-cc3a508173e4 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:35.748535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c597564-db02-437d-92e2-cc3a508173e4 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:35.750153 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 447/887] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:37:35.755294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87e02a-6a61-4fb2-9b89-9eaea639acd9 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:35.755589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87e02a-6a61-4fb2-9b89-9eaea639acd9 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:35.755856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87e02a-6a61-4fb2-9b89-9eaea639acd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:35.761421 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446c787-f266-4ed7-b425-4856c36e17df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:35.762204 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446c787-f266-4ed7-b425-4856c36e17df None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:35.762542 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446c787-f266-4ed7-b425-4856c36e17df None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:35.762824 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446c787-f266-4ed7-b425-4856c36e17df None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:35.763552 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446c787-f266-4ed7-b425-4856c36e17df None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:35.763860 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446c787-f266-4ed7-b425-4856c36e17df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:35.765240 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446c787-f266-4ed7-b425-4856c36e17df None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:35.775452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-9e87e02a-6a61-4fb2-9b89-9eaea639acd9 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:35.777361 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-9e87e02a-6a61-4fb2-9b89-9eaea639acd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:35.784757 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 441/888] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:35.786136 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446c787-f266-4ed7-b425-4856c36e17df None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:35.866315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 448/889] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:37:35.886052 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87e02a-6a61-4fb2-9b89-9eaea639acd9 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:35.899032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:35.905053 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446c787-f266-4ed7-b425-4856c36e17df None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:35.943431 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fab98-d4a7-43cf-9c4c-4cc757baf43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:35.944541 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fab98-d4a7-43cf-9c4c-4cc757baf43f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:35.944826 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fab98-d4a7-43cf-9c4c-4cc757baf43f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:35.945101 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fab98-d4a7-43cf-9c4c-4cc757baf43f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:35.953976 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591fab98-d4a7-43cf-9c4c-4cc757baf43f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:35.956090 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591fab98-d4a7-43cf-9c4c-4cc757baf43f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-591fab98-d4a7-43cf-9c4c-4cc757baf43f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:35.974201 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:35.975279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:35.976536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:35.976536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:35.976536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:35.976536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:35.976536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:35.976536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:35.976536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:35.976536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:35.976536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:37:35.976536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:35.979240 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 442/890] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:37:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:35.990294 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fab98-d4a7-43cf-9c4c-4cc757baf43f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:36.012488 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87e02a-6a61-4fb2-9b89-9eaea639acd9 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:36.045503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:36.049572 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f4be2-1a8f-4881-ae71-1040adb5a12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:36.050415 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f4be2-1a8f-4881-ae71-1040adb5a12f None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:36.050688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f4be2-1a8f-4881-ae71-1040adb5a12f None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:36.050970 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f4be2-1a8f-4881-ae71-1040adb5a12f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:36.051602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f4be2-1a8f-4881-ae71-1040adb5a12f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:36.052978 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f4be2-1a8f-4881-ae71-1040adb5a12f None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-173f4be2-1a8f-4881-ae71-1040adb5a12f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:36.065162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:36.066332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:36.067146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:36.067146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:36.067146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:36.067146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:36.067146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:36.067146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:36.067146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:36.067146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:36.067146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:36.067146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:36.067146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:36.067146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:36.067146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:37:36.067146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:36.067146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 449/891] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:37:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:36.072314 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f4be2-1a8f-4881-ae71-1040adb5a12f None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:36.127562 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fab98-d4a7-43cf-9c4c-4cc757baf43f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:36.154190 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:36.167357 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f85ac4-352f-409b-9cc7-717d89fc70a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:36.168497 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f85ac4-352f-409b-9cc7-717d89fc70a5 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:36.168838 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f85ac4-352f-409b-9cc7-717d89fc70a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:36.169171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f85ac4-352f-409b-9cc7-717d89fc70a5 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:36.170073 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f85ac4-352f-409b-9cc7-717d89fc70a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:36.170480 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f85ac4-352f-409b-9cc7-717d89fc70a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:36.172839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f85ac4-352f-409b-9cc7-717d89fc70a5 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:36.187460 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f4be2-1a8f-4881-ae71-1040adb5a12f None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:36.197277 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 443/892] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 19:37:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:36.199228 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f85ac4-352f-409b-9cc7-717d89fc70a5 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:36.226731 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d7b8f-11d9-41a8-bc92-bec82b7d1996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:36.227511 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d7b8f-11d9-41a8-bc92-bec82b7d1996 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:36.227877 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d7b8f-11d9-41a8-bc92-bec82b7d1996 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:36.228239 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d7b8f-11d9-41a8-bc92-bec82b7d1996 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:36.229034 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d7b8f-11d9-41a8-bc92-bec82b7d1996 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:36.229428 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d7b8f-11d9-41a8-bc92-bec82b7d1996 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:36.230959 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d7b8f-11d9-41a8-bc92-bec82b7d1996 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:36.241728 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 450/893] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:37:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:36.242756 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d7b8f-11d9-41a8-bc92-bec82b7d1996 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:36.320565 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:36.326682 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f85ac4-352f-409b-9cc7-717d89fc70a5 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:36.351476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d7b8f-11d9-41a8-bc92-bec82b7d1996 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:36.373903 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d55c4-5e6c-4049-8663-6eaf3c691011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:36.374581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d55c4-5e6c-4049-8663-6eaf3c691011 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:36.374855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d55c4-5e6c-4049-8663-6eaf3c691011 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:36.375123 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d55c4-5e6c-4049-8663-6eaf3c691011 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:36.389349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa77c08-fb70-4c24-be71-197d97bf141a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:36.390359 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa77c08-fb70-4c24-be71-197d97bf141a None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:36.390847 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa77c08-fb70-4c24-be71-197d97bf141a None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:36.391369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa77c08-fb70-4c24-be71-197d97bf141a None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:36.392263 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa77c08-fb70-4c24-be71-197d97bf141a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:36.392772 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa77c08-fb70-4c24-be71-197d97bf141a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:36.394369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa77c08-fb70-4c24-be71-197d97bf141a None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:36.400094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 451/894] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:37:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:36.405703 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa77c08-fb70-4c24-be71-197d97bf141a None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:36.409219 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:36.454729 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d55c4-5e6c-4049-8663-6eaf3c691011 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:36.456188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d55c4-5e6c-4049-8663-6eaf3c691011 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:36.519540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa77c08-fb70-4c24-be71-197d97bf141a None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:36.555868 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075c5ee-89e1-4f08-94b6-1c50616772e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:36.556906 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075c5ee-89e1-4f08-94b6-1c50616772e3 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:36.557400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075c5ee-89e1-4f08-94b6-1c50616772e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:36.557869 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075c5ee-89e1-4f08-94b6-1c50616772e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:36.558961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075c5ee-89e1-4f08-94b6-1c50616772e3 None admin] RBAC: Adding query filter params (interface=public, service_id=6d85a773d5ea4afaaeea36f380c459ee, region_id=RegionOne) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:36.559487 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075c5ee-89e1-4f08-94b6-1c50616772e3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:36.561072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075c5ee-89e1-4f08-94b6-1c50616772e3 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:36.575215 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 452/895] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:37:36 2022] GET /identity/v3/endpoints?service_id=6d85a773d5ea4afaaeea36f380c459ee&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:36.576403 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075c5ee-89e1-4f08-94b6-1c50616772e3 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:36.597429 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:36.606153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 444/896] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:37:36 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:37:36.620564 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d55c4-5e6c-4049-8663-6eaf3c691011 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:36.682795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:36.728315 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d55c4-5e6c-4049-8663-6eaf3c691011 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:36.731636 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075c5ee-89e1-4f08-94b6-1c50616772e3 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:36.765101 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9339ce-9bd6-4dd3-96d9-49887bf92e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:36.765847 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9339ce-9bd6-4dd3-96d9-49887bf92e6a None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:36.766123 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9339ce-9bd6-4dd3-96d9-49887bf92e6a None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:36.766581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9339ce-9bd6-4dd3-96d9-49887bf92e6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:36.776819 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23748b6b-ce76-433d-93e0-6185cb4a030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:36.778009 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23748b6b-ce76-433d-93e0-6185cb4a030f None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:36.778119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9339ce-9bd6-4dd3-96d9-49887bf92e6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:36.778607 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23748b6b-ce76-433d-93e0-6185cb4a030f None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:36.779057 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23748b6b-ce76-433d-93e0-6185cb4a030f None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:36.779601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9339ce-9bd6-4dd3-96d9-49887bf92e6a None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:36.780130 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23748b6b-ce76-433d-93e0-6185cb4a030f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:36.781979 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23748b6b-ce76-433d-93e0-6185cb4a030f None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:36.788889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 453/897] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:37:36 2022] GET /identity/v3/services => generated 1268 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:37:36.795723 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24112d5-4510-4da0-a19d-5c345589b90d None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:36.796235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24112d5-4510-4da0-a19d-5c345589b90d None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:36.796651 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24112d5-4510-4da0-a19d-5c345589b90d None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:36.798764 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 454/898] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:36.799939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-3d9339ce-9bd6-4dd3-96d9-49887bf92e6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:36.799939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:36.799939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:36.799939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:36.799939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:36.799939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:36.799939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:36.799939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:36.799939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:36.799939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:36.799939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:36.799939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:36.799939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:36.799939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:36.799939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:36.800805 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:36.801691 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:36.801691 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:36.801691 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:36.801691 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:36.803734 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 445/899] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:36.805525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151bc31-a1f4-4ba2-a6d9-af47585435e3 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:36.805998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151bc31-a1f4-4ba2-a6d9-af47585435e3 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:36.806475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151bc31-a1f4-4ba2-a6d9-af47585435e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:36.811617 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9339ce-9bd6-4dd3-96d9-49887bf92e6a None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:36.832802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-d151bc31-a1f4-4ba2-a6d9-af47585435e3 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:36.834409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-d151bc31-a1f4-4ba2-a6d9-af47585435e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:36.872288 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:36.926340 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 455/900] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:36.944317 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9339ce-9bd6-4dd3-96d9-49887bf92e6a None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:36.946013 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151bc31-a1f4-4ba2-a6d9-af47585435e3 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:36.995958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f3ff5-a1f8-4162-a3e9-0de1365ce3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:36.996888 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f3ff5-a1f8-4162-a3e9-0de1365ce3b9 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:36.997333 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f3ff5-a1f8-4162-a3e9-0de1365ce3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:36.997745 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f3ff5-a1f8-4162-a3e9-0de1365ce3b9 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:36.998644 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f3ff5-a1f8-4162-a3e9-0de1365ce3b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:36.999101 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f3ff5-a1f8-4162-a3e9-0de1365ce3b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:37.001494 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f3ff5-a1f8-4162-a3e9-0de1365ce3b9 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:37.016906 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 446/901] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:37.023128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f3ff5-a1f8-4162-a3e9-0de1365ce3b9 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:37.030415 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:37.086573 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151bc31-a1f4-4ba2-a6d9-af47585435e3 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:37.131067 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bab06b-0f83-43f4-b9c3-240bcc042dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:37.132005 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bab06b-0f83-43f4-b9c3-240bcc042dbd None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:37.132288 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bab06b-0f83-43f4-b9c3-240bcc042dbd None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:37.132563 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bab06b-0f83-43f4-b9c3-240bcc042dbd None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:37.133668 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bab06b-0f83-43f4-b9c3-240bcc042dbd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:37.135734 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bab06b-0f83-43f4-b9c3-240bcc042dbd None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:37.140896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f3ff5-a1f8-4162-a3e9-0de1365ce3b9 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:37.153364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:37.154423 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 456/902] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:37:36 2022] POST /identity/v3/services => generated 241 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:37:37.156007 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52860253-de6c-4478-8885-2a053c14b864 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:37.156399 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52860253-de6c-4478-8885-2a053c14b864 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:37.156832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52860253-de6c-4478-8885-2a053c14b864 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:37.158360 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 457/903] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:37.163807 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc345586-d93a-4631-be75-75111469555f None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:37.164151 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc345586-d93a-4631-be75-75111469555f None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:37.164499 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc345586-d93a-4631-be75-75111469555f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:37.185006 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-dc345586-d93a-4631-be75-75111469555f None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:37.186604 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-dc345586-d93a-4631-be75-75111469555f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:37.196374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5c155-b10f-44e5-85b5-7bbc5b2d2e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:37.197313 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5c155-b10f-44e5-85b5-7bbc5b2d2e77 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:37.197658 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5c155-b10f-44e5-85b5-7bbc5b2d2e77 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:37.197995 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5c155-b10f-44e5-85b5-7bbc5b2d2e77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:37.206033 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5c155-b10f-44e5-85b5-7bbc5b2d2e77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:37.207656 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5c155-b10f-44e5-85b5-7bbc5b2d2e77 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:37.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-55b5c155-b10f-44e5-85b5-7bbc5b2d2e77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:37.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:37.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:37.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:37.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:37.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:37.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:37.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:37.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:37.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:37.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:37.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:37.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:37.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:37.223563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:37.224501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:37.225374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:37.225374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:37.225374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:37.225374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:37.225374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:37.225374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:37.225374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:37.225374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:37.225374 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 447/904] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:37.231529 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5c155-b10f-44e5-85b5-7bbc5b2d2e77 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:37.281130 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 458/905] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:37:37.303504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc345586-d93a-4631-be75-75111469555f None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:37.370063 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5c155-b10f-44e5-85b5-7bbc5b2d2e77 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:37.375769 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:37.413658 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29464f00-8353-4468-8964-80b475ec8f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:37.416135 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29464f00-8353-4468-8964-80b475ec8f9a None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:37.416135 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29464f00-8353-4468-8964-80b475ec8f9a None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:37.416135 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29464f00-8353-4468-8964-80b475ec8f9a None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:37.416135 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29464f00-8353-4468-8964-80b475ec8f9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:37.416135 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29464f00-8353-4468-8964-80b475ec8f9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:37.417651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29464f00-8353-4468-8964-80b475ec8f9a None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:37.431545 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 448/906] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:37.437347 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29464f00-8353-4468-8964-80b475ec8f9a None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:37.446414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc345586-d93a-4631-be75-75111469555f None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:37.467628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:37.487509 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5c415-8102-4ec1-b87c-124a63f113d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:37.488664 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5c415-8102-4ec1-b87c-124a63f113d1 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:37.489168 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5c415-8102-4ec1-b87c-124a63f113d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:37.489646 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5c415-8102-4ec1-b87c-124a63f113d1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:37.490525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5c415-8102-4ec1-b87c-124a63f113d1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:37.492106 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5c415-8102-4ec1-b87c-124a63f113d1 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-0bc5c415-8102-4ec1-b87c-124a63f113d1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:37.505519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:37.506540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:37.507392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:37.507392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:37.507392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:37.507392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:37.507392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:37.507392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:37.507392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:37.507392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:37.507392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:37.507392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:37.507392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:37.507392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:37.507392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:37.507392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:37:37.507392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:37.510432 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 459/907] 10.222.0.84 () {56 vars in 1132 bytes} [Thu May 19 19:37:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:37.518435 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5c415-8102-4ec1-b87c-124a63f113d1 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:37.541545 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29464f00-8353-4468-8964-80b475ec8f9a None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:37.582399 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3b25c-9a70-442d-accc-71086413a810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:37.583847 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3b25c-9a70-442d-accc-71086413a810 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:37.584542 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3b25c-9a70-442d-accc-71086413a810 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:37.584962 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3b25c-9a70-442d-accc-71086413a810 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:37.586210 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3b25c-9a70-442d-accc-71086413a810 None admin] RBAC: Adding query filter params (role.id=de6fe15685d647c69e91e3772074a8b1, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=9e05338c14d642f3b31095f85e1e6ca4) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:37.587098 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3b25c-9a70-442d-accc-71086413a810 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:37.589210 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3b25c-9a70-442d-accc-71086413a810 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:37.597889 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 449/908] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:37 2022] GET /identity/v3/role_assignments?user.id=9e05338c14d642f3b31095f85e1e6ca4&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=de6fe15685d647c69e91e3772074a8b1 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:37.651951 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5c415-8102-4ec1-b87c-124a63f113d1 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:37.668820 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:37.692410 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4165b1-52d4-4bed-9838-bb12fb6e2eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:37.693438 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4165b1-52d4-4bed-9838-bb12fb6e2eb8 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:37.693860 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4165b1-52d4-4bed-9838-bb12fb6e2eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:37.694370 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4165b1-52d4-4bed-9838-bb12fb6e2eb8 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:37.695273 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4165b1-52d4-4bed-9838-bb12fb6e2eb8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:37.695725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4165b1-52d4-4bed-9838-bb12fb6e2eb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:37.697291 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4165b1-52d4-4bed-9838-bb12fb6e2eb8 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:37.703820 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 460/909] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:37:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:37.709983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3b25c-9a70-442d-accc-71086413a810 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:37.815398 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:37.821861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3b25c-9a70-442d-accc-71086413a810 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:37.857400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4bb00-bad7-45ed-ba93-1d50f9f27e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:37.858069 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4bb00-bad7-45ed-ba93-1d50f9f27e80 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:37.858373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4bb00-bad7-45ed-ba93-1d50f9f27e80 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:37.858647 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4bb00-bad7-45ed-ba93-1d50f9f27e80 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:37.859347 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4bb00-bad7-45ed-ba93-1d50f9f27e80 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:37.859649 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4bb00-bad7-45ed-ba93-1d50f9f27e80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:37.861086 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4bb00-bad7-45ed-ba93-1d50f9f27e80 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:37.866284 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 450/910] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:37:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:37.872142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4165b1-52d4-4bed-9838-bb12fb6e2eb8 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:37.973457 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4165b1-52d4-4bed-9838-bb12fb6e2eb8 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:38.008202 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c52a4-d18b-4ad5-9672-eed1db9bf49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:38.008883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c52a4-d18b-4ad5-9672-eed1db9bf49b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:38.009159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c52a4-d18b-4ad5-9672-eed1db9bf49b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:38.009426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c52a4-d18b-4ad5-9672-eed1db9bf49b None admin] PATH_INFO: `/v3/endpoints` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:38.010239 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c52a4-d18b-4ad5-9672-eed1db9bf49b None admin] RBAC: Adding query filter params (interface=public, service_id=fd631d24fe044ec3897b9e7ce661ad66, region_id=RegionOne) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:38.010591 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c52a4-d18b-4ad5-9672-eed1db9bf49b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:38.012039 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c52a4-d18b-4ad5-9672-eed1db9bf49b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:38.018766 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 461/911] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:37:37 2022] GET /identity/v3/endpoints?service_id=fd631d24fe044ec3897b9e7ce661ad66&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:38.024425 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4bb00-bad7-45ed-ba93-1d50f9f27e80 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:38.127302 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4bb00-bad7-45ed-ba93-1d50f9f27e80 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:38.162543 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984428b-ee12-4dac-a2ab-78b2d77fa9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:38.163215 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984428b-ee12-4dac-a2ab-78b2d77fa9f1 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:38.163493 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984428b-ee12-4dac-a2ab-78b2d77fa9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:38.163764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984428b-ee12-4dac-a2ab-78b2d77fa9f1 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:38.164444 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984428b-ee12-4dac-a2ab-78b2d77fa9f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:38.165820 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984428b-ee12-4dac-a2ab-78b2d77fa9f1 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:38.172291 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 451/912] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:37:38 2022] GET /identity/v3/services => generated 1497 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:37:39.173534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378df8c-e567-4c82-97a8-938e2b73b063 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:39.174073 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378df8c-e567-4c82-97a8-938e2b73b063 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:39.174073 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378df8c-e567-4c82-97a8-938e2b73b063 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:39.175751 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 462/913] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:39.181401 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e05ba7-ca84-4ebd-96fe-27ae53d2305d None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:39.181806 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e05ba7-ca84-4ebd-96fe-27ae53d2305d None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:39.181986 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e05ba7-ca84-4ebd-96fe-27ae53d2305d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:39.199624 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-02e05ba7-ca84-4ebd-96fe-27ae53d2305d None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:39.200817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-02e05ba7-ca84-4ebd-96fe-27ae53d2305d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:39.215622 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:39.279312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 452/914] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:39.303691 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378df8c-e567-4c82-97a8-938e2b73b063 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:39.350554 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:39.411978 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378df8c-e567-4c82-97a8-938e2b73b063 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:39.435887 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25ec08-172c-4ebb-930e-a88a50ca6434 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:39.436075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25ec08-172c-4ebb-930e-a88a50ca6434 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:39.436343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25ec08-172c-4ebb-930e-a88a50ca6434 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:39.438076 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 453/915] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:39.443563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78509184-7c69-4e3a-b105-54b17475631e None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:39.443858 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78509184-7c69-4e3a-b105-54b17475631e None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:39.444111 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78509184-7c69-4e3a-b105-54b17475631e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:39.447794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5533c-0359-43e9-b73c-2c81fd98736a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:39.448603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5533c-0359-43e9-b73c-2c81fd98736a None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:39.448878 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5533c-0359-43e9-b73c-2c81fd98736a None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:39.449213 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5533c-0359-43e9-b73c-2c81fd98736a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:39.449797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5533c-0359-43e9-b73c-2c81fd98736a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:39.451251 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5533c-0359-43e9-b73c-2c81fd98736a None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:39.461622 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-78509184-7c69-4e3a-b105-54b17475631e None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:39.462947 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-78509184-7c69-4e3a-b105-54b17475631e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-51c5533c-0359-43e9-b73c-2c81fd98736a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:39.467101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:39.468074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:39.468889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:39.468889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:39.468889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:39.468889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:39.468889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:39.468889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:39.468889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:39.468889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:39.468889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:39.468889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:39.468889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:39.468889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:39.468889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:37:39.468889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:39.471713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 463/916] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:37:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:39.477626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5533c-0359-43e9-b73c-2c81fd98736a None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:39.543296 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 454/917] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:39.565662 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78509184-7c69-4e3a-b105-54b17475631e None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:39.602386 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5533c-0359-43e9-b73c-2c81fd98736a None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:39.611590 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:39.649608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b3f75-4b98-4724-8be2-782a43e9e0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:39.650295 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b3f75-4b98-4724-8be2-782a43e9e0bd None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:39.650585 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b3f75-4b98-4724-8be2-782a43e9e0bd None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:39.650861 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b3f75-4b98-4724-8be2-782a43e9e0bd None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:39.651730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b3f75-4b98-4724-8be2-782a43e9e0bd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:39.652029 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b3f75-4b98-4724-8be2-782a43e9e0bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:39.654016 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b3f75-4b98-4724-8be2-782a43e9e0bd None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:39.662375 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 464/918] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:37:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:39.667090 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b3f75-4b98-4724-8be2-782a43e9e0bd None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:39.677194 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:39.701679 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78509184-7c69-4e3a-b105-54b17475631e None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:39.745107 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5f83f-76d2-4ae5-b820-dc9163032204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:39.746370 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5f83f-76d2-4ae5-b820-dc9163032204 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:39.746902 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5f83f-76d2-4ae5-b820-dc9163032204 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:39.747406 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5f83f-76d2-4ae5-b820-dc9163032204 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:39.748249 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5f83f-76d2-4ae5-b820-dc9163032204 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:39.749808 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5f83f-76d2-4ae5-b820-dc9163032204 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-99d5f83f-76d2-4ae5-b820-dc9163032204 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:39.762715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:39.763772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:39.764589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:39.764589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:39.764589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:39.764589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:39.764589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:39.764589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:39.764589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:39.764589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:39.764589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:39.764589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:39.764589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:39.764589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:39.764589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:37:39.764589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:39.772868 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 455/919] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:37:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:39.773993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5f83f-76d2-4ae5-b820-dc9163032204 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:39.799468 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b3f75-4b98-4724-8be2-782a43e9e0bd None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:39.843928 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2762097-b09d-4b1b-b767-abb112e8ae22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:39.844721 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2762097-b09d-4b1b-b767-abb112e8ae22 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:39.845003 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2762097-b09d-4b1b-b767-abb112e8ae22 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:39.845277 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2762097-b09d-4b1b-b767-abb112e8ae22 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:39.845991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2762097-b09d-4b1b-b767-abb112e8ae22 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:39.846314 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2762097-b09d-4b1b-b767-abb112e8ae22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:39.848676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2762097-b09d-4b1b-b767-abb112e8ae22 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:39.858346 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 465/920] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:37:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:39.860978 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2762097-b09d-4b1b-b767-abb112e8ae22 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:39.891146 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5f83f-76d2-4ae5-b820-dc9163032204 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:39.898939 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:39.928109 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174301a5-0576-4dc1-93a7-faf3b74c4ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:39.928922 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174301a5-0576-4dc1-93a7-faf3b74c4ab2 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:39.929303 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174301a5-0576-4dc1-93a7-faf3b74c4ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:39.929579 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174301a5-0576-4dc1-93a7-faf3b74c4ab2 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:39.930303 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174301a5-0576-4dc1-93a7-faf3b74c4ab2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:39.930621 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174301a5-0576-4dc1-93a7-faf3b74c4ab2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:39.932111 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174301a5-0576-4dc1-93a7-faf3b74c4ab2 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:39.942002 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 456/921] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:37:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:39.942002 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356ed5e-9075-405d-8123-31ef6ff06238 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:39.942002 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356ed5e-9075-405d-8123-31ef6ff06238 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:39.942002 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356ed5e-9075-405d-8123-31ef6ff06238 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:39.942441 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 457/922] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:39.945971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8baa5b-4c79-424e-b0f3-7d35cf2c85c3 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:39.946249 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8baa5b-4c79-424e-b0f3-7d35cf2c85c3 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:39.946552 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8baa5b-4c79-424e-b0f3-7d35cf2c85c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:39.958934 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:39.965763 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-ab8baa5b-4c79-424e-b0f3-7d35cf2c85c3 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:39.966960 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-ab8baa5b-4c79-424e-b0f3-7d35cf2c85c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:39.969981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2762097-b09d-4b1b-b767-abb112e8ae22 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:40.004238 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac7b71-90b5-4bae-93b3-20f0efdbe4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:40.005289 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac7b71-90b5-4bae-93b3-20f0efdbe4c4 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:40.005289 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac7b71-90b5-4bae-93b3-20f0efdbe4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:40.005469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac7b71-90b5-4bae-93b3-20f0efdbe4c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:40.006183 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac7b71-90b5-4bae-93b3-20f0efdbe4c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:40.007593 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac7b71-90b5-4bae-93b3-20f0efdbe4c4 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:40.038302 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 466/923] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:37:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:37:40.038302 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac7b71-90b5-4bae-93b3-20f0efdbe4c4 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:40.057412 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 458/924] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:40.081237 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8baa5b-4c79-424e-b0f3-7d35cf2c85c3 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:40.161913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:40.177635 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac7b71-90b5-4bae-93b3-20f0efdbe4c4 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:40.201584 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8baa5b-4c79-424e-b0f3-7d35cf2c85c3 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:40.233933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a996a1-3f63-429d-81f4-c79f3a5ceff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:40.235456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a996a1-3f63-429d-81f4-c79f3a5ceff2 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:40.235986 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a996a1-3f63-429d-81f4-c79f3a5ceff2 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:40.236764 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a996a1-3f63-429d-81f4-c79f3a5ceff2 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:40.238330 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a996a1-3f63-429d-81f4-c79f3a5ceff2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:40.239103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a996a1-3f63-429d-81f4-c79f3a5ceff2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:40.241111 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a996a1-3f63-429d-81f4-c79f3a5ceff2 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:40.249061 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 467/925] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:37:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:40.254782 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a996a1-3f63-429d-81f4-c79f3a5ceff2 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:40.256034 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:40.264718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db151505-dcb2-4fd0-93aa-5a39bd0f2930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:40.265591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db151505-dcb2-4fd0-93aa-5a39bd0f2930 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:40.265841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db151505-dcb2-4fd0-93aa-5a39bd0f2930 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:40.266235 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db151505-dcb2-4fd0-93aa-5a39bd0f2930 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:40.284243 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db151505-dcb2-4fd0-93aa-5a39bd0f2930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-db151505-dcb2-4fd0-93aa-5a39bd0f2930 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:40.295740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:40.296770 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:40.297865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:40.297865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:40.297865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:40.297865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:40.297865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:40.297865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:40.297865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:40.297865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:40.297865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:40.297865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:40.297865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:37:40.297865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:40.298590 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 459/926] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 19:37:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:40.304961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db151505-dcb2-4fd0-93aa-5a39bd0f2930 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:40.394961 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a996a1-3f63-429d-81f4-c79f3a5ceff2 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:40.418761 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db151505-dcb2-4fd0-93aa-5a39bd0f2930 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:40.431174 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44f605-0dd6-4691-a350-57845a23949b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:40.431843 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44f605-0dd6-4691-a350-57845a23949b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:40.432172 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44f605-0dd6-4691-a350-57845a23949b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:40.432493 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44f605-0dd6-4691-a350-57845a23949b None admin] PATH_INFO: `/v3/endpoints` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:40.433295 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44f605-0dd6-4691-a350-57845a23949b None admin] RBAC: Adding query filter params (interface=public, service_id=cee42b6658564730a06f4202fa34f38d, region_id=RegionOne) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:40.433601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44f605-0dd6-4691-a350-57845a23949b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:40.435013 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44f605-0dd6-4691-a350-57845a23949b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:40.441234 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 468/927] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:37:40 2022] GET /identity/v3/endpoints?service_id=cee42b6658564730a06f4202fa34f38d&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:40.442687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245947d-4ea7-47b0-94c2-33c9c61c6fca None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:40.442971 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245947d-4ea7-47b0-94c2-33c9c61c6fca None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:40.443244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245947d-4ea7-47b0-94c2-33c9c61c6fca None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:40.444729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 469/928] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:37:40.446053 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245947d-4ea7-47b0-94c2-33c9c61c6fca None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:40.453932 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacdeaa-4f9a-45ed-b86f-12c96a24517b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:40.454634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacdeaa-4f9a-45ed-b86f-12c96a24517b None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:40.454914 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacdeaa-4f9a-45ed-b86f-12c96a24517b None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:40.455184 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacdeaa-4f9a-45ed-b86f-12c96a24517b None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:40.455948 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacdeaa-4f9a-45ed-b86f-12c96a24517b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:40.456219 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacdeaa-4f9a-45ed-b86f-12c96a24517b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:40.457585 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacdeaa-4f9a-45ed-b86f-12c96a24517b None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:40.470322 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 460/929] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 19:37:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:40.470322 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4b4b-5a96-4629-b071-396754359a5d None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:40.470322 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4b4b-5a96-4629-b071-396754359a5d None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:40.470322 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4b4b-5a96-4629-b071-396754359a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:40.472710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:40.484601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-91fa4b4b-5a96-4629-b071-396754359a5d None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:40.485799 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-91fa4b4b-5a96-4629-b071-396754359a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:40.537999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:40.556751 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 461/930] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:40.557664 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa4b4b-5a96-4629-b071-396754359a5d None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:40.577811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245947d-4ea7-47b0-94c2-33c9c61c6fca None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:40.618371 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ba25d-9f4d-40e0-b67e-b8f1f6ebeee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:40.619043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ba25d-9f4d-40e0-b67e-b8f1f6ebeee8 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:40.619317 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ba25d-9f4d-40e0-b67e-b8f1f6ebeee8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:40.619581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ba25d-9f4d-40e0-b67e-b8f1f6ebeee8 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:40.620346 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ba25d-9f4d-40e0-b67e-b8f1f6ebeee8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:40.621724 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ba25d-9f4d-40e0-b67e-b8f1f6ebeee8 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:40.628497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 470/931] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:37:40 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 19:37:40.628843 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ba25d-9f4d-40e0-b67e-b8f1f6ebeee8 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:40.688476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa4b4b-5a96-4629-b071-396754359a5d None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:40.729112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9046a-7caa-4961-8e4f-e4781da44982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:40.729919 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9046a-7caa-4961-8e4f-e4781da44982 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:40.730219 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9046a-7caa-4961-8e4f-e4781da44982 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:40.730531 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9046a-7caa-4961-8e4f-e4781da44982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:40.742244 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9046a-7caa-4961-8e4f-e4781da44982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:40.743672 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9046a-7caa-4961-8e4f-e4781da44982 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:40.763158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-ebf9046a-7caa-4961-8e4f-e4781da44982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:40.763158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:40.763158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:40.763158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:40.763158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:40.763158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:40.763158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:40.763158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:40.763158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:40.763158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:40.763158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:40.763158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:40.763158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:40.763158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:40.763158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:40.764116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:40.764957 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:40.764957 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:40.764957 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:40.764957 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:40.769529 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 462/932] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:40.770627 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9046a-7caa-4961-8e4f-e4781da44982 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:40.770832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:40.799655 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ba25d-9f4d-40e0-b67e-b8f1f6ebeee8 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:40.813834 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:40.840838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284079b-bbcc-4062-8498-734391804adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:40.841677 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284079b-bbcc-4062-8498-734391804adc None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:40.841953 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284079b-bbcc-4062-8498-734391804adc None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:40.842235 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284079b-bbcc-4062-8498-734391804adc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:40.843354 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284079b-bbcc-4062-8498-734391804adc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:40.844374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284079b-bbcc-4062-8498-734391804adc None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-b284079b-bbcc-4062-8498-734391804adc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:40.857730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:40.858891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:40.860248 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 471/933] 10.222.0.84 () {56 vars in 1132 bytes} [Thu May 19 19:37:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:40.867510 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284079b-bbcc-4062-8498-734391804adc None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:40.898362 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9046a-7caa-4961-8e4f-e4781da44982 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:40.943349 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b434918-092e-472f-a3b1-53b7df389043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:40.944508 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b434918-092e-472f-a3b1-53b7df389043 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:40.944928 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b434918-092e-472f-a3b1-53b7df389043 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:40.945565 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b434918-092e-472f-a3b1-53b7df389043 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:40.947014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b434918-092e-472f-a3b1-53b7df389043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:40.947331 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b434918-092e-472f-a3b1-53b7df389043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:40.948723 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b434918-092e-472f-a3b1-53b7df389043 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:40.968339 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 463/934] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:40.974306 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b434918-092e-472f-a3b1-53b7df389043 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:40.992993 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284079b-bbcc-4062-8498-734391804adc None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:41.030837 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149411e1-8755-4b80-a54c-9747b0f50fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:41.031954 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149411e1-8755-4b80-a54c-9747b0f50fe4 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:41.032374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149411e1-8755-4b80-a54c-9747b0f50fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:41.032780 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149411e1-8755-4b80-a54c-9747b0f50fe4 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:41.033653 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149411e1-8755-4b80-a54c-9747b0f50fe4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:41.034284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149411e1-8755-4b80-a54c-9747b0f50fe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:41.035983 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149411e1-8755-4b80-a54c-9747b0f50fe4 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:41.041770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 472/935] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:37:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:41.051243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149411e1-8755-4b80-a54c-9747b0f50fe4 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:41.064700 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:41.096878 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:41.105531 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b434918-092e-472f-a3b1-53b7df389043 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:41.149368 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323a420-e4c7-44dd-bf9b-cd29bce33d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:41.150904 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323a420-e4c7-44dd-bf9b-cd29bce33d8f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:41.151484 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323a420-e4c7-44dd-bf9b-cd29bce33d8f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:41.152020 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323a420-e4c7-44dd-bf9b-cd29bce33d8f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:41.160221 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149411e1-8755-4b80-a54c-9747b0f50fe4 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:41.163368 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323a420-e4c7-44dd-bf9b-cd29bce33d8f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:41.165018 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323a420-e4c7-44dd-bf9b-cd29bce33d8f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-1323a420-e4c7-44dd-bf9b-cd29bce33d8f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:41.184001 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:41.185199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:41.186384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:41.186384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:41.186384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:41.186384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:41.186384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:41.186384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:41.186384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:41.186384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:41.186384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:37:41.186384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:41.189493 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 464/936] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:37:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:41.194994 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323a420-e4c7-44dd-bf9b-cd29bce33d8f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:41.198985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc02d4c-1da3-484c-a35b-afa8683ce5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:41.199661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc02d4c-1da3-484c-a35b-afa8683ce5e7 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:41.199936 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc02d4c-1da3-484c-a35b-afa8683ce5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:41.200197 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc02d4c-1da3-484c-a35b-afa8683ce5e7 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:41.200984 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc02d4c-1da3-484c-a35b-afa8683ce5e7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:41.201285 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc02d4c-1da3-484c-a35b-afa8683ce5e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:41.202832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc02d4c-1da3-484c-a35b-afa8683ce5e7 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:41.213722 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 473/937] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:37:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:41.214712 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc02d4c-1da3-484c-a35b-afa8683ce5e7 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:41.315041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323a420-e4c7-44dd-bf9b-cd29bce33d8f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:41.319856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:41.357420 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc02d4c-1da3-484c-a35b-afa8683ce5e7 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:41.358441 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3ddd2-d789-4af0-bf69-805ec2a49d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:41.359196 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3ddd2-d789-4af0-bf69-805ec2a49d35 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:41.359537 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3ddd2-d789-4af0-bf69-805ec2a49d35 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:41.360016 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3ddd2-d789-4af0-bf69-805ec2a49d35 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:41.360864 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3ddd2-d789-4af0-bf69-805ec2a49d35 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:41.361314 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3ddd2-d789-4af0-bf69-805ec2a49d35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:41.362861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3ddd2-d789-4af0-bf69-805ec2a49d35 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:41.382128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 465/938] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 19:37:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:41.387085 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3ddd2-d789-4af0-bf69-805ec2a49d35 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:41.403073 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9fac2-278b-438f-9949-5e2662dbcbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:41.404489 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:41.405472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9fac2-278b-438f-9949-5e2662dbcbe7 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:41.406293 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9fac2-278b-438f-9949-5e2662dbcbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:41.406939 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9fac2-278b-438f-9949-5e2662dbcbe7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:41.408688 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9fac2-278b-438f-9949-5e2662dbcbe7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:41.411127 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9fac2-278b-438f-9949-5e2662dbcbe7 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:41.440006 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 474/939] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:37:41 2022] POST /identity/v3/endpoints => generated 347 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:37:41.516287 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3ddd2-d789-4af0-bf69-805ec2a49d35 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:41.564437 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d3058-b0bf-4334-80c6-3409fc43672f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:41.565350 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d3058-b0bf-4334-80c6-3409fc43672f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:41.565568 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d3058-b0bf-4334-80c6-3409fc43672f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:41.565836 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d3058-b0bf-4334-80c6-3409fc43672f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:41.577713 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d3058-b0bf-4334-80c6-3409fc43672f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:41.579437 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d3058-b0bf-4334-80c6-3409fc43672f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:41.599509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-7b6d3058-b0bf-4334-80c6-3409fc43672f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:41.599509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:41.599509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:41.599509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:41.599509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:41.599509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:41.599509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:41.599509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:41.599509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:41.599509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:41.599509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:41.599509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:41.599509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:41.599509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:41.599509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:41.600476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:41.601686 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:41.601686 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:41.601686 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:41.601686 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:41.601686 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 466/940] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:41.606981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9fac2-278b-438f-9949-5e2662dbcbe7 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:41.720349 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9fac2-278b-438f-9949-5e2662dbcbe7 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:41.753066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:41.760806 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7afafb-9eb2-4f6a-ab10-18088adb982b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:41.761982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7afafb-9eb2-4f6a-ab10-18088adb982b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:41.762718 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7afafb-9eb2-4f6a-ab10-18088adb982b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:41.762986 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7afafb-9eb2-4f6a-ab10-18088adb982b None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:41.763717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7afafb-9eb2-4f6a-ab10-18088adb982b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:41.764019 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7afafb-9eb2-4f6a-ab10-18088adb982b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:41.765425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7afafb-9eb2-4f6a-ab10-18088adb982b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:41.778051 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 475/941] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:41.783455 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d3058-b0bf-4334-80c6-3409fc43672f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:41.851390 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:41.891628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d3058-b0bf-4334-80c6-3409fc43672f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:41.928626 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4563819-87ca-4045-9914-f98885392c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:41.929428 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4563819-87ca-4045-9914-f98885392c2a None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:41.929705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4563819-87ca-4045-9914-f98885392c2a None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:41.930031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4563819-87ca-4045-9914-f98885392c2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:41.937380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4563819-87ca-4045-9914-f98885392c2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:41.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4563819-87ca-4045-9914-f98885392c2a None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:41.953182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-e4563819-87ca-4045-9914-f98885392c2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:41.953182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:41.953182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:41.953182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:41.953182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:41.953182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:41.953182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:41.953182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:41.953182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:41.953182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:41.953182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:41.953182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:41.953182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:41.953182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:41.953182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:41.955577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:41.957625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:41.957625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:41.957625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:41.957625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:41.957625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:41.957625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:41.957625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:41.957625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:41.957625 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 467/942] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:41.960457 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7afafb-9eb2-4f6a-ab10-18088adb982b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:42.064698 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7afafb-9eb2-4f6a-ab10-18088adb982b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:42.098843 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7f7e3-1bf7-4cd7-8546-aeced6eb2da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:42.099823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7f7e3-1bf7-4cd7-8546-aeced6eb2da4 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:42.100303 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7f7e3-1bf7-4cd7-8546-aeced6eb2da4 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:42.100769 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7f7e3-1bf7-4cd7-8546-aeced6eb2da4 None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:42.102142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7f7e3-1bf7-4cd7-8546-aeced6eb2da4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:42.102662 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7f7e3-1bf7-4cd7-8546-aeced6eb2da4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:42.104281 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7f7e3-1bf7-4cd7-8546-aeced6eb2da4 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:42.117006 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 476/943] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:42.123037 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4563819-87ca-4045-9914-f98885392c2a None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:42.227953 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4563819-87ca-4045-9914-f98885392c2a None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:42.252735 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:42.265754 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1408bbc-733a-4b5e-9add-1e301edf107e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:42.266845 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1408bbc-733a-4b5e-9add-1e301edf107e None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:42.267123 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1408bbc-733a-4b5e-9add-1e301edf107e None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:42.267394 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1408bbc-733a-4b5e-9add-1e301edf107e None admin] PATH_INFO: `/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/9e05338c14d642f3b31095f85e1e6ca4/roles/de6fe15685d647c69e91e3772074a8b1` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:42.269773 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1408bbc-733a-4b5e-9add-1e301edf107e None admin] RBAC: Authorizing `identity:create_grant(project_id=ecf87429758641eeb462b3e409cadc9e, user_id=9e05338c14d642f3b31095f85e1e6ca4, role_id=de6fe15685d647c69e91e3772074a8b1)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:42.271277 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1408bbc-733a-4b5e-9add-1e301edf107e None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:42.287971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 468/944] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:37:42 2022] PUT /identity/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/9e05338c14d642f3b31095f85e1e6ca4/roles/de6fe15685d647c69e91e3772074a8b1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:37:42.451658 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19c7f1-a620-4dd6-b9da-f17ca0c206da None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:42.452160 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19c7f1-a620-4dd6-b9da-f17ca0c206da None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:42.452160 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19c7f1-a620-4dd6-b9da-f17ca0c206da None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:42.453966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 477/945] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:42.459803 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070cdaa-226e-44a7-9c52-f7ec2ef07d2b None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:42.460305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070cdaa-226e-44a7-9c52-f7ec2ef07d2b None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:42.460395 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070cdaa-226e-44a7-9c52-f7ec2ef07d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:42.480896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-8070cdaa-226e-44a7-9c52-f7ec2ef07d2b None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:42.482070 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-8070cdaa-226e-44a7-9c52-f7ec2ef07d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:42.555748 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 469/946] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:37:42.578995 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19c7f1-a620-4dd6-b9da-f17ca0c206da None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:42.634517 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:42.689427 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19c7f1-a620-4dd6-b9da-f17ca0c206da None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:42.726254 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee348303-f746-49e9-b6d7-203a514119e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:42.726937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee348303-f746-49e9-b6d7-203a514119e9 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:42.727213 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee348303-f746-49e9-b6d7-203a514119e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:42.727478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee348303-f746-49e9-b6d7-203a514119e9 None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:42.728193 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee348303-f746-49e9-b6d7-203a514119e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:42.729646 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee348303-f746-49e9-b6d7-203a514119e9 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:42.731119 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: WARNING keystone.server.flask.common [None req-ee348303-f746-49e9-b6d7-203a514119e9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 19:37:42.755893 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.sql.core [None req-ee348303-f746-49e9-b6d7-203a514119e9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 19:37:42.755893 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 19:37:42.755893 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [parameters: {'id': 'e9d623f4b97b45e5979d932ff3c9bc15', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 19:37:42.755893 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86981) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 19:37:42.763794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-ee348303-f746-49e9-b6d7-203a514119e9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e9d623f4b97b45e5979d932ff3c9bc15.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e9d623f4b97b45e5979d932ff3c9bc15. May 19 19:37:42.763794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:42.763794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:37:42.763794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:37:42.763794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:37:42.763794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:37:42.763794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:37:42.763794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:37:42.763794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:37:42.763794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application conn.query(q) May 19 19:37:42.763794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:37:42.763794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:37:42.763794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application result.read() May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 19:37:42.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application next(self.gen) May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application yield resource May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application next(self.gen) May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self.session.rollback() May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self.force_reraise() May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise self.value May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application session.commit() May 19 19:37:42.765516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._prepare_impl() May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self.session.flush() May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._flush(objects) May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application compat.raise_( May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application flush_context.execute() May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 19:37:42.767104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rec.execute(self) May 19 19:37:42.769188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 19:37:42.769188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 19:37:42.769188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 19:37:42.769188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 19:37:42.769188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 19:37:42.769188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 19:37:42.769188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 19:37:42.769188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 19:37:42.769188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 19:37:42.769188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 19:37:42.769188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 19:37:42.769188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 19:37:42.769188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 19:37:42.769188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 19:37:42.769188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application conn.query(q) May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application result.read() May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:37:42.770623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:37:42.771478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:37:42.771478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:37:42.771478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:37:42.771478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:37:42.771478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:37:42.771478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:37:42.771478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 19:37:42.771478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 19:37:42.771478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [parameters: {'id': 'e9d623f4b97b45e5979d932ff3c9bc15', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 19:37:42.771478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 19:37:42.771478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:42.771478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:42.771478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:42.771478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:42.771478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:42.772252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:42.773159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:42.773159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 19:37:42.773159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 19:37:42.773159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:42.773159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:42.773159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 19:37:42.773159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 19:37:42.773159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e9d623f4b97b45e5979d932ff3c9bc15. May 19 19:37:42.773159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:42.773159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 478/947] 10.222.0.84 () {60 vars in 1140 bytes} [Thu May 19 19:37:42 2022] POST /identity/v3/projects => generated 262 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:42.779269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070cdaa-226e-44a7-9c52-f7ec2ef07d2b None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:42.889312 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070cdaa-226e-44a7-9c52-f7ec2ef07d2b None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:42.903528 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:42.925815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab9034-d68a-47ba-a05e-602eb360b25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:42.926927 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab9034-d68a-47ba-a05e-602eb360b25c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:42.927355 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab9034-d68a-47ba-a05e-602eb360b25c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:42.927773 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab9034-d68a-47ba-a05e-602eb360b25c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:42.935284 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab9034-d68a-47ba-a05e-602eb360b25c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:42.936934 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab9034-d68a-47ba-a05e-602eb360b25c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:42.951092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-7cab9034-d68a-47ba-a05e-602eb360b25c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:42.951092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:42.951092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:42.951092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:42.951092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:42.951092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:42.951092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:42.951092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:42.951092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:42.951092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:42.951092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:42.951092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:42.951092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:42.951092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:42.951092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:42.952171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:42.953269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:42.953269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:42.953269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:42.953269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:42.953269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:42.953269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:42.953269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:42.953269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:42.955145 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 470/948] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:42.966311 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee348303-f746-49e9-b6d7-203a514119e9 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:42.971519 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa490f-ec36-4855-9bb9-5d8a711563f3 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:42.972285 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa490f-ec36-4855-9bb9-5d8a711563f3 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:42.972757 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa490f-ec36-4855-9bb9-5d8a711563f3 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:42.974526 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 471/949] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:42.979819 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad6dbf-7672-4986-bff7-c9a7a48ecadf None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:42.980044 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad6dbf-7672-4986-bff7-c9a7a48ecadf None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:42.980316 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad6dbf-7672-4986-bff7-c9a7a48ecadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:42.998311 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-33ad6dbf-7672-4986-bff7-c9a7a48ecadf None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:42.999474 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-33ad6dbf-7672-4986-bff7-c9a7a48ecadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:43.078908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee348303-f746-49e9-b6d7-203a514119e9 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:43.083611 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 472/950] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:37:43.092316 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:43.106857 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad6dbf-7672-4986-bff7-c9a7a48ecadf None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:43.124292 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296af6e-ccf9-45b9-b5a9-c23da3d810a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:43.126369 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296af6e-ccf9-45b9-b5a9-c23da3d810a9 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:43.126866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296af6e-ccf9-45b9-b5a9-c23da3d810a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:43.127343 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296af6e-ccf9-45b9-b5a9-c23da3d810a9 None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:43.129004 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9296af6e-ccf9-45b9-b5a9-c23da3d810a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:43.129494 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9296af6e-ccf9-45b9-b5a9-c23da3d810a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:43.131381 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9296af6e-ccf9-45b9-b5a9-c23da3d810a9 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:43.160447 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 479/951] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:37:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:43.228711 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad6dbf-7672-4986-bff7-c9a7a48ecadf None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:43.231009 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:43.268703 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb99704-fd99-4b32-849e-73144eab1fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:43.269935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb99704-fd99-4b32-849e-73144eab1fa5 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:43.270453 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb99704-fd99-4b32-849e-73144eab1fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:43.270928 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb99704-fd99-4b32-849e-73144eab1fa5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:43.271815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb99704-fd99-4b32-849e-73144eab1fa5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:43.273455 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb99704-fd99-4b32-849e-73144eab1fa5 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-adb99704-fd99-4b32-849e-73144eab1fa5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:43.287081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:43.288387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:43.289297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:43.289297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:43.289297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:43.289297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:43.289297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:43.289297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:43.289297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:43.289297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:43.289297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:43.289297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:43.289297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:43.289297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:43.289297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:37:43.289297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:43.292364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 473/952] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:37:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:43.299626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296af6e-ccf9-45b9-b5a9-c23da3d810a9 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:43.408387 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296af6e-ccf9-45b9-b5a9-c23da3d810a9 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:43.448782 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429fb57b-5c3f-4ea7-a63a-13663fac1c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:43.449453 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429fb57b-5c3f-4ea7-a63a-13663fac1c5c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:43.449725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429fb57b-5c3f-4ea7-a63a-13663fac1c5c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:43.449989 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429fb57b-5c3f-4ea7-a63a-13663fac1c5c None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:43.450742 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429fb57b-5c3f-4ea7-a63a-13663fac1c5c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:43.451039 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429fb57b-5c3f-4ea7-a63a-13663fac1c5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:43.452411 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429fb57b-5c3f-4ea7-a63a-13663fac1c5c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:43.462366 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 480/953] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:37:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:43.464820 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb99704-fd99-4b32-849e-73144eab1fa5 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:43.589223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb99704-fd99-4b32-849e-73144eab1fa5 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:43.629534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd58fa-f209-472c-bff5-6036e6f51f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:43.630226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd58fa-f209-472c-bff5-6036e6f51f5c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:43.630533 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd58fa-f209-472c-bff5-6036e6f51f5c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:43.630813 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd58fa-f209-472c-bff5-6036e6f51f5c None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:43.631596 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd58fa-f209-472c-bff5-6036e6f51f5c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:43.631905 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd58fa-f209-472c-bff5-6036e6f51f5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:43.633413 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd58fa-f209-472c-bff5-6036e6f51f5c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:43.639190 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 474/954] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:37:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:43.645145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429fb57b-5c3f-4ea7-a63a-13663fac1c5c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:43.680549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:43.755969 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429fb57b-5c3f-4ea7-a63a-13663fac1c5c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:43.790438 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632dd8f8-37bc-4356-947b-1e9e94f2551a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:43.791094 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632dd8f8-37bc-4356-947b-1e9e94f2551a None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:43.791367 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632dd8f8-37bc-4356-947b-1e9e94f2551a None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:43.791635 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632dd8f8-37bc-4356-947b-1e9e94f2551a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:43.792370 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632dd8f8-37bc-4356-947b-1e9e94f2551a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:43.794030 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632dd8f8-37bc-4356-947b-1e9e94f2551a None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:43.814216 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 481/955] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:37:43 2022] POST /identity/v3/endpoints => generated 349 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:37:43.870031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b73681-94e9-4b49-94a0-6d37d0be1325 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:43.870232 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b73681-94e9-4b49-94a0-6d37d0be1325 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:43.870531 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b73681-94e9-4b49-94a0-6d37d0be1325 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:43.872196 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 475/956] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:43.878354 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cc749-4cb5-471f-9077-e9a9fea2e985 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:43.878659 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cc749-4cb5-471f-9077-e9a9fea2e985 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:43.878902 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cc749-4cb5-471f-9077-e9a9fea2e985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:43.898402 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-e78cc749-4cb5-471f-9077-e9a9fea2e985 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:43.899945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-e78cc749-4cb5-471f-9077-e9a9fea2e985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:43.984838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 482/957] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:37:44.006990 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b73681-94e9-4b49-94a0-6d37d0be1325 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:44.041466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:44.120186 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b73681-94e9-4b49-94a0-6d37d0be1325 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:44.156556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3521b92-bc3a-44cf-9f1b-96a731df747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:44.158058 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3521b92-bc3a-44cf-9f1b-96a731df747d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:44.158364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3521b92-bc3a-44cf-9f1b-96a731df747d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:44.158641 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3521b92-bc3a-44cf-9f1b-96a731df747d None admin] PATH_INFO: `/v3/services/compute` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:44.159311 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3521b92-bc3a-44cf-9f1b-96a731df747d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:44.160695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3521b92-bc3a-44cf-9f1b-96a731df747d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-b3521b92-bc3a-44cf-9f1b-96a731df747d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:44.173467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:44.174571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:44.176096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:44.176096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:44.176096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:44.176096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:44.176096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:44.176096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:44.176096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:44.176096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:44.176096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:44.176096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:44.176096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:44.176096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:44.176096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:37:44.176096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:44.176096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 476/958] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:44.180673 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cc749-4cb5-471f-9077-e9a9fea2e985 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:44.195195 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:44.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cc749-4cb5-471f-9077-e9a9fea2e985 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:44.315291 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27757d51-a8fe-40c2-bdd1-03c831de56c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:44.315964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27757d51-a8fe-40c2-bdd1-03c831de56c3 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:44.316237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27757d51-a8fe-40c2-bdd1-03c831de56c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:44.316503 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27757d51-a8fe-40c2-bdd1-03c831de56c3 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:44.317213 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27757d51-a8fe-40c2-bdd1-03c831de56c3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:44.317511 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27757d51-a8fe-40c2-bdd1-03c831de56c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:44.318984 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27757d51-a8fe-40c2-bdd1-03c831de56c3 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:44.324355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 483/959] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:37:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:44.330540 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3521b92-bc3a-44cf-9f1b-96a731df747d None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:44.427686 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:44.435226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3521b92-bc3a-44cf-9f1b-96a731df747d None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:44.476642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f421d-f1ae-4f25-90a1-bdb48d59e688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:44.477407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f421d-f1ae-4f25-90a1-bdb48d59e688 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:44.477599 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f421d-f1ae-4f25-90a1-bdb48d59e688 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:44.477869 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f421d-f1ae-4f25-90a1-bdb48d59e688 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:44.478637 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f421d-f1ae-4f25-90a1-bdb48d59e688 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:44.478945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f421d-f1ae-4f25-90a1-bdb48d59e688 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:44.480397 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f421d-f1ae-4f25-90a1-bdb48d59e688 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:44.487329 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 477/960] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:37:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:44.682585 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4c018-7095-449f-bce1-c0c96d4d4998 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:44.683185 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4c018-7095-449f-bce1-c0c96d4d4998 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:44.683185 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4c018-7095-449f-bce1-c0c96d4d4998 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:44.684384 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 484/961] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:44.690287 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c4cba-f5eb-496b-bbe9-0201432bba17 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:44.690577 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c4cba-f5eb-496b-bbe9-0201432bba17 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:44.690856 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c4cba-f5eb-496b-bbe9-0201432bba17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:44.711851 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-767c4cba-f5eb-496b-bbe9-0201432bba17 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:44.713048 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-767c4cba-f5eb-496b-bbe9-0201432bba17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:44.786177 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 478/962] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:44.809206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4c018-7095-449f-bce1-c0c96d4d4998 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:44.887731 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:44.913302 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4c018-7095-449f-bce1-c0c96d4d4998 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:44.947944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a656e02-db30-478c-9cc8-f9f97373f580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:44.948676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a656e02-db30-478c-9cc8-f9f97373f580 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:44.948947 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a656e02-db30-478c-9cc8-f9f97373f580 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:44.949212 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a656e02-db30-478c-9cc8-f9f97373f580 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:44.959930 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a656e02-db30-478c-9cc8-f9f97373f580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-1a656e02-db30-478c-9cc8-f9f97373f580 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:44.967998 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:44.969100 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:44.973800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:44.973800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:44.973800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:44.973800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:44.973800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:44.973800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:44.973800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:44.973800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:44.973800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:44.973800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:44.973800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:37:44.973800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:44.973800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 485/963] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 19:37:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:44.975607 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c4cba-f5eb-496b-bbe9-0201432bba17 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:45.072267 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:45.086833 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c4cba-f5eb-496b-bbe9-0201432bba17 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:45.122828 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b0a50-67e0-4ac4-a85e-6b1d2056bba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:45.123916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b0a50-67e0-4ac4-a85e-6b1d2056bba8 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:45.124387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b0a50-67e0-4ac4-a85e-6b1d2056bba8 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:45.124849 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b0a50-67e0-4ac4-a85e-6b1d2056bba8 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:45.125846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b0a50-67e0-4ac4-a85e-6b1d2056bba8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:45.126386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b0a50-67e0-4ac4-a85e-6b1d2056bba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:45.127955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b0a50-67e0-4ac4-a85e-6b1d2056bba8 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:45.137154 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 479/964] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 19:37:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:45.143185 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a656e02-db30-478c-9cc8-f9f97373f580 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:45.246452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a656e02-db30-478c-9cc8-f9f97373f580 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:45.282488 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d71d1d-1c1a-44e0-8a4f-10df06d684f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:45.283683 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d71d1d-1c1a-44e0-8a4f-10df06d684f3 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:45.284154 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d71d1d-1c1a-44e0-8a4f-10df06d684f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:45.284615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d71d1d-1c1a-44e0-8a4f-10df06d684f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:45.287731 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:45.296346 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d71d1d-1c1a-44e0-8a4f-10df06d684f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:45.298045 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d71d1d-1c1a-44e0-8a4f-10df06d684f3 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:45.315925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-b5d71d1d-1c1a-44e0-8a4f-10df06d684f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:45.315925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:45.315925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:45.315925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:45.315925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:45.315925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:45.315925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:45.315925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:45.315925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:45.315925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:45.315925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:45.315925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:45.315925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:45.315925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:45.315925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:45.316834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:45.318186 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:45.318186 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:45.318186 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:45.318186 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:45.318186 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 486/965] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:45.324236 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b0a50-67e0-4ac4-a85e-6b1d2056bba8 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:45.430130 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b0a50-67e0-4ac4-a85e-6b1d2056bba8 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:45.464558 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f17512-165a-4a5d-8f13-ed0e52867103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:45.465640 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f17512-165a-4a5d-8f13-ed0e52867103 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:45.466141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f17512-165a-4a5d-8f13-ed0e52867103 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:45.466635 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f17512-165a-4a5d-8f13-ed0e52867103 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:45.467576 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f17512-165a-4a5d-8f13-ed0e52867103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:45.468084 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f17512-165a-4a5d-8f13-ed0e52867103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:45.470106 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f17512-165a-4a5d-8f13-ed0e52867103 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:45.486214 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 480/966] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:45.494134 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d71d1d-1c1a-44e0-8a4f-10df06d684f3 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:45.595354 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d71d1d-1c1a-44e0-8a4f-10df06d684f3 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:45.633442 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc005b3-535c-4f5b-8ff5-54f48cc46341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:45.634287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc005b3-535c-4f5b-8ff5-54f48cc46341 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:45.634589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc005b3-535c-4f5b-8ff5-54f48cc46341 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:45.634859 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc005b3-535c-4f5b-8ff5-54f48cc46341 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:45.648166 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc005b3-535c-4f5b-8ff5-54f48cc46341 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:45.649649 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc005b3-535c-4f5b-8ff5-54f48cc46341 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-ebc005b3-535c-4f5b-8ff5-54f48cc46341 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:45.665222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:45.666210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:45.667710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:45.667710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:45.667710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:45.667710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:45.667710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:45.667710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:45.667710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:45.667710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:45.667710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:37:45.667710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:45.667710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 487/967] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:37:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:45.672286 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f17512-165a-4a5d-8f13-ed0e52867103 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:45.682430 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979f777-7c4b-4658-b94d-e18f4d0f5542 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:45.682764 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979f777-7c4b-4658-b94d-e18f4d0f5542 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:45.683321 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979f777-7c4b-4658-b94d-e18f4d0f5542 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:45.684934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 488/968] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:37:45.690449 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:45.691460 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb632836-aa50-4c33-80c0-91adb0f07d71 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:45.691600 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb632836-aa50-4c33-80c0-91adb0f07d71 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:45.691871 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb632836-aa50-4c33-80c0-91adb0f07d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:45.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-eb632836-aa50-4c33-80c0-91adb0f07d71 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:45.715890 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-eb632836-aa50-4c33-80c0-91adb0f07d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:45.767762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:45.793199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f17512-165a-4a5d-8f13-ed0e52867103 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:45.798755 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 489/969] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:37:45.817605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb632836-aa50-4c33-80c0-91adb0f07d71 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:45.831079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a28542-eb6d-44a1-8934-96bc0576944d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:45.832095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a28542-eb6d-44a1-8934-96bc0576944d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:45.832564 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a28542-eb6d-44a1-8934-96bc0576944d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:45.833068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a28542-eb6d-44a1-8934-96bc0576944d None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:45.834217 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a28542-eb6d-44a1-8934-96bc0576944d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:45.834746 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a28542-eb6d-44a1-8934-96bc0576944d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:45.836524 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a28542-eb6d-44a1-8934-96bc0576944d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:45.856459 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 481/970] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 19:37:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:45.862462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a28542-eb6d-44a1-8934-96bc0576944d None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:45.928164 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb632836-aa50-4c33-80c0-91adb0f07d71 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:45.940144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:45.964753 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa908263-c0f8-4293-b788-93d62a7ce1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:45.965467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa908263-c0f8-4293-b788-93d62a7ce1e0 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:45.965753 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa908263-c0f8-4293-b788-93d62a7ce1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:45.966021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa908263-c0f8-4293-b788-93d62a7ce1e0 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:45.966782 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa908263-c0f8-4293-b788-93d62a7ce1e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:45.968263 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa908263-c0f8-4293-b788-93d62a7ce1e0 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:45.981444 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a28542-eb6d-44a1-8934-96bc0576944d None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:45.991952 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.sql.core [None req-aa908263-c0f8-4293-b788-93d62a7ce1e0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 19:37:45.991952 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 19:37:45.991952 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [parameters: {'user_id': '6c323539818d4d949db070772903c202', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 19:37:45.991952 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86981) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 19:37:45.999452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-aa908263-c0f8-4293-b788-93d62a7ce1e0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 19:37:45.999452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:45.999452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:37:45.999452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:37:45.999452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:37:45.999452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:37:45.999452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:37:45.999452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:37:45.999452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:37:45.999452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application conn.query(q) May 19 19:37:45.999452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:37:45.999452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:37:45.999452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:37:45.999452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application result.read() May 19 19:37:45.999452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:37:46.000492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application next(self.gen) May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application yield resource May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application next(self.gen) May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self.session.rollback() May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self.force_reraise() May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise self.value May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application session.commit() May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 19:37:46.001456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._prepare_impl() May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self.session.flush() May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._flush(objects) May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application compat.raise_( May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application flush_context.execute() May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rec.execute(self) May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 19:37:46.003023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:37:46.003913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application conn.query(q) May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application result.read() May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:37:46.004717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:37:46.005526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:37:46.005526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:37:46.005526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:37:46.005526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:37:46.005526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 19:37:46.005526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 19:37:46.005526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [parameters: {'user_id': '6c323539818d4d949db070772903c202', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 19:37:46.005526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 19:37:46.005526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:46.005526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:46.005526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:46.005526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:46.005526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:46.005526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:46.005526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 19:37:46.007230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 19:37:46.008218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 19:37:46.008218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 19:37:46.008218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 19:37:46.008218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 19:37:46.008218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 19:37:46.008218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:46.008218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 490/971] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 19:37:45 2022] POST /identity/v3/users => generated 160 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:46.008218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa908263-c0f8-4293-b788-93d62a7ce1e0 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:46.033575 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e0ca9-1d99-49f1-8295-8958f25bcbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:46.034567 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e0ca9-1d99-49f1-8295-8958f25bcbe5 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:46.035096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e0ca9-1d99-49f1-8295-8958f25bcbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:46.035570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e0ca9-1d99-49f1-8295-8958f25bcbe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:46.069614 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:46.116165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e0ca9-1d99-49f1-8295-8958f25bcbe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:46.118033 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e0ca9-1d99-49f1-8295-8958f25bcbe5 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:46.143831 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa908263-c0f8-4293-b788-93d62a7ce1e0 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:46.187640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb3685-a305-43ac-a633-7c6842a92c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:46.188529 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb3685-a305-43ac-a633-7c6842a92c82 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:46.188797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb3685-a305-43ac-a633-7c6842a92c82 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:46.189060 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb3685-a305-43ac-a633-7c6842a92c82 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:46.190453 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:46.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb3685-a305-43ac-a633-7c6842a92c82 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:46.199506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb3685-a305-43ac-a633-7c6842a92c82 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-2abb3685-a305-43ac-a633-7c6842a92c82 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:46.214167 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:46.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:46.216075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:46.216075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:46.216075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:46.216075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:46.216075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:46.216075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:46.216075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:46.216075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:46.216075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:37:46.216075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:46.216075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 491/972] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:37:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:46.217421 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458cbbf-c6f6-4191-a58e-1f9538aad7a1 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:46.217421 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458cbbf-c6f6-4191-a58e-1f9538aad7a1 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:46.217421 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458cbbf-c6f6-4191-a58e-1f9538aad7a1 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:46.222024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 492/973] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:46.222791 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458cbbf-c6f6-4191-a58e-1f9538aad7a1 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:46.233177 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 482/974] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:37:45 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:37:46.234944 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902aad3-efc2-4454-b4b6-7a4218c7b974 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:46.235227 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902aad3-efc2-4454-b4b6-7a4218c7b974 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:46.235551 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902aad3-efc2-4454-b4b6-7a4218c7b974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:46.253169 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-c902aad3-efc2-4454-b4b6-7a4218c7b974 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:46.254460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-c902aad3-efc2-4454-b4b6-7a4218c7b974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:46.324612 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 483/975] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:46.325703 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902aad3-efc2-4454-b4b6-7a4218c7b974 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:46.342193 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:46.347711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458cbbf-c6f6-4191-a58e-1f9538aad7a1 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:46.385467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee71677-77c1-4210-96b9-5a75284d79f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:46.386190 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee71677-77c1-4210-96b9-5a75284d79f0 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:46.386508 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee71677-77c1-4210-96b9-5a75284d79f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:46.386776 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee71677-77c1-4210-96b9-5a75284d79f0 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:46.387574 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee71677-77c1-4210-96b9-5a75284d79f0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:46.387870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee71677-77c1-4210-96b9-5a75284d79f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:46.389306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee71677-77c1-4210-96b9-5a75284d79f0 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:46.425679 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:46.429841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902aad3-efc2-4454-b4b6-7a4218c7b974 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:46.434474 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 493/976] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:37:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:46.435695 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee71677-77c1-4210-96b9-5a75284d79f0 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:46.471979 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4d848-cec6-4c61-a61e-e63cfe458c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:46.473072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4d848-cec6-4c61-a61e-e63cfe458c05 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:46.473442 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4d848-cec6-4c61-a61e-e63cfe458c05 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:46.473858 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4d848-cec6-4c61-a61e-e63cfe458c05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:46.486709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4d848-cec6-4c61-a61e-e63cfe458c05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:46.488783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4d848-cec6-4c61-a61e-e63cfe458c05 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:46.508961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-13b4d848-cec6-4c61-a61e-e63cfe458c05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:46.508961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:46.508961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:46.508961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:46.508961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:46.508961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:46.508961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:46.508961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:46.508961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:46.508961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:46.508961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:46.508961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:46.508961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:46.508961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:46.508961 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:46.510103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:46.511261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:46.511261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:46.511261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:46.511261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:46.517897 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 484/977] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:46.518483 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4d848-cec6-4c61-a61e-e63cfe458c05 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:46.592656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee71677-77c1-4210-96b9-5a75284d79f0 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:46.640498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4d848-cec6-4c61-a61e-e63cfe458c05 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:46.651856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684148c-cc47-4204-8ba6-daa59875b520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:46.652795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684148c-cc47-4204-8ba6-daa59875b520 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:46.653075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684148c-cc47-4204-8ba6-daa59875b520 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:46.653345 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684148c-cc47-4204-8ba6-daa59875b520 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:46.654015 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684148c-cc47-4204-8ba6-daa59875b520 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:46.655596 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684148c-cc47-4204-8ba6-daa59875b520 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-f684148c-cc47-4204-8ba6-daa59875b520 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:46.671789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:46.673315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:46.674158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:46.674158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:46.674158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:46.674158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:46.674158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:46.674158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:46.674158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:46.674158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:46.674158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:46.674158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:46.674158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:46.674158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:46.674158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:37:46.674158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:46.675591 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 494/978] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:37:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:46.682220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684148c-cc47-4204-8ba6-daa59875b520 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:46.686797 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c39c66-cd1e-4f58-98a5-be53d0dcb5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:46.687478 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c39c66-cd1e-4f58-98a5-be53d0dcb5fd None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:46.688194 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c39c66-cd1e-4f58-98a5-be53d0dcb5fd None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:46.688194 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c39c66-cd1e-4f58-98a5-be53d0dcb5fd None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:46.688889 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c39c66-cd1e-4f58-98a5-be53d0dcb5fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:46.689188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c39c66-cd1e-4f58-98a5-be53d0dcb5fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:46.690628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c39c66-cd1e-4f58-98a5-be53d0dcb5fd None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:46.707263 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 485/979] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:46.712486 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c39c66-cd1e-4f58-98a5-be53d0dcb5fd None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:46.716972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:46.728641 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:46.805947 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684148c-cc47-4204-8ba6-daa59875b520 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:46.840388 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c39c66-cd1e-4f58-98a5-be53d0dcb5fd None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:46.841950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40faebd-3efe-4561-951a-2761a4e155e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:46.842660 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40faebd-3efe-4561-951a-2761a4e155e2 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:46.842937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40faebd-3efe-4561-951a-2761a4e155e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:46.843210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40faebd-3efe-4561-951a-2761a4e155e2 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:46.843922 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40faebd-3efe-4561-951a-2761a4e155e2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:46.844407 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40faebd-3efe-4561-951a-2761a4e155e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:46.845781 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40faebd-3efe-4561-951a-2761a4e155e2 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:46.850717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 495/980] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:37:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:46.852265 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055eb81-e533-47da-9ccd-31fe0cc4d89d None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:46.852546 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055eb81-e533-47da-9ccd-31fe0cc4d89d None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:46.852817 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055eb81-e533-47da-9ccd-31fe0cc4d89d None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:46.854048 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 496/981] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:46.856996 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055eb81-e533-47da-9ccd-31fe0cc4d89d None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:46.876631 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574453f1-0053-406d-a6fd-f40447984d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:46.877443 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574453f1-0053-406d-a6fd-f40447984d75 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:46.877723 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574453f1-0053-406d-a6fd-f40447984d75 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:46.877996 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574453f1-0053-406d-a6fd-f40447984d75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:46.885036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574453f1-0053-406d-a6fd-f40447984d75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:46.886487 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574453f1-0053-406d-a6fd-f40447984d75 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:46.900822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-574453f1-0053-406d-a6fd-f40447984d75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:46.900822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:46.900822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:46.900822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:46.900822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:46.900822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:46.900822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:46.900822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:46.900822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:46.900822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:46.900822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:46.900822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:46.900822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:46.900822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:46.900822 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:46.901855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:46.903023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:46.903023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:46.903023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:46.903023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:46.903023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:46.903023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:46.903023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:46.903023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:46.903023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 486/982] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:46.903525 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac4547-999c-4149-b844-aa24d69d3688 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:46.903592 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac4547-999c-4149-b844-aa24d69d3688 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:46.903826 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac4547-999c-4149-b844-aa24d69d3688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:46.919834 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-b6ac4547-999c-4149-b844-aa24d69d3688 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:46.921003 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-b6ac4547-999c-4149-b844-aa24d69d3688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:46.946335 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:46.962476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055eb81-e533-47da-9ccd-31fe0cc4d89d None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:46.985488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:46.990841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 487/983] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:46.991874 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac4547-999c-4149-b844-aa24d69d3688 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:47.004355 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1404f6-7f97-4089-bb15-428fa73afb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:47.005153 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1404f6-7f97-4089-bb15-428fa73afb12 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:47.005410 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1404f6-7f97-4089-bb15-428fa73afb12 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:47.005679 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1404f6-7f97-4089-bb15-428fa73afb12 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:47.006616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1404f6-7f97-4089-bb15-428fa73afb12 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:47.006946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1404f6-7f97-4089-bb15-428fa73afb12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:47.008416 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1404f6-7f97-4089-bb15-428fa73afb12 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:47.014000 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 497/984] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:37:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:47.014857 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1404f6-7f97-4089-bb15-428fa73afb12 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:47.094684 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac4547-999c-4149-b844-aa24d69d3688 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:47.124547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1404f6-7f97-4089-bb15-428fa73afb12 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:47.131423 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa66d9f-69d1-4569-94f2-e42d109bfd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:47.132123 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa66d9f-69d1-4569-94f2-e42d109bfd2f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:47.132397 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa66d9f-69d1-4569-94f2-e42d109bfd2f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:47.132653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa66d9f-69d1-4569-94f2-e42d109bfd2f None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:47.133448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa66d9f-69d1-4569-94f2-e42d109bfd2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:47.133751 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa66d9f-69d1-4569-94f2-e42d109bfd2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:47.135217 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa66d9f-69d1-4569-94f2-e42d109bfd2f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:47.147369 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 488/985] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:47.147972 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa66d9f-69d1-4569-94f2-e42d109bfd2f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:47.165220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0012bc-e842-4a84-a7dc-b032c7564d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:47.165821 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0012bc-e842-4a84-a7dc-b032c7564d0f None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:47.166103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0012bc-e842-4a84-a7dc-b032c7564d0f None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:47.166416 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0012bc-e842-4a84-a7dc-b032c7564d0f None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:47.167147 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0012bc-e842-4a84-a7dc-b032c7564d0f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:47.168528 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0012bc-e842-4a84-a7dc-b032c7564d0f None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:47.183619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 498/986] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:37:47 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:37:47.184412 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0012bc-e842-4a84-a7dc-b032c7564d0f None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:47.198985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:47.215096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:47.264773 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa66d9f-69d1-4569-94f2-e42d109bfd2f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:47.310440 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6194f12-1fe3-4997-ade1-c2035996c4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:47.311120 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6194f12-1fe3-4997-ade1-c2035996c4d1 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:47.311400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6194f12-1fe3-4997-ade1-c2035996c4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:47.311671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6194f12-1fe3-4997-ade1-c2035996c4d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:47.312478 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6194f12-1fe3-4997-ade1-c2035996c4d1 None admin] RBAC: Adding query filter params (interface=admin, service_id=cee42b6658564730a06f4202fa34f38d, region_id=RegionOne) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:47.313870 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6194f12-1fe3-4997-ade1-c2035996c4d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:47.314488 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0012bc-e842-4a84-a7dc-b032c7564d0f None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:47.315364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6194f12-1fe3-4997-ade1-c2035996c4d1 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:47.321181 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 489/987] 10.222.0.84 () {56 vars in 1249 bytes} [Thu May 19 19:37:47 2022] GET /identity/v3/endpoints?service_id=cee42b6658564730a06f4202fa34f38d&interface=admin®ion_id=RegionOne => generated 193 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:47.326977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6194f12-1fe3-4997-ade1-c2035996c4d1 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:47.359819 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a984062-d61e-429a-8035-1defe9279767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:47.368874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a984062-d61e-429a-8035-1defe9279767 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:47.369453 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a984062-d61e-429a-8035-1defe9279767 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:47.369883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a984062-d61e-429a-8035-1defe9279767 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:47.370801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a984062-d61e-429a-8035-1defe9279767 None admin] RBAC: Adding query filter params (role.id=de6fe15685d647c69e91e3772074a8b1, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=9e05338c14d642f3b31095f85e1e6ca4) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:47.371118 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a984062-d61e-429a-8035-1defe9279767 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:47.372601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a984062-d61e-429a-8035-1defe9279767 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:47.381069 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 499/988] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:47 2022] GET /identity/v3/role_assignments?user.id=9e05338c14d642f3b31095f85e1e6ca4&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=de6fe15685d647c69e91e3772074a8b1 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:47.447636 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6194f12-1fe3-4997-ade1-c2035996c4d1 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:47.471211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:47.488965 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047c82c-c3bb-43b9-800e-7eb52f6f4ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:47.489640 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047c82c-c3bb-43b9-800e-7eb52f6f4ade None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:47.489913 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047c82c-c3bb-43b9-800e-7eb52f6f4ade None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:47.490181 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047c82c-c3bb-43b9-800e-7eb52f6f4ade None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:47.490912 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047c82c-c3bb-43b9-800e-7eb52f6f4ade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:47.492286 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047c82c-c3bb-43b9-800e-7eb52f6f4ade None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:47.499588 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 490/989] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:37:47 2022] GET /identity/v3/services => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 19:37:49.439859 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831a832-d7f2-434d-a55e-96f27d4e4a4f None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:49.440405 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831a832-d7f2-434d-a55e-96f27d4e4a4f None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:49.440405 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831a832-d7f2-434d-a55e-96f27d4e4a4f None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:49.441869 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 500/990] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:49.448049 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf55db4-a795-48b7-bf12-bbf42e1afe83 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:49.448598 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf55db4-a795-48b7-bf12-bbf42e1afe83 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:49.448598 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf55db4-a795-48b7-bf12-bbf42e1afe83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:49.469678 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-0cf55db4-a795-48b7-bf12-bbf42e1afe83 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:49.471492 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-0cf55db4-a795-48b7-bf12-bbf42e1afe83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:49.553592 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 491/991] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:49.580448 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831a832-d7f2-434d-a55e-96f27d4e4a4f None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:49.641709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb132dd-a5d0-44a9-bca5-b815384c7f45 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:49.642008 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb132dd-a5d0-44a9-bca5-b815384c7f45 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:49.642545 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb132dd-a5d0-44a9-bca5-b815384c7f45 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:49.643914 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 492/992] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:49.650784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b60a48-3d10-4bdc-91e8-f20f455c6c9b None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:49.651603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b60a48-3d10-4bdc-91e8-f20f455c6c9b None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:49.651968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b60a48-3d10-4bdc-91e8-f20f455c6c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:49.664383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:49.674027 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-46b60a48-3d10-4bdc-91e8-f20f455c6c9b None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:49.675268 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-46b60a48-3d10-4bdc-91e8-f20f455c6c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:49.703844 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831a832-d7f2-434d-a55e-96f27d4e4a4f None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:49.740796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c58296-b68c-4534-84a6-ddc15aae10e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:49.741606 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c58296-b68c-4534-84a6-ddc15aae10e5 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:49.741881 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c58296-b68c-4534-84a6-ddc15aae10e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:49.742149 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c58296-b68c-4534-84a6-ddc15aae10e5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:49.742830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c58296-b68c-4534-84a6-ddc15aae10e5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:49.744202 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c58296-b68c-4534-84a6-ddc15aae10e5 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-69c58296-b68c-4534-84a6-ddc15aae10e5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:49.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:49.758804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:49.760024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:49.760024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:49.760024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:49.760024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:49.760024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:49.760024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:49.760024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:49.760024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:49.760024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:49.760024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:49.760024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:49.760024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:49.760024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:37:49.760024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:49.760024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 501/993] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:49.763229 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8926a6-49b6-4122-bc05-9c8be4978195 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:49.763509 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8926a6-49b6-4122-bc05-9c8be4978195 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:49.763807 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8926a6-49b6-4122-bc05-9c8be4978195 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:49.765615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 502/994] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:37:49.767824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8926a6-49b6-4122-bc05-9c8be4978195 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:49.772467 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 493/995] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:49.773761 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bfb4e-fd83-4619-a105-c994b1232dc5 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:49.774045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bfb4e-fd83-4619-a105-c994b1232dc5 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:49.774341 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bfb4e-fd83-4619-a105-c994b1232dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:49.792972 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-104bfb4e-fd83-4619-a105-c994b1232dc5 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:49.794827 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-104bfb4e-fd83-4619-a105-c994b1232dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:49.837144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:49.866850 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 494/996] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:49.867529 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bfb4e-fd83-4619-a105-c994b1232dc5 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:49.900298 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8926a6-49b6-4122-bc05-9c8be4978195 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:49.940895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7403-2c49-476b-9b02-ec7e9d1914cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:49.941560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7403-2c49-476b-9b02-ec7e9d1914cd None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:49.941833 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7403-2c49-476b-9b02-ec7e9d1914cd None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:49.942098 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7403-2c49-476b-9b02-ec7e9d1914cd None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:49.942843 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da7403-2c49-476b-9b02-ec7e9d1914cd None admin] RBAC: Adding query filter params (name=compute) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:49.943293 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da7403-2c49-476b-9b02-ec7e9d1914cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:49.944799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da7403-2c49-476b-9b02-ec7e9d1914cd None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:49.945962 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:49.951532 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 503/997] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:37:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:49.953187 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7403-2c49-476b-9b02-ec7e9d1914cd None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:49.985175 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bfb4e-fd83-4619-a105-c994b1232dc5 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:50.023521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914d772-c9d3-4fe8-bd46-11a308593fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:50.024327 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914d772-c9d3-4fe8-bd46-11a308593fdc None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:50.024608 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914d772-c9d3-4fe8-bd46-11a308593fdc None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:50.024887 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914d772-c9d3-4fe8-bd46-11a308593fdc None admin] PATH_INFO: `/v3/services/image` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:50.025537 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914d772-c9d3-4fe8-bd46-11a308593fdc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:50.026953 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914d772-c9d3-4fe8-bd46-11a308593fdc None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-e914d772-c9d3-4fe8-bd46-11a308593fdc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:50.039260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:50.040308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:50.042006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:50.042006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:50.042006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:50.042006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:50.042006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:50.042006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:50.042006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:50.042006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:50.042006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:50.042006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:50.042006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:50.042006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:50.042006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:37:50.042006 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:50.043154 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 495/998] 10.222.0.84 () {56 vars in 1105 bytes} [Thu May 19 19:37:49 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:50.043709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914d772-c9d3-4fe8-bd46-11a308593fdc None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:50.068019 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7403-2c49-476b-9b02-ec7e9d1914cd None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:50.089368 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:50.103794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c47339-5215-427e-9573-f9045283a10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:50.104617 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c47339-5215-427e-9573-f9045283a10d None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:50.104892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c47339-5215-427e-9573-f9045283a10d None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:50.105161 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c47339-5215-427e-9573-f9045283a10d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:50.105812 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c47339-5215-427e-9573-f9045283a10d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:50.107220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c47339-5215-427e-9573-f9045283a10d None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-26c47339-5215-427e-9573-f9045283a10d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:50.119999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:50.121010 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:50.121823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:50.121823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:50.121823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:50.121823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:50.121823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:50.121823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:50.121823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:50.121823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:50.121823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:50.121823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:50.121823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:50.121823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:50.121823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:37:50.121823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:50.121823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 504/999] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:37:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:50.122630 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c47339-5215-427e-9573-f9045283a10d None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:50.153427 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914d772-c9d3-4fe8-bd46-11a308593fdc None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:50.185729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:50.188715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a4a90-0c48-4144-b4eb-efc928ab4818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:50.189385 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a4a90-0c48-4144-b4eb-efc928ab4818 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:50.189665 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a4a90-0c48-4144-b4eb-efc928ab4818 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:50.189937 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a4a90-0c48-4144-b4eb-efc928ab4818 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:50.190719 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a4a90-0c48-4144-b4eb-efc928ab4818 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:50.191088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a4a90-0c48-4144-b4eb-efc928ab4818 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:50.192517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a4a90-0c48-4144-b4eb-efc928ab4818 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:50.197552 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 496/1000] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:37:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:50.198361 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a4a90-0c48-4144-b4eb-efc928ab4818 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:50.227622 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c47339-5215-427e-9573-f9045283a10d None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:50.260192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0eb7f3-3a05-47c1-9661-fdf91a4499b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:50.260781 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0eb7f3-3a05-47c1-9661-fdf91a4499b8 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:50.261054 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0eb7f3-3a05-47c1-9661-fdf91a4499b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:50.261315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0eb7f3-3a05-47c1-9661-fdf91a4499b8 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:50.262019 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0eb7f3-3a05-47c1-9661-fdf91a4499b8 None admin] RBAC: Adding query filter params (name=image) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:50.262339 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0eb7f3-3a05-47c1-9661-fdf91a4499b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:50.263710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0eb7f3-3a05-47c1-9661-fdf91a4499b8 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:50.268736 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 505/1001] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:50.273940 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0eb7f3-3a05-47c1-9661-fdf91a4499b8 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:50.299637 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a4a90-0c48-4144-b4eb-efc928ab4818 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:50.301208 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:50.334416 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dbb01-fe98-42bc-becc-d367c9ded3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:50.335080 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dbb01-fe98-42bc-becc-d367c9ded3ea None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:50.335355 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dbb01-fe98-42bc-becc-d367c9ded3ea None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:50.335620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dbb01-fe98-42bc-becc-d367c9ded3ea None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:50.336313 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dbb01-fe98-42bc-becc-d367c9ded3ea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:50.336620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dbb01-fe98-42bc-becc-d367c9ded3ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:50.337973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dbb01-fe98-42bc-becc-d367c9ded3ea None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:50.347409 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 497/1002] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:37:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:50.347409 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dbb01-fe98-42bc-becc-d367c9ded3ea None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:50.377502 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0eb7f3-3a05-47c1-9661-fdf91a4499b8 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:50.402192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:50.411136 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c018bf4-88da-4a35-a9b4-b143b8fc54e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:50.411792 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c018bf4-88da-4a35-a9b4-b143b8fc54e5 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:50.412065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c018bf4-88da-4a35-a9b4-b143b8fc54e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:50.412332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c018bf4-88da-4a35-a9b4-b143b8fc54e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:50.413116 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c018bf4-88da-4a35-a9b4-b143b8fc54e5 None admin] RBAC: Adding query filter params (interface=public, service_id=9602164cecb44e1eba1a875a86f71416, region_id=RegionOne) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:50.413418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c018bf4-88da-4a35-a9b4-b143b8fc54e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:50.414822 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c018bf4-88da-4a35-a9b4-b143b8fc54e5 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:50.419725 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 506/1003] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:37:50 2022] GET /identity/v3/endpoints?service_id=9602164cecb44e1eba1a875a86f71416&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:50.420477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c018bf4-88da-4a35-a9b4-b143b8fc54e5 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:50.448555 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dbb01-fe98-42bc-becc-d367c9ded3ea None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:50.482528 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc10b3-24f7-4de0-8c76-37283faf9f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:50.483200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc10b3-24f7-4de0-8c76-37283faf9f3d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:50.483475 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc10b3-24f7-4de0-8c76-37283faf9f3d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:50.483744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc10b3-24f7-4de0-8c76-37283faf9f3d None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:50.484538 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc10b3-24f7-4de0-8c76-37283faf9f3d None admin] RBAC: Adding query filter params (type=image) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:50.484916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc10b3-24f7-4de0-8c76-37283faf9f3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:50.486480 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc10b3-24f7-4de0-8c76-37283faf9f3d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:50.490943 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 498/1004] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:50.491704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc10b3-24f7-4de0-8c76-37283faf9f3d None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:50.522513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c018bf4-88da-4a35-a9b4-b143b8fc54e5 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:50.529230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:50.563464 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b9ac4-5813-423d-8d55-873edfc38541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:50.564120 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b9ac4-5813-423d-8d55-873edfc38541 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:50.564396 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b9ac4-5813-423d-8d55-873edfc38541 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:50.564664 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b9ac4-5813-423d-8d55-873edfc38541 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:50.565394 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b9ac4-5813-423d-8d55-873edfc38541 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:50.565693 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b9ac4-5813-423d-8d55-873edfc38541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:50.567141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b9ac4-5813-423d-8d55-873edfc38541 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:50.572749 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 507/1005] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:37:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:50.578699 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b9ac4-5813-423d-8d55-873edfc38541 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:50.622837 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc10b3-24f7-4de0-8c76-37283faf9f3d None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:50.646115 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:50.671760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad27a32-88ac-4c75-9b90-5afb4b7864c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:50.672480 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad27a32-88ac-4c75-9b90-5afb4b7864c8 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:50.672754 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad27a32-88ac-4c75-9b90-5afb4b7864c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:50.673217 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad27a32-88ac-4c75-9b90-5afb4b7864c8 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:50.673911 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad27a32-88ac-4c75-9b90-5afb4b7864c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:50.675341 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad27a32-88ac-4c75-9b90-5afb4b7864c8 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:50.681855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 499/1006] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:37:50 2022] GET /identity/v3/services => generated 1727 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:37:50.696327 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b9ac4-5813-423d-8d55-873edfc38541 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:50.735686 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a018a-a8af-4b39-9ba4-2c0b2bd7e61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:50.736364 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a018a-a8af-4b39-9ba4-2c0b2bd7e61f None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:50.736640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a018a-a8af-4b39-9ba4-2c0b2bd7e61f None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:50.736905 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a018a-a8af-4b39-9ba4-2c0b2bd7e61f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:50.737648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a018a-a8af-4b39-9ba4-2c0b2bd7e61f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:50.739065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a018a-a8af-4b39-9ba4-2c0b2bd7e61f None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:50.756867 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 508/1007] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:37:50 2022] POST /identity/v3/endpoints => generated 325 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:37:52.767478 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0525d-9cd0-4961-8476-dbfbd6ea94d3 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:52.768040 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0525d-9cd0-4961-8476-dbfbd6ea94d3 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:52.768040 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0525d-9cd0-4961-8476-dbfbd6ea94d3 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:52.769413 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 500/1008] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:52.775057 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79397a-3ed7-4e39-9a73-c37ba38e7a6a None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:52.775531 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79397a-3ed7-4e39-9a73-c37ba38e7a6a None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:52.775609 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79397a-3ed7-4e39-9a73-c37ba38e7a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:52.794564 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-ca79397a-3ed7-4e39-9a73-c37ba38e7a6a None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:52.796138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-ca79397a-3ed7-4e39-9a73-c37ba38e7a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:52.875031 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 509/1009] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:52 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:52.897586 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0525d-9cd0-4961-8476-dbfbd6ea94d3 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:52.898935 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ea503-e703-48ea-a95c-23f61efdfd3d None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:52.899220 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ea503-e703-48ea-a95c-23f61efdfd3d None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:52.899488 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ea503-e703-48ea-a95c-23f61efdfd3d None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:52.900873 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 510/1010] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:52.907080 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9851f-97fc-4649-b92a-c7534e59e497 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:52.907358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9851f-97fc-4649-b92a-c7534e59e497 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:52.907627 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9851f-97fc-4649-b92a-c7534e59e497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:52.924765 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-2cd9851f-97fc-4649-b92a-c7534e59e497 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:52.925947 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-2cd9851f-97fc-4649-b92a-c7534e59e497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:52.961892 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:52.995861 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 511/1011] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:52 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:52.997185 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7a708-68b8-43c3-b467-bcee0937fdc6 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:52.997467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7a708-68b8-43c3-b467-bcee0937fdc6 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:52.997735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7a708-68b8-43c3-b467-bcee0937fdc6 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:52.999174 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 512/1012] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:53.004337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d8a7c-f6bb-47a4-86ca-72e9950a5955 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:53.004624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d8a7c-f6bb-47a4-86ca-72e9950a5955 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:53.004901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d8a7c-f6bb-47a4-86ca-72e9950a5955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:53.005243 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:53.014405 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0525d-9cd0-4961-8476-dbfbd6ea94d3 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:53.022377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-506d8a7c-f6bb-47a4-86ca-72e9950a5955 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:53.023558 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-506d8a7c-f6bb-47a4-86ca-72e9950a5955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:53.048533 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3cb18-8c98-4ea4-b60a-51a4235cf2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:53.049196 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3cb18-8c98-4ea4-b60a-51a4235cf2c2 None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:53.049472 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3cb18-8c98-4ea4-b60a-51a4235cf2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:53.049740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3cb18-8c98-4ea4-b60a-51a4235cf2c2 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:53.050491 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3cb18-8c98-4ea4-b60a-51a4235cf2c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:53.051855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3cb18-8c98-4ea4-b60a-51a4235cf2c2 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:53.066910 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 501/1013] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:37:52 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:37:53.067715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3cb18-8c98-4ea4-b60a-51a4235cf2c2 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:53.099569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 513/1014] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:53 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:37:53.122982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d8a7c-f6bb-47a4-86ca-72e9950a5955 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:53.189872 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3cb18-8c98-4ea4-b60a-51a4235cf2c2 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:53.230014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:53.232975 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca48fc5-c8f6-46bc-bff9-2534a90dc727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:53.234649 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca48fc5-c8f6-46bc-bff9-2534a90dc727 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:53.235306 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca48fc5-c8f6-46bc-bff9-2534a90dc727 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:53.235851 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca48fc5-c8f6-46bc-bff9-2534a90dc727 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:53.236744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca48fc5-c8f6-46bc-bff9-2534a90dc727 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:53.236960 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d8a7c-f6bb-47a4-86ca-72e9950a5955 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:53.238623 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca48fc5-c8f6-46bc-bff9-2534a90dc727 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-aca48fc5-c8f6-46bc-bff9-2534a90dc727 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:53.251530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:53.252690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:53.253766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:53.253766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:53.253766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:53.253766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:53.253766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:53.253766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:53.253766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:53.253766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:53.253766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:53.253766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:53.253766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:53.253766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:53.253766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:37:53.253766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:53.253766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 502/1015] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:53.258757 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca48fc5-c8f6-46bc-bff9-2534a90dc727 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:53.278896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:53.282076 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794083e-c8f7-4fb0-baaf-cedae40a403e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:53.283194 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794083e-c8f7-4fb0-baaf-cedae40a403e None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:53.283666 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794083e-c8f7-4fb0-baaf-cedae40a403e None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:53.283940 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794083e-c8f7-4fb0-baaf-cedae40a403e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:53.285145 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794083e-c8f7-4fb0-baaf-cedae40a403e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:53.286761 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794083e-c8f7-4fb0-baaf-cedae40a403e None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:53.288644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 514/1016] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:53.294486 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794083e-c8f7-4fb0-baaf-cedae40a403e None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:53.370432 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca48fc5-c8f6-46bc-bff9-2534a90dc727 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:53.396070 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794083e-c8f7-4fb0-baaf-cedae40a403e None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:53.407007 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3ebac-4c49-46ce-be6c-5d3301987d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:53.407727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3ebac-4c49-46ce-be6c-5d3301987d20 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:53.408011 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3ebac-4c49-46ce-be6c-5d3301987d20 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:53.408279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3ebac-4c49-46ce-be6c-5d3301987d20 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:53.409092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b3ebac-4c49-46ce-be6c-5d3301987d20 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:53.409395 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b3ebac-4c49-46ce-be6c-5d3301987d20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:53.411023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b3ebac-4c49-46ce-be6c-5d3301987d20 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:53.420993 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 503/1017] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:37:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:53.421725 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3ebac-4c49-46ce-be6c-5d3301987d20 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:53.431123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494b099-00e6-4a27-9765-28a19ebb96e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:53.431790 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494b099-00e6-4a27-9765-28a19ebb96e6 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:53.432064 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494b099-00e6-4a27-9765-28a19ebb96e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:53.432333 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494b099-00e6-4a27-9765-28a19ebb96e6 None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:53.433049 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494b099-00e6-4a27-9765-28a19ebb96e6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:53.434521 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494b099-00e6-4a27-9765-28a19ebb96e6 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:53.470544 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 515/1018] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:37:53 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:37:53.516774 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:53.538951 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3ebac-4c49-46ce-be6c-5d3301987d20 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:53.576336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d914f4b-a8dd-418a-997f-93703641b439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:53.577012 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d914f4b-a8dd-418a-997f-93703641b439 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:53.577292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d914f4b-a8dd-418a-997f-93703641b439 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:53.577599 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d914f4b-a8dd-418a-997f-93703641b439 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:53.578408 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d914f4b-a8dd-418a-997f-93703641b439 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:53.578708 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d914f4b-a8dd-418a-997f-93703641b439 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:53.580101 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d914f4b-a8dd-418a-997f-93703641b439 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:53.585846 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 504/1019] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:37:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:53.591813 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494b099-00e6-4a27-9765-28a19ebb96e6 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:53.612814 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:53.714867 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494b099-00e6-4a27-9765-28a19ebb96e6 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:53.750408 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b237e9-6bea-4c2d-978a-f1305975c839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:53.750989 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b237e9-6bea-4c2d-978a-f1305975c839 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:53.751266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b237e9-6bea-4c2d-978a-f1305975c839 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:53.751537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b237e9-6bea-4c2d-978a-f1305975c839 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:53.752290 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b237e9-6bea-4c2d-978a-f1305975c839 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:53.753674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b237e9-6bea-4c2d-978a-f1305975c839 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:53.774013 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 516/1020] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:37:53 2022] POST /identity/v3/endpoints => generated 334 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:37:55.278071 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc0cc4-c359-4b19-aeae-270ddecff771 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:55.278658 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc0cc4-c359-4b19-aeae-270ddecff771 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:55.278658 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc0cc4-c359-4b19-aeae-270ddecff771 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:55.279973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 505/1021] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:55.285771 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389765c-b583-4291-a6f1-046c64a83e71 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:55.286205 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389765c-b583-4291-a6f1-046c64a83e71 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:55.286308 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389765c-b583-4291-a6f1-046c64a83e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:55.303334 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-d389765c-b583-4291-a6f1-046c64a83e71 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:55.304489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-d389765c-b583-4291-a6f1-046c64a83e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:55.373684 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:55.386578 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 517/1022] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:55.403226 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc0cc4-c359-4b19-aeae-270ddecff771 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:55.510066 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc0cc4-c359-4b19-aeae-270ddecff771 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:55.545174 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150f463-24f4-41eb-8c4e-38a0b0e47838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:55.546000 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150f463-24f4-41eb-8c4e-38a0b0e47838 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:55.546311 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150f463-24f4-41eb-8c4e-38a0b0e47838 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:55.546575 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150f463-24f4-41eb-8c4e-38a0b0e47838 None admin] PATH_INFO: `/v3/services/image` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:55.547397 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150f463-24f4-41eb-8c4e-38a0b0e47838 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:55.548813 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150f463-24f4-41eb-8c4e-38a0b0e47838 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-5150f463-24f4-41eb-8c4e-38a0b0e47838 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:55.561380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:55.562839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:55.563784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:55.563784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:55.563784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:55.563784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:55.563784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:55.563784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:55.563784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:55.563784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:55.563784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:55.563784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:55.563784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:55.563784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:55.563784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:37:55.563784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:55.566699 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 506/1023] 10.222.0.84 () {56 vars in 1105 bytes} [Thu May 19 19:37:55 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:55.568973 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389765c-b583-4291-a6f1-046c64a83e71 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:55.681467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389765c-b583-4291-a6f1-046c64a83e71 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:55.715666 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fde2d7-af22-4caa-891f-3a02084029ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:55.716323 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fde2d7-af22-4caa-891f-3a02084029ec None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:55.716597 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fde2d7-af22-4caa-891f-3a02084029ec None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:55.716863 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fde2d7-af22-4caa-891f-3a02084029ec None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:55.717570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fde2d7-af22-4caa-891f-3a02084029ec None admin] RBAC: Adding query filter params (name=image) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:55.717866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fde2d7-af22-4caa-891f-3a02084029ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:55.719332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fde2d7-af22-4caa-891f-3a02084029ec None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:55.729841 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 518/1024] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:55.730844 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150f463-24f4-41eb-8c4e-38a0b0e47838 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:55.734037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708f9da-d318-451d-92b2-49451652d352 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:55.734501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708f9da-d318-451d-92b2-49451652d352 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:55.734881 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708f9da-d318-451d-92b2-49451652d352 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:55.737217 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 519/1025] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:37:55.743692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65325c55-1c49-4602-ac73-18092bec26c8 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:55.743948 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65325c55-1c49-4602-ac73-18092bec26c8 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:55.744244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65325c55-1c49-4602-ac73-18092bec26c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:55.745092 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:55.762175 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-65325c55-1c49-4602-ac73-18092bec26c8 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:55.763356 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-65325c55-1c49-4602-ac73-18092bec26c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:55.834166 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:55.845914 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150f463-24f4-41eb-8c4e-38a0b0e47838 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:55.848549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 520/1026] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:37:55.874376 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65325c55-1c49-4602-ac73-18092bec26c8 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:55.882311 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad593a-8ced-4fd1-a92c-a49ef8b3e1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:55.883245 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad593a-8ced-4fd1-a92c-a49ef8b3e1cc None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:55.883702 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad593a-8ced-4fd1-a92c-a49ef8b3e1cc None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:55.884161 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad593a-8ced-4fd1-a92c-a49ef8b3e1cc None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:55.885048 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad593a-8ced-4fd1-a92c-a49ef8b3e1cc None admin] RBAC: Adding query filter params (type=image) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:55.885355 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad593a-8ced-4fd1-a92c-a49ef8b3e1cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:55.886765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad593a-8ced-4fd1-a92c-a49ef8b3e1cc None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:55.892107 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 507/1027] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:55.897747 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad593a-8ced-4fd1-a92c-a49ef8b3e1cc None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:55.981727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:55.985222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65325c55-1c49-4602-ac73-18092bec26c8 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:56.007693 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad593a-8ced-4fd1-a92c-a49ef8b3e1cc None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:56.021770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5502f5d-853d-46f7-9166-046dd6865dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:56.022618 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5502f5d-853d-46f7-9166-046dd6865dd4 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:56.022893 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5502f5d-853d-46f7-9166-046dd6865dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:56.023160 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5502f5d-853d-46f7-9166-046dd6865dd4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:56.024316 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5502f5d-853d-46f7-9166-046dd6865dd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:56.025762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5502f5d-853d-46f7-9166-046dd6865dd4 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:56.027422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 521/1028] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:56.033819 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5502f5d-853d-46f7-9166-046dd6865dd4 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:56.044174 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf276012-dd71-4822-92c1-44bcf27a8e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:56.045071 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf276012-dd71-4822-92c1-44bcf27a8e83 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:56.045629 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf276012-dd71-4822-92c1-44bcf27a8e83 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:56.046076 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf276012-dd71-4822-92c1-44bcf27a8e83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:56.047083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf276012-dd71-4822-92c1-44bcf27a8e83 None admin] RBAC: Adding query filter params (interface=public, service_id=1520ea6b387745c5b0bd46d2bb097f75, region_id=RegionOne) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:56.047589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf276012-dd71-4822-92c1-44bcf27a8e83 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:56.049344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf276012-dd71-4822-92c1-44bcf27a8e83 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:56.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 508/1029] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:37:55 2022] GET /identity/v3/endpoints?service_id=1520ea6b387745c5b0bd46d2bb097f75&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:56.060191 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf276012-dd71-4822-92c1-44bcf27a8e83 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:56.099925 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:56.144089 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5502f5d-853d-46f7-9166-046dd6865dd4 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:56.166526 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf276012-dd71-4822-92c1-44bcf27a8e83 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:56.179750 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783ca97-0ef1-4e4d-998c-7c66d05b55e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:56.180428 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783ca97-0ef1-4e4d-998c-7c66d05b55e5 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:56.180799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783ca97-0ef1-4e4d-998c-7c66d05b55e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:56.181082 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783ca97-0ef1-4e4d-998c-7c66d05b55e5 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:56.181783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783ca97-0ef1-4e4d-998c-7c66d05b55e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:56.183263 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783ca97-0ef1-4e4d-998c-7c66d05b55e5 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:56.205331 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 522/1030] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:37:56 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:37:56.205509 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc0e5d-dce7-4209-8ce9-5c2ab1e3f792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:56.206018 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc0e5d-dce7-4209-8ce9-5c2ab1e3f792 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:56.206326 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc0e5d-dce7-4209-8ce9-5c2ab1e3f792 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:56.206605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc0e5d-dce7-4209-8ce9-5c2ab1e3f792 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:56.206705 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a199eb6-87ef-4a6c-a0c6-23a437719fe3 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:56.206905 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a199eb6-87ef-4a6c-a0c6-23a437719fe3 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:56.207182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a199eb6-87ef-4a6c-a0c6-23a437719fe3 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:56.207388 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc0e5d-dce7-4209-8ce9-5c2ab1e3f792 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:56.208594 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 523/1031] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:56.208703 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc0e5d-dce7-4209-8ce9-5c2ab1e3f792 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:56.214296 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf6fa6-6e34-4ed7-8a65-30e41573d75e None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:56.214604 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf6fa6-6e34-4ed7-8a65-30e41573d75e None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:56.214871 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf6fa6-6e34-4ed7-8a65-30e41573d75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:56.222096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 509/1032] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:37:56 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 19:37:56.234887 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-edbf6fa6-6e34-4ed7-8a65-30e41573d75e None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:56.236170 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-edbf6fa6-6e34-4ed7-8a65-30e41573d75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:56.317863 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 524/1033] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:56.342228 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc0e5d-dce7-4209-8ce9-5c2ab1e3f792 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:56.345541 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:56.462188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc0e5d-dce7-4209-8ce9-5c2ab1e3f792 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:56.498305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828c8e9-1434-47f1-840e-c47f01e21080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:56.499127 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828c8e9-1434-47f1-840e-c47f01e21080 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:56.499417 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828c8e9-1434-47f1-840e-c47f01e21080 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:56.499690 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828c8e9-1434-47f1-840e-c47f01e21080 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:56.510566 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828c8e9-1434-47f1-840e-c47f01e21080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-d828c8e9-1434-47f1-840e-c47f01e21080 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:56.519137 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:56.520031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:56.520968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:56.520968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:56.520968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:56.520968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:56.520968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:56.520968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:56.520968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:56.520968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:56.520968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:56.520968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:56.520968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:37:56.520968 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:56.522726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 510/1034] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 19:37:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:56.528514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf6fa6-6e34-4ed7-8a65-30e41573d75e None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:56.576994 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:56.642537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf6fa6-6e34-4ed7-8a65-30e41573d75e None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:56.680900 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9bad8-cb90-42b9-80ed-2b0df87e30c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:56.682037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9bad8-cb90-42b9-80ed-2b0df87e30c3 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:56.682582 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9bad8-cb90-42b9-80ed-2b0df87e30c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:56.683071 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9bad8-cb90-42b9-80ed-2b0df87e30c3 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:56.684059 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9bad8-cb90-42b9-80ed-2b0df87e30c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:56.684625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9bad8-cb90-42b9-80ed-2b0df87e30c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:56.686207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9bad8-cb90-42b9-80ed-2b0df87e30c3 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:56.696271 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 525/1035] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 19:37:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:56.701556 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828c8e9-1434-47f1-840e-c47f01e21080 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:56.770743 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:56.812527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828c8e9-1434-47f1-840e-c47f01e21080 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:56.847965 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d660d-7281-4afa-bb4c-8b573674bab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:56.848767 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d660d-7281-4afa-bb4c-8b573674bab4 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:56.849050 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d660d-7281-4afa-bb4c-8b573674bab4 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:56.849322 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d660d-7281-4afa-bb4c-8b573674bab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:56.861483 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d660d-7281-4afa-bb4c-8b573674bab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:56.862975 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d660d-7281-4afa-bb4c-8b573674bab4 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:56.880522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-881d660d-7281-4afa-bb4c-8b573674bab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:56.880522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:56.880522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:56.880522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:56.880522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:56.880522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:56.880522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:56.880522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:56.880522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:56.880522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:56.880522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:56.880522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:56.880522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:56.880522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:56.880522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:56.881384 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:56.887153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:56.887153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:56.887153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:56.887153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:56.887153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 511/1036] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:56.887422 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9bad8-cb90-42b9-80ed-2b0df87e30c3 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:56.996817 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9bad8-cb90-42b9-80ed-2b0df87e30c3 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:57.009712 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:57.037157 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce854c-dad5-498c-92d8-537193cd8e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:57.037842 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce854c-dad5-498c-92d8-537193cd8e26 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:57.038121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce854c-dad5-498c-92d8-537193cd8e26 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:57.038426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce854c-dad5-498c-92d8-537193cd8e26 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:57.039171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce854c-dad5-498c-92d8-537193cd8e26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:57.039477 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce854c-dad5-498c-92d8-537193cd8e26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:57.040871 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce854c-dad5-498c-92d8-537193cd8e26 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:57.055741 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 526/1037] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:57.061263 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d660d-7281-4afa-bb4c-8b573674bab4 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:57.166070 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d660d-7281-4afa-bb4c-8b573674bab4 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:57.179628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:57.200468 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc66b9-bbe8-468c-9f91-e92aa27d99bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:57.201265 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc66b9-bbe8-468c-9f91-e92aa27d99bd None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:57.201544 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc66b9-bbe8-468c-9f91-e92aa27d99bd None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:57.201812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc66b9-bbe8-468c-9f91-e92aa27d99bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:57.209314 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc66b9-bbe8-468c-9f91-e92aa27d99bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:57.210794 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc66b9-bbe8-468c-9f91-e92aa27d99bd None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-7dcc66b9-bbe8-468c-9f91-e92aa27d99bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:37:57.224896 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:57.226141 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:57.227501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:57.227501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:57.227501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:37:57.227501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:37:57.227501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:37:57.227501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:37:57.227501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:37:57.227501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:37:57.227501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:37:57.227501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:57.227501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 512/1038] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:37:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:57.232392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce854c-dad5-498c-92d8-537193cd8e26 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:57.334943 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce854c-dad5-498c-92d8-537193cd8e26 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:57.376287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9b8a1-efb2-40a9-bec5-88e840b58815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:57.377003 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9b8a1-efb2-40a9-bec5-88e840b58815 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:57.377282 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9b8a1-efb2-40a9-bec5-88e840b58815 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:57.377553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9b8a1-efb2-40a9-bec5-88e840b58815 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:57.378438 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9b8a1-efb2-40a9-bec5-88e840b58815 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:57.378759 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9b8a1-efb2-40a9-bec5-88e840b58815 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:57.380206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9b8a1-efb2-40a9-bec5-88e840b58815 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:57.398791 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 527/1039] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:37:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:57.404611 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc66b9-bbe8-468c-9f91-e92aa27d99bd None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:57.509056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc66b9-bbe8-468c-9f91-e92aa27d99bd None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:57.542847 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc152c9e-6fe9-4af9-aeaa-e1c831279741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:57.543502 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc152c9e-6fe9-4af9-aeaa-e1c831279741 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:57.543782 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc152c9e-6fe9-4af9-aeaa-e1c831279741 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:57.544055 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc152c9e-6fe9-4af9-aeaa-e1c831279741 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:57.587652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:57.614926 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc152c9e-6fe9-4af9-aeaa-e1c831279741 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:57.616583 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc152c9e-6fe9-4af9-aeaa-e1c831279741 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:57.725224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 513/1040] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:37:57 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:37:57.726929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9b8a1-efb2-40a9-bec5-88e840b58815 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:57.755701 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:57.833478 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9b8a1-efb2-40a9-bec5-88e840b58815 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:57.870826 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9fba9-1cc9-40a8-959c-60b50f8de95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:57.871550 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9fba9-1cc9-40a8-959c-60b50f8de95c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:57.871827 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9fba9-1cc9-40a8-959c-60b50f8de95c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:57.872097 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9fba9-1cc9-40a8-959c-60b50f8de95c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:57.882473 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9fba9-1cc9-40a8-959c-60b50f8de95c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:57.883970 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9fba9-1cc9-40a8-959c-60b50f8de95c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:57.902021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-e2e9fba9-1cc9-40a8-959c-60b50f8de95c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:57.902021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:57.902021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:57.902021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:57.902021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:57.902021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:57.902021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:57.902021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:57.902021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:57.902021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:57.902021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:57.902021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:57.902021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:37:57.902021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:37:57.902021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:57.903210 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:57.908673 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:37:57.908673 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:37:57.908673 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:37:57.908673 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:57.908673 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 528/1041] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:57.908953 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc152c9e-6fe9-4af9-aeaa-e1c831279741 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:57.987041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:58.014232 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc152c9e-6fe9-4af9-aeaa-e1c831279741 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:58.048727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2f703-2b4e-4f6c-adb8-2cf1d60c6619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:58.049400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2f703-2b4e-4f6c-adb8-2cf1d60c6619 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:58.049683 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2f703-2b4e-4f6c-adb8-2cf1d60c6619 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:58.049957 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2f703-2b4e-4f6c-adb8-2cf1d60c6619 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:58.050713 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2f703-2b4e-4f6c-adb8-2cf1d60c6619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:58.051021 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2f703-2b4e-4f6c-adb8-2cf1d60c6619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:58.052431 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2f703-2b4e-4f6c-adb8-2cf1d60c6619 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:58.065071 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 514/1042] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:58.070554 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9fba9-1cc9-40a8-959c-60b50f8de95c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:58.170224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:58.196255 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9fba9-1cc9-40a8-959c-60b50f8de95c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:58.232430 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fdd9c-fd5a-4593-8800-48ceef8adcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:58.233467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fdd9c-fd5a-4593-8800-48ceef8adcff None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:58.233893 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fdd9c-fd5a-4593-8800-48ceef8adcff None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:58.235803 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fdd9c-fd5a-4593-8800-48ceef8adcff None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:58.243080 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32fdd9c-fd5a-4593-8800-48ceef8adcff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:58.244694 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32fdd9c-fd5a-4593-8800-48ceef8adcff None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:58.259744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-f32fdd9c-fd5a-4593-8800-48ceef8adcff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:58.259744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:58.259744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:58.259744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:58.259744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:58.259744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:58.259744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:58.259744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:58.259744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:58.259744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:58.259744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:58.259744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:58.259744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:37:58.259744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:37:58.259744 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:58.260816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:58.261713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:37:58.261713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:37:58.261713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:37:58.261713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:37:58.261713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:37:58.261713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:37:58.261713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:37:58.261713 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:58.266427 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 529/1043] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:37:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:58.269040 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2f703-2b4e-4f6c-adb8-2cf1d60c6619 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:58.387457 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2f703-2b4e-4f6c-adb8-2cf1d60c6619 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:58.425219 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbac74a-2836-45e7-8d2e-2f4da121ed4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:58.425847 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbac74a-2836-45e7-8d2e-2f4da121ed4d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:58.426094 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbac74a-2836-45e7-8d2e-2f4da121ed4d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:58.426618 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbac74a-2836-45e7-8d2e-2f4da121ed4d None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:58.427486 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbac74a-2836-45e7-8d2e-2f4da121ed4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:58.427797 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbac74a-2836-45e7-8d2e-2f4da121ed4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:58.429283 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbac74a-2836-45e7-8d2e-2f4da121ed4d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:58.431271 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:58.448668 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 515/1044] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:37:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:58.449957 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fdd9c-fd5a-4593-8800-48ceef8adcff None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:58.472136 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af439d4-dd58-4b43-9816-7aeb579901c9 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:58.472458 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af439d4-dd58-4b43-9816-7aeb579901c9 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:58.472689 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af439d4-dd58-4b43-9816-7aeb579901c9 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:58.474336 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 516/1045] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:37:58.479800 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3722694-3ca4-45be-8841-c43450e4389e None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:58.480325 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3722694-3ca4-45be-8841-c43450e4389e None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:58.480603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3722694-3ca4-45be-8841-c43450e4389e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:58.498382 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-e3722694-3ca4-45be-8841-c43450e4389e None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:58.499553 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-e3722694-3ca4-45be-8841-c43450e4389e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:58.554461 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fdd9c-fd5a-4593-8800-48ceef8adcff None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:58.568399 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 517/1046] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:37:58.569791 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34062ed-6c5e-49af-8432-276e52b0872f None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:58.570080 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34062ed-6c5e-49af-8432-276e52b0872f None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:58.570387 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34062ed-6c5e-49af-8432-276e52b0872f None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:58.571842 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 518/1047] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:37:58.580377 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee886374-fa7e-401d-b8ca-8e376b6e05cc None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:58.580914 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee886374-fa7e-401d-b8ca-8e376b6e05cc None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:58.581193 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee886374-fa7e-401d-b8ca-8e376b6e05cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:58.590874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd205c-3e9b-4c29-9be4-3e2d5898462e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:58.591532 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd205c-3e9b-4c29-9be4-3e2d5898462e None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:58.591811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd205c-3e9b-4c29-9be4-3e2d5898462e None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:58.592082 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd205c-3e9b-4c29-9be4-3e2d5898462e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:58.592913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd205c-3e9b-4c29-9be4-3e2d5898462e None admin] RBAC: Adding query filter params (role.id=de6fe15685d647c69e91e3772074a8b1, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=abde7cc02fae4d9bb561a6642c5252a6) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:58.593222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd205c-3e9b-4c29-9be4-3e2d5898462e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:58.595006 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd205c-3e9b-4c29-9be4-3e2d5898462e None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:58.599610 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-ee886374-fa7e-401d-b8ca-8e376b6e05cc None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:37:58.611243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 530/1048] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:58 2022] GET /identity/v3/role_assignments?user.id=abde7cc02fae4d9bb561a6642c5252a6&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=de6fe15685d647c69e91e3772074a8b1 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:58.611471 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-ee886374-fa7e-401d-b8ca-8e376b6e05cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:37:58.611588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd205c-3e9b-4c29-9be4-3e2d5898462e None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:58.623264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:58.685736 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 519/1049] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:37:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:37:58.709880 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee886374-fa7e-401d-b8ca-8e376b6e05cc None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:58.739252 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd205c-3e9b-4c29-9be4-3e2d5898462e None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:58.774616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc98bc1-fd32-493c-98f8-dc0b06849de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:58.775424 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc98bc1-fd32-493c-98f8-dc0b06849de4 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:58.775696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc98bc1-fd32-493c-98f8-dc0b06849de4 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:58.775964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc98bc1-fd32-493c-98f8-dc0b06849de4 None admin] PATH_INFO: `/v3/services/image` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:58.776621 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc98bc1-fd32-493c-98f8-dc0b06849de4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:58.777999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc98bc1-fd32-493c-98f8-dc0b06849de4 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:58.779145 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-7bc98bc1-fd32-493c-98f8-dc0b06849de4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:58.792390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:58.793425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:58.794310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:58.794310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:37:58.794310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:58.794310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:58.794310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:58.794310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:58.794310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:58.794310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:58.794310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:58.794310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:58.794310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:37:58.794310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:37:58.794310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:37:58.794310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:37:58.794310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 531/1050] 10.222.0.84 () {56 vars in 1105 bytes} [Thu May 19 19:37:58 2022] GET /identity/v3/services/image => generated 86 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:37:58.799839 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc98bc1-fd32-493c-98f8-dc0b06849de4 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:58.830614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee886374-fa7e-401d-b8ca-8e376b6e05cc None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:58.867314 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476f5af-6a36-446a-8a36-1eca263d23ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:58.868126 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476f5af-6a36-446a-8a36-1eca263d23ee None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:58.868406 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476f5af-6a36-446a-8a36-1eca263d23ee None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:58.868678 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476f5af-6a36-446a-8a36-1eca263d23ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:58.872049 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:58.879494 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476f5af-6a36-446a-8a36-1eca263d23ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-4476f5af-6a36-446a-8a36-1eca263d23ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:37:58.887187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:37:58.888222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:37:58.889065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:37:58.889065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:37:58.889065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:37:58.889065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:37:58.889065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:37:58.889065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:37:58.889065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:37:58.889065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:37:58.889065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:37:58.889065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:37:58.889065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:37:58.889065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:37:58.889065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 520/1051] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:37:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:37:58.895400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476f5af-6a36-446a-8a36-1eca263d23ee None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:58.918672 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc98bc1-fd32-493c-98f8-dc0b06849de4 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:58.953481 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11311554-a6fb-4904-a020-f55ce3c9ef38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:58.954062 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11311554-a6fb-4904-a020-f55ce3c9ef38 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:58.954370 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11311554-a6fb-4904-a020-f55ce3c9ef38 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:58.954673 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11311554-a6fb-4904-a020-f55ce3c9ef38 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:58.955353 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11311554-a6fb-4904-a020-f55ce3c9ef38 None admin] RBAC: Adding query filter params (name=image) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:58.955660 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11311554-a6fb-4904-a020-f55ce3c9ef38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:58.957043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11311554-a6fb-4904-a020-f55ce3c9ef38 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:58.961993 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 532/1052] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:58.967621 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11311554-a6fb-4904-a020-f55ce3c9ef38 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:59.012711 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476f5af-6a36-446a-8a36-1eca263d23ee None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:59.030506 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:59.058121 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a01eb-b109-4910-8b77-9aef22d9c7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:59.058830 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a01eb-b109-4910-8b77-9aef22d9c7ee None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:59.059112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a01eb-b109-4910-8b77-9aef22d9c7ee None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:59.059386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a01eb-b109-4910-8b77-9aef22d9c7ee None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:59.060136 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a01eb-b109-4910-8b77-9aef22d9c7ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:59.060452 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a01eb-b109-4910-8b77-9aef22d9c7ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:59.061844 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a01eb-b109-4910-8b77-9aef22d9c7ee None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:59.071264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11311554-a6fb-4904-a020-f55ce3c9ef38 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:59.071512 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 521/1053] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:37:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:59.077266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a01eb-b109-4910-8b77-9aef22d9c7ee None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:59.103653 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:59.110566 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157b2d4-b801-472b-ba98-5d3657ffe3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:59.111346 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157b2d4-b801-472b-ba98-5d3657ffe3a6 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:59.111766 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157b2d4-b801-472b-ba98-5d3657ffe3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:59.112020 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157b2d4-b801-472b-ba98-5d3657ffe3a6 None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:59.112753 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f157b2d4-b801-472b-ba98-5d3657ffe3a6 None admin] RBAC: Adding query filter params (type=image) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:59.113077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f157b2d4-b801-472b-ba98-5d3657ffe3a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:59.114526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f157b2d4-b801-472b-ba98-5d3657ffe3a6 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:59.119728 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 533/1054] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:37:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:59.125127 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157b2d4-b801-472b-ba98-5d3657ffe3a6 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:59.188807 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a01eb-b109-4910-8b77-9aef22d9c7ee None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:59.223864 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaae8be-1ca6-44dd-969a-036bc2f9d254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:59.225102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaae8be-1ca6-44dd-969a-036bc2f9d254 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:59.226066 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaae8be-1ca6-44dd-969a-036bc2f9d254 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:59.226372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaae8be-1ca6-44dd-969a-036bc2f9d254 None admin] PATH_INFO: `/v3/users/d5313b0a299d48ccba2fa1f3ab16b99d` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:59.228849 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157b2d4-b801-472b-ba98-5d3657ffe3a6 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:59.241179 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaae8be-1ca6-44dd-969a-036bc2f9d254 None admin] RBAC: Authorizing `identity:get_user(user_id=d5313b0a299d48ccba2fa1f3ab16b99d)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:59.242776 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaae8be-1ca6-44dd-969a-036bc2f9d254 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:59.249319 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 522/1055] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:37:59 2022] GET /identity/v3/users/d5313b0a299d48ccba2fa1f3ab16b99d => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:37:59.250290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaae8be-1ca6-44dd-969a-036bc2f9d254 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:59.263230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4404539-b32f-48bc-82d1-7ae977d2297b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:59.263893 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4404539-b32f-48bc-82d1-7ae977d2297b None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:59.264171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4404539-b32f-48bc-82d1-7ae977d2297b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:59.264440 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4404539-b32f-48bc-82d1-7ae977d2297b None admin] PATH_INFO: `/v3/endpoints` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:59.265177 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4404539-b32f-48bc-82d1-7ae977d2297b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:59.266729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4404539-b32f-48bc-82d1-7ae977d2297b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:59.285862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:59.291903 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 534/1056] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:37:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:37:59.366412 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaae8be-1ca6-44dd-969a-036bc2f9d254 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:59.414538 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c06b9-b316-4a2d-af47-b7c9cdbbdafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:59.415576 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c06b9-b316-4a2d-af47-b7c9cdbbdafe None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:59.416052 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c06b9-b316-4a2d-af47-b7c9cdbbdafe None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:59.416522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c06b9-b316-4a2d-af47-b7c9cdbbdafe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:59.495936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c06b9-b316-4a2d-af47-b7c9cdbbdafe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:59.497323 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c06b9-b316-4a2d-af47-b7c9cdbbdafe None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:59.527738 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:59.598969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 523/1057] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:37:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:37:59.604798 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4404539-b32f-48bc-82d1-7ae977d2297b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:59.664467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:59.721687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4404539-b32f-48bc-82d1-7ae977d2297b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:59.757786 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b4c24-11ef-4a4a-a0b1-316280f03089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:59.758604 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4c24-11ef-4a4a-a0b1-316280f03089 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:59.758878 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4c24-11ef-4a4a-a0b1-316280f03089 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:59.759142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4c24-11ef-4a4a-a0b1-316280f03089 None admin] PATH_INFO: `/v3/projects/1e9cf0a5668746ce94f8f598719cb56b` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:59.760311 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b4c24-11ef-4a4a-a0b1-316280f03089 None admin] RBAC: Authorizing `identity:get_project(project_id=1e9cf0a5668746ce94f8f598719cb56b)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:59.761870 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b4c24-11ef-4a4a-a0b1-316280f03089 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:59.763461 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 535/1058] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:37:59 2022] GET /identity/v3/projects/1e9cf0a5668746ce94f8f598719cb56b => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:37:59.769668 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c06b9-b316-4a2d-af47-b7c9cdbbdafe None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:37:59.871862 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c06b9-b316-4a2d-af47-b7c9cdbbdafe None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:37:59.901163 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:37:59.908153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74b1c3-9ed5-484d-b8a0-ffada264ddad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:37:59.908807 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74b1c3-9ed5-484d-b8a0-ffada264ddad None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:37:59.909083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74b1c3-9ed5-484d-b8a0-ffada264ddad None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:37:59.909352 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74b1c3-9ed5-484d-b8a0-ffada264ddad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:37:59.910174 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a74b1c3-9ed5-484d-b8a0-ffada264ddad None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=1e9cf0a5668746ce94f8f598719cb56b, user.id=d5313b0a299d48ccba2fa1f3ab16b99d) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:37:59.910511 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a74b1c3-9ed5-484d-b8a0-ffada264ddad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:37:59.911928 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a74b1c3-9ed5-484d-b8a0-ffada264ddad None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:37:59.917693 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 524/1059] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:37:59 2022] GET /identity/v3/role_assignments?user.id=d5313b0a299d48ccba2fa1f3ab16b99d&scope.project.id=1e9cf0a5668746ce94f8f598719cb56b&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:00.794706 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1555985-52b7-4733-a95b-73ef68f0036a None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:00.795262 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1555985-52b7-4733-a95b-73ef68f0036a None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:00.795262 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1555985-52b7-4733-a95b-73ef68f0036a None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:00.796882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 536/1060] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:00.801962 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0726-f0f1-49cd-8cae-047f20a01247 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:00.802421 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0726-f0f1-49cd-8cae-047f20a01247 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:00.802499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0726-f0f1-49cd-8cae-047f20a01247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:00.819756 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-408c0726-f0f1-49cd-8cae-047f20a01247 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:00.821106 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-408c0726-f0f1-49cd-8cae-047f20a01247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:00.892405 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 525/1061] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:00.914686 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1555985-52b7-4733-a95b-73ef68f0036a None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:01.026784 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1555985-52b7-4733-a95b-73ef68f0036a None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:01.044153 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:01.063399 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f0b38-d6ef-424b-a168-b7c54e97d8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:01.064499 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f0b38-d6ef-424b-a168-b7c54e97d8ac None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:01.064775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f0b38-d6ef-424b-a168-b7c54e97d8ac None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:01.065043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f0b38-d6ef-424b-a168-b7c54e97d8ac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:01.075708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f0b38-d6ef-424b-a168-b7c54e97d8ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-a77f0b38-d6ef-424b-a168-b7c54e97d8ac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:38:01.083895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:01.084811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:01.090711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:01.090711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:01.090711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:01.090711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:01.090711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:38:01.090711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:38:01.090711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:38:01.090711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:38:01.090711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:38:01.090711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:38:01.090711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:38:01.090711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:38:01.090711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 537/1062] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 19:38:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:01.091740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c0726-f0f1-49cd-8cae-047f20a01247 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:01.195634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c0726-f0f1-49cd-8cae-047f20a01247 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:01.219513 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:01.231680 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d882bd-c71c-496e-859c-dded78adf370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:01.232359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d882bd-c71c-496e-859c-dded78adf370 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:01.232672 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d882bd-c71c-496e-859c-dded78adf370 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:01.232988 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d882bd-c71c-496e-859c-dded78adf370 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:01.233726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d882bd-c71c-496e-859c-dded78adf370 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:01.234134 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d882bd-c71c-496e-859c-dded78adf370 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:01.235538 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d882bd-c71c-496e-859c-dded78adf370 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:01.244813 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 526/1063] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 19:38:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:01.251139 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f0b38-d6ef-424b-a168-b7c54e97d8ac None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:01.352942 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f0b38-d6ef-424b-a168-b7c54e97d8ac None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:01.389777 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164d283-6cb4-4afe-a8cf-c6b80f8b6211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:01.390837 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164d283-6cb4-4afe-a8cf-c6b80f8b6211 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:01.391116 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164d283-6cb4-4afe-a8cf-c6b80f8b6211 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:01.391380 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164d283-6cb4-4afe-a8cf-c6b80f8b6211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:01.401782 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164d283-6cb4-4afe-a8cf-c6b80f8b6211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:01.403263 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164d283-6cb4-4afe-a8cf-c6b80f8b6211 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:01.420810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-2164d283-6cb4-4afe-a8cf-c6b80f8b6211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:38:01.420810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:01.420810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:01.420810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:01.420810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:01.420810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:01.420810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:01.420810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:01.420810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:01.420810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:01.420810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:01.420810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:01.420810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:38:01.420810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:38:01.420810 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:01.421775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:01.422718 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:38:01.422718 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:38:01.422718 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:38:01.422718 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:38:01.426637 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 538/1064] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:01.427917 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d882bd-c71c-496e-859c-dded78adf370 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:01.534743 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d882bd-c71c-496e-859c-dded78adf370 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:01.541700 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ed474-c748-4812-a514-170aba55d5ec None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:01.542178 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ed474-c748-4812-a514-170aba55d5ec None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:01.542656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ed474-c748-4812-a514-170aba55d5ec None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:01.545021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 539/1065] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:01.551189 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6cd61-18e8-4e5b-8667-98a4ca8d2ff0 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:01.551458 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6cd61-18e8-4e5b-8667-98a4ca8d2ff0 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:01.551730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6cd61-18e8-4e5b-8667-98a4ca8d2ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:01.569429 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-ddb6cd61-18e8-4e5b-8667-98a4ca8d2ff0 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:01.570631 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-ddb6cd61-18e8-4e5b-8667-98a4ca8d2ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:01.572337 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5b642-3e9e-457e-9d33-c962b61adf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:01.573164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5b642-3e9e-457e-9d33-c962b61adf55 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:01.574214 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5b642-3e9e-457e-9d33-c962b61adf55 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:01.574531 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5b642-3e9e-457e-9d33-c962b61adf55 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:01.575286 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5b642-3e9e-457e-9d33-c962b61adf55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:01.575591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5b642-3e9e-457e-9d33-c962b61adf55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:01.576957 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5b642-3e9e-457e-9d33-c962b61adf55 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:01.590494 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 527/1066] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:01.596312 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:01.596985 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5b642-3e9e-457e-9d33-c962b61adf55 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:01.645696 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 540/1067] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:38:01.657600 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:01.667333 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6cd61-18e8-4e5b-8667-98a4ca8d2ff0 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:01.718207 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5b642-3e9e-457e-9d33-c962b61adf55 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:01.754747 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cccfd-bd5f-43dc-81c1-be14c09d85b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:01.755539 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cccfd-bd5f-43dc-81c1-be14c09d85b4 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:01.755817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cccfd-bd5f-43dc-81c1-be14c09d85b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:01.756086 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cccfd-bd5f-43dc-81c1-be14c09d85b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:01.763576 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964cccfd-bd5f-43dc-81c1-be14c09d85b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:01.765035 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964cccfd-bd5f-43dc-81c1-be14c09d85b4 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:01.773667 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6cd61-18e8-4e5b-8667-98a4ca8d2ff0 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-964cccfd-bd5f-43dc-81c1-be14c09d85b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:38:01.780083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:01.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:01.783695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:01.783695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:01.783695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:38:01.783695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:38:01.783695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:38:01.783695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:38:01.783695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:38:01.783695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:38:01.783695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:38:01.783695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:38:01.783695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 528/1068] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:38:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:01.786885 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cccfd-bd5f-43dc-81c1-be14c09d85b4 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:01.817351 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8bbd9-5567-4f4d-9115-ce0e8983bb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:01.818104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8bbd9-5567-4f4d-9115-ce0e8983bb41 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:01.818309 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8bbd9-5567-4f4d-9115-ce0e8983bb41 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:01.818596 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8bbd9-5567-4f4d-9115-ce0e8983bb41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:01.848597 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:01.892714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8bbd9-5567-4f4d-9115-ce0e8983bb41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:01.894218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8bbd9-5567-4f4d-9115-ce0e8983bb41 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:01.896448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cccfd-bd5f-43dc-81c1-be14c09d85b4 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:01.905356 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:01.940602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fa29d-747c-46e2-8ad6-812ded98507d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:01.941279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fa29d-747c-46e2-8ad6-812ded98507d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:01.941563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fa29d-747c-46e2-8ad6-812ded98507d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:01.941834 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fa29d-747c-46e2-8ad6-812ded98507d None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:01.942912 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fa29d-747c-46e2-8ad6-812ded98507d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:01.943023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fa29d-747c-46e2-8ad6-812ded98507d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:01.944465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fa29d-747c-46e2-8ad6-812ded98507d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:01.964311 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 529/1069] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:38:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:38:01.969899 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fa29d-747c-46e2-8ad6-812ded98507d None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:02.011260 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 541/1070] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:38:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:38:02.017123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8bbd9-5567-4f4d-9115-ce0e8983bb41 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:02.076766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fa29d-747c-46e2-8ad6-812ded98507d None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:02.084571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:02.112247 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129237c-4691-4a1e-90c9-d3ef9668862c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:02.113045 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129237c-4691-4a1e-90c9-d3ef9668862c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:02.113318 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129237c-4691-4a1e-90c9-d3ef9668862c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:02.113583 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129237c-4691-4a1e-90c9-d3ef9668862c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:02.123448 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129237c-4691-4a1e-90c9-d3ef9668862c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:02.125015 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129237c-4691-4a1e-90c9-d3ef9668862c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:02.126844 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8bbd9-5567-4f4d-9115-ce0e8983bb41 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:02.141852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-a129237c-4691-4a1e-90c9-d3ef9668862c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:38:02.141852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:02.141852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:02.141852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:02.141852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:02.141852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:02.141852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:02.141852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:02.141852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:02.141852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:02.141852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:02.141852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:02.141852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:38:02.141852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:38:02.141852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:02.142817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:02.148873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:38:02.148873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:38:02.148873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:38:02.148873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:38:02.148873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 530/1071] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:02.148873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518e3a4-6bd8-46c8-a7ad-193697344dbe None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:02.148873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518e3a4-6bd8-46c8-a7ad-193697344dbe None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:02.148873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518e3a4-6bd8-46c8-a7ad-193697344dbe None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:02.148873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 531/1072] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:02.149539 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518e3a4-6bd8-46c8-a7ad-193697344dbe None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:02.163995 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980496e1-b59c-4fc0-baf9-c75ed1877d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:02.164782 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980496e1-b59c-4fc0-baf9-c75ed1877d78 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:02.165056 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980496e1-b59c-4fc0-baf9-c75ed1877d78 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:02.165320 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980496e1-b59c-4fc0-baf9-c75ed1877d78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:02.167235 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:02.176149 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980496e1-b59c-4fc0-baf9-c75ed1877d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:02.177602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980496e1-b59c-4fc0-baf9-c75ed1877d78 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:02.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-980496e1-b59c-4fc0-baf9-c75ed1877d78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:38:02.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:02.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:02.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:02.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:02.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:02.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:02.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:02.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:02.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:02.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:02.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:02.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:38:02.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:38:02.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:02.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:02.198139 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:38:02.198139 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:38:02.198139 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:38:02.198139 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:38:02.198139 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 542/1073] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:02.198816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03409eb-1eb3-4d3e-b440-261208a859e2 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:02.198979 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03409eb-1eb3-4d3e-b440-261208a859e2 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:02.199236 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03409eb-1eb3-4d3e-b440-261208a859e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:02.215415 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-e03409eb-1eb3-4d3e-b440-261208a859e2 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:02.216605 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-e03409eb-1eb3-4d3e-b440-261208a859e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:02.256071 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518e3a4-6bd8-46c8-a7ad-193697344dbe None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:02.281720 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 543/1074] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:02.282563 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03409eb-1eb3-4d3e-b440-261208a859e2 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:02.290228 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb7101-8567-4c89-b6e1-7685d80fe2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:02.290911 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb7101-8567-4c89-b6e1-7685d80fe2fe None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:02.291186 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb7101-8567-4c89-b6e1-7685d80fe2fe None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:02.291457 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb7101-8567-4c89-b6e1-7685d80fe2fe None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:02.292173 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb7101-8567-4c89-b6e1-7685d80fe2fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:02.292826 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb7101-8567-4c89-b6e1-7685d80fe2fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:02.293827 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb7101-8567-4c89-b6e1-7685d80fe2fe None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:02.305725 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 532/1075] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:02.306538 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb7101-8567-4c89-b6e1-7685d80fe2fe None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:02.329048 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:02.384167 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:02.387445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03409eb-1eb3-4d3e-b440-261208a859e2 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:02.407603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb7101-8567-4c89-b6e1-7685d80fe2fe None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:02.421135 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9ccf7-ac74-4f1f-a7c3-e18a351e03f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:02.421845 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9ccf7-ac74-4f1f-a7c3-e18a351e03f2 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:02.422114 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9ccf7-ac74-4f1f-a7c3-e18a351e03f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:02.422411 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9ccf7-ac74-4f1f-a7c3-e18a351e03f2 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:02.423134 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9ccf7-ac74-4f1f-a7c3-e18a351e03f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:02.423434 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9ccf7-ac74-4f1f-a7c3-e18a351e03f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:02.424804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9ccf7-ac74-4f1f-a7c3-e18a351e03f2 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:02.436614 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 544/1076] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:02.437423 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9ccf7-ac74-4f1f-a7c3-e18a351e03f2 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:02.455392 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57290114-a37a-4d46-8106-476747352d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:02.456320 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57290114-a37a-4d46-8106-476747352d2c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:02.456666 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57290114-a37a-4d46-8106-476747352d2c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:02.456943 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57290114-a37a-4d46-8106-476747352d2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:02.474366 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57290114-a37a-4d46-8106-476747352d2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-57290114-a37a-4d46-8106-476747352d2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:38:02.482971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:02.484144 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:02.485031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:02.485031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:02.485031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:02.485031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:02.485031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:38:02.485031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:38:02.485031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:38:02.485031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:38:02.485031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:38:02.485031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:38:02.485031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:02.485031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:38:02.487920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 533/1077] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:38:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:02.493921 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57290114-a37a-4d46-8106-476747352d2c None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:02.554059 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9ccf7-ac74-4f1f-a7c3-e18a351e03f2 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:02.555864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:02.597508 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745468e-f257-46b9-addb-c2a77cef1dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:02.598224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745468e-f257-46b9-addb-c2a77cef1dba None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:02.598522 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745468e-f257-46b9-addb-c2a77cef1dba None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:02.598784 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745468e-f257-46b9-addb-c2a77cef1dba None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:02.607424 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745468e-f257-46b9-addb-c2a77cef1dba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:02.609251 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745468e-f257-46b9-addb-c2a77cef1dba None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:02.613866 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57290114-a37a-4d46-8106-476747352d2c None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:02.627624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-3745468e-f257-46b9-addb-c2a77cef1dba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:38:02.627624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:02.627624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:02.627624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:02.627624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:02.627624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:02.627624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:02.627624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:02.627624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:02.627624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:02.627624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:02.627624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:02.627624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:38:02.627624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:38:02.627624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:02.628608 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:02.629479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:38:02.629479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:38:02.629479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:38:02.629479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:38:02.629479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:38:02.629479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:38:02.629479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:38:02.629479 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:38:02.633438 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 545/1078] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:38:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:02.635207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745468e-f257-46b9-addb-c2a77cef1dba None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:02.660536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacd28d-4d0e-4230-9909-3e4c5290bf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:02.661217 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacd28d-4d0e-4230-9909-3e4c5290bf91 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:02.661492 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacd28d-4d0e-4230-9909-3e4c5290bf91 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:02.661761 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacd28d-4d0e-4230-9909-3e4c5290bf91 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:02.662588 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacd28d-4d0e-4230-9909-3e4c5290bf91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:02.662975 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacd28d-4d0e-4230-9909-3e4c5290bf91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:02.664438 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacd28d-4d0e-4230-9909-3e4c5290bf91 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:02.666816 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:02.675956 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 534/1079] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:38:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:38:02.683012 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacd28d-4d0e-4230-9909-3e4c5290bf91 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:02.748266 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745468e-f257-46b9-addb-c2a77cef1dba None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:02.783742 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79893809-fa88-43ba-962b-067073aeb7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:02.784400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79893809-fa88-43ba-962b-067073aeb7e6 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:02.784671 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79893809-fa88-43ba-962b-067073aeb7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:02.784937 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79893809-fa88-43ba-962b-067073aeb7e6 None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:02.785712 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79893809-fa88-43ba-962b-067073aeb7e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:02.786013 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79893809-fa88-43ba-962b-067073aeb7e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:02.787501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79893809-fa88-43ba-962b-067073aeb7e6 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:02.794243 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacd28d-4d0e-4230-9909-3e4c5290bf91 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:02.799546 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 546/1080] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:38:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:02.805392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79893809-fa88-43ba-962b-067073aeb7e6 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:02.828888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:02.829483 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952ff18-5afa-4d0e-91b3-fdc7f1334a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:02.830326 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952ff18-5afa-4d0e-91b3-fdc7f1334a44 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:02.830326 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952ff18-5afa-4d0e-91b3-fdc7f1334a44 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:02.830555 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952ff18-5afa-4d0e-91b3-fdc7f1334a44 None admin] PATH_INFO: `/v3/users/d5313b0a299d48ccba2fa1f3ab16b99d` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:02.831622 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952ff18-5afa-4d0e-91b3-fdc7f1334a44 None admin] RBAC: Authorizing `identity:get_user(user_id=d5313b0a299d48ccba2fa1f3ab16b99d)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:02.833058 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952ff18-5afa-4d0e-91b3-fdc7f1334a44 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:02.834713 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 535/1081] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:38:02 2022] GET /identity/v3/users/d5313b0a299d48ccba2fa1f3ab16b99d => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:02.840913 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952ff18-5afa-4d0e-91b3-fdc7f1334a44 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:02.903317 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79893809-fa88-43ba-962b-067073aeb7e6 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:02.907340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:02.936790 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20cedf-5da1-4711-ac12-ee33dafbe7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:02.937947 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20cedf-5da1-4711-ac12-ee33dafbe7c1 None admin] REQUEST_METHOD: `PUT` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:02.938223 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20cedf-5da1-4711-ac12-ee33dafbe7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:02.938526 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20cedf-5da1-4711-ac12-ee33dafbe7c1 None admin] PATH_INFO: `/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/abde7cc02fae4d9bb561a6642c5252a6/roles/de6fe15685d647c69e91e3772074a8b1` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:02.940537 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952ff18-5afa-4d0e-91b3-fdc7f1334a44 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:02.940907 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20cedf-5da1-4711-ac12-ee33dafbe7c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecf87429758641eeb462b3e409cadc9e, user_id=abde7cc02fae4d9bb561a6642c5252a6, role_id=de6fe15685d647c69e91e3772074a8b1)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:02.942379 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20cedf-5da1-4711-ac12-ee33dafbe7c1 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:02.957420 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 547/1082] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:38:02 2022] PUT /identity/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/abde7cc02fae4d9bb561a6642c5252a6/roles/de6fe15685d647c69e91e3772074a8b1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:38:02.975321 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc7095-3bd8-44b5-a89b-bc8c62b259f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:02.976527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc7095-3bd8-44b5-a89b-bc8c62b259f7 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:02.976799 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc7095-3bd8-44b5-a89b-bc8c62b259f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:02.977067 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc7095-3bd8-44b5-a89b-bc8c62b259f7 None admin] PATH_INFO: `/v3/projects/1e9cf0a5668746ce94f8f598719cb56b` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:02.978456 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc7095-3bd8-44b5-a89b-bc8c62b259f7 None admin] RBAC: Authorizing `identity:get_project(project_id=1e9cf0a5668746ce94f8f598719cb56b)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:02.979944 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc7095-3bd8-44b5-a89b-bc8c62b259f7 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:02.981722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 536/1083] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:38:02 2022] GET /identity/v3/projects/1e9cf0a5668746ce94f8f598719cb56b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:02.987970 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20cedf-5da1-4711-ac12-ee33dafbe7c1 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:03.074300 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:03.094627 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20cedf-5da1-4711-ac12-ee33dafbe7c1 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:03.132869 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30460a1-ea99-4a24-bf7e-60391cd396f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:03.133131 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30460a1-ea99-4a24-bf7e-60391cd396f8 None admin] REQUEST_METHOD: `PUT` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:03.133218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30460a1-ea99-4a24-bf7e-60391cd396f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:03.134254 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30460a1-ea99-4a24-bf7e-60391cd396f8 None admin] PATH_INFO: `/v3/projects/1e9cf0a5668746ce94f8f598719cb56b/users/d5313b0a299d48ccba2fa1f3ab16b99d/roles/a33caf2301fb44db90304eb3d9afc3a5` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:03.136234 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30460a1-ea99-4a24-bf7e-60391cd396f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e9cf0a5668746ce94f8f598719cb56b, user_id=d5313b0a299d48ccba2fa1f3ab16b99d, role_id=a33caf2301fb44db90304eb3d9afc3a5)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:03.137861 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30460a1-ea99-4a24-bf7e-60391cd396f8 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:03.153897 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 548/1084] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:38:02 2022] PUT /identity/v3/projects/1e9cf0a5668746ce94f8f598719cb56b/users/d5313b0a299d48ccba2fa1f3ab16b99d/roles/a33caf2301fb44db90304eb3d9afc3a5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:38:05.148282 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6caa7-fbba-4a07-9bdf-aa3d7d02589d None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:05.148873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6caa7-fbba-4a07-9bdf-aa3d7d02589d None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:05.148873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6caa7-fbba-4a07-9bdf-aa3d7d02589d None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:05.150213 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 537/1085] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:05.156556 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b97fb-ebc2-4b27-a8b1-5500f736a5fe None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:05.157004 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b97fb-ebc2-4b27-a8b1-5500f736a5fe None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:05.157216 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b97fb-ebc2-4b27-a8b1-5500f736a5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:05.175052 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-324b97fb-ebc2-4b27-a8b1-5500f736a5fe None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:05.176232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-324b97fb-ebc2-4b27-a8b1-5500f736a5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:05.247175 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 549/1086] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:05.272113 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6caa7-fbba-4a07-9bdf-aa3d7d02589d None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:05.303538 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12bab8-02b3-4bd7-aff2-a4a795c79574 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:05.303850 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12bab8-02b3-4bd7-aff2-a4a795c79574 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:05.304242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12bab8-02b3-4bd7-aff2-a4a795c79574 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:05.306216 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 550/1087] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:38:05.312505 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc54663-f8d9-4fb8-8f3d-10d8d0fa1397 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:05.312834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc54663-f8d9-4fb8-8f3d-10d8d0fa1397 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:05.313123 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc54663-f8d9-4fb8-8f3d-10d8d0fa1397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:05.330804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-bbc54663-f8d9-4fb8-8f3d-10d8d0fa1397 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:05.331972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-bbc54663-f8d9-4fb8-8f3d-10d8d0fa1397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:05.387234 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6caa7-fbba-4a07-9bdf-aa3d7d02589d None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:05.402109 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 551/1088] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:38:05.420958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95998e-aef1-4311-91c0-282194ac4e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:05.422142 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95998e-aef1-4311-91c0-282194ac4e22 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:05.422460 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95998e-aef1-4311-91c0-282194ac4e22 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:05.422728 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95998e-aef1-4311-91c0-282194ac4e22 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:05.424683 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc54663-f8d9-4fb8-8f3d-10d8d0fa1397 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:05.425058 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:05.426843 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:05.434838 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95998e-aef1-4311-91c0-282194ac4e22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-2a95998e-aef1-4311-91c0-282194ac4e22 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:38:05.443496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:05.444446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:05.445426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:05.445426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:05.445426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:05.445426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:05.445426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:38:05.445426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:38:05.445426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:38:05.445426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:38:05.445426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:38:05.445426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:38:05.445426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:38:05.445426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:38:05.445426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 538/1089] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 19:38:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:05.450634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95998e-aef1-4311-91c0-282194ac4e22 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:05.522063 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc54663-f8d9-4fb8-8f3d-10d8d0fa1397 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:05.550182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95998e-aef1-4311-91c0-282194ac4e22 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:05.555661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e168eb8-93b2-4513-b985-15f3ff302fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:05.556447 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e168eb8-93b2-4513-b985-15f3ff302fac None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:05.556710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e168eb8-93b2-4513-b985-15f3ff302fac None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:05.556958 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e168eb8-93b2-4513-b985-15f3ff302fac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:05.567085 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e168eb8-93b2-4513-b985-15f3ff302fac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-9e168eb8-93b2-4513-b985-15f3ff302fac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:38:05.574824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:05.575832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:05.576661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:05.576661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:05.576661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:05.576661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:05.576661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:38:05.576661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:38:05.576661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:38:05.576661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:38:05.576661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:38:05.576661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:38:05.576661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:05.576661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:38:05.576661 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 552/1090] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:38:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:38:05.582512 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e168eb8-93b2-4513-b985-15f3ff302fac None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:05.586222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d113802-3b3b-44b9-a186-6addca84e8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:05.586935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d113802-3b3b-44b9-a186-6addca84e8c8 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:05.587216 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d113802-3b3b-44b9-a186-6addca84e8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:05.587487 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d113802-3b3b-44b9-a186-6addca84e8c8 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:05.588232 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d113802-3b3b-44b9-a186-6addca84e8c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:05.588539 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d113802-3b3b-44b9-a186-6addca84e8c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:05.589915 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d113802-3b3b-44b9-a186-6addca84e8c8 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:05.598580 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 539/1091] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 19:38:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:38:05.604429 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d113802-3b3b-44b9-a186-6addca84e8c8 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:05.646882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:05.665159 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:05.686525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e168eb8-93b2-4513-b985-15f3ff302fac None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:05.712520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d113802-3b3b-44b9-a186-6addca84e8c8 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:05.721358 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716ae71-f2fa-4b74-a2d5-21951be1a7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:05.722027 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716ae71-f2fa-4b74-a2d5-21951be1a7b8 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:05.722339 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716ae71-f2fa-4b74-a2d5-21951be1a7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:05.722612 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716ae71-f2fa-4b74-a2d5-21951be1a7b8 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:05.723363 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716ae71-f2fa-4b74-a2d5-21951be1a7b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:05.723666 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716ae71-f2fa-4b74-a2d5-21951be1a7b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:05.725042 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716ae71-f2fa-4b74-a2d5-21951be1a7b8 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:05.733571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 553/1092] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:38:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:38:05.739469 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716ae71-f2fa-4b74-a2d5-21951be1a7b8 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:05.747453 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79533d90-abef-43fc-abca-1e5471d63bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:05.748250 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79533d90-abef-43fc-abca-1e5471d63bac None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:05.748530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79533d90-abef-43fc-abca-1e5471d63bac None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:05.748797 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79533d90-abef-43fc-abca-1e5471d63bac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:05.760551 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79533d90-abef-43fc-abca-1e5471d63bac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:05.762020 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79533d90-abef-43fc-abca-1e5471d63bac None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:05.778852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-79533d90-abef-43fc-abca-1e5471d63bac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:38:05.778852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:05.778852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:05.778852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:05.778852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:05.778852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:05.778852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:05.778852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:05.778852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:05.778852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:05.778852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:05.778852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:05.778852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:38:05.778852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:38:05.778852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:05.779920 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:05.781066 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:38:05.781066 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:38:05.781066 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:38:05.781066 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:38:05.781066 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 540/1093] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:05.785875 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79533d90-abef-43fc-abca-1e5471d63bac None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:05.839357 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716ae71-f2fa-4b74-a2d5-21951be1a7b8 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:05.876413 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:05.885925 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79533d90-abef-43fc-abca-1e5471d63bac None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:05.888307 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af660d-8056-4b08-bdea-af8f22651e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:05.889216 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af660d-8056-4b08-bdea-af8f22651e0d None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:05.889489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af660d-8056-4b08-bdea-af8f22651e0d None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:05.889754 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af660d-8056-4b08-bdea-af8f22651e0d None admin] PATH_INFO: `/v3/users/d5313b0a299d48ccba2fa1f3ab16b99d` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:05.891516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af660d-8056-4b08-bdea-af8f22651e0d None admin] RBAC: Authorizing `identity:get_user(user_id=d5313b0a299d48ccba2fa1f3ab16b99d)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:05.892975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af660d-8056-4b08-bdea-af8f22651e0d None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:05.894651 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 554/1094] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:38:05 2022] GET /identity/v3/users/d5313b0a299d48ccba2fa1f3ab16b99d => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:05.895031 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:05.900596 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af660d-8056-4b08-bdea-af8f22651e0d None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:05.921010 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ec49e-17fa-4a25-913f-e1fc715cae18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:05.922079 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ec49e-17fa-4a25-913f-e1fc715cae18 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:05.922407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ec49e-17fa-4a25-913f-e1fc715cae18 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:05.922682 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ec49e-17fa-4a25-913f-e1fc715cae18 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:05.923407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ec49e-17fa-4a25-913f-e1fc715cae18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:05.923715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ec49e-17fa-4a25-913f-e1fc715cae18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:05.925097 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ec49e-17fa-4a25-913f-e1fc715cae18 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:05.941543 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 541/1095] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:38:05.942372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ec49e-17fa-4a25-913f-e1fc715cae18 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:06.005323 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af660d-8056-4b08-bdea-af8f22651e0d None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:06.040310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efec5e-b1dc-4ff0-b137-0d1af72e67ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:06.040956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efec5e-b1dc-4ff0-b137-0d1af72e67ee None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:06.041231 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efec5e-b1dc-4ff0-b137-0d1af72e67ee None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:06.041494 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efec5e-b1dc-4ff0-b137-0d1af72e67ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:06.042037 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ec49e-17fa-4a25-913f-e1fc715cae18 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:06.076906 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004e8c2-91a2-4cbc-9aa7-2276cfa2c34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:06.077734 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004e8c2-91a2-4cbc-9aa7-2276cfa2c34c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:06.078010 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004e8c2-91a2-4cbc-9aa7-2276cfa2c34c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:06.078279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004e8c2-91a2-4cbc-9aa7-2276cfa2c34c None admin] PATH_INFO: `/v3/users/nova` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:06.085334 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004e8c2-91a2-4cbc-9aa7-2276cfa2c34c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:06.086810 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004e8c2-91a2-4cbc-9aa7-2276cfa2c34c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-4004e8c2-91a2-4cbc-9aa7-2276cfa2c34c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:38:06.101591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:06.102654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:06.103923 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:06.103923 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:06.103923 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:38:06.103923 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:38:06.103923 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:38:06.103923 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:38:06.103923 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:38:06.103923 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:38:06.103923 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:38:06.103923 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:38:06.104531 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:06.104615 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 542/1096] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:38:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:38:06.110793 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004e8c2-91a2-4cbc-9aa7-2276cfa2c34c None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:06.118646 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35efec5e-b1dc-4ff0-b137-0d1af72e67ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:06.120113 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35efec5e-b1dc-4ff0-b137-0d1af72e67ee None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:06.139160 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:06.211100 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004e8c2-91a2-4cbc-9aa7-2276cfa2c34c None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:06.221156 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 555/1097] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:38:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:38:06.222207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efec5e-b1dc-4ff0-b137-0d1af72e67ee None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:06.244787 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3289f57-512e-4911-8319-75fde95fe344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:06.245455 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3289f57-512e-4911-8319-75fde95fe344 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:06.245733 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3289f57-512e-4911-8319-75fde95fe344 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:06.246004 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3289f57-512e-4911-8319-75fde95fe344 None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:06.246865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3289f57-512e-4911-8319-75fde95fe344 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:06.247172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3289f57-512e-4911-8319-75fde95fe344 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:06.248601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3289f57-512e-4911-8319-75fde95fe344 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:06.265938 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 543/1098] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:38:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:06.271541 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3289f57-512e-4911-8319-75fde95fe344 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:06.312532 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:06.322522 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efec5e-b1dc-4ff0-b137-0d1af72e67ee None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:06.355403 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:06.357943 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd8a9a-ca25-44b1-9c1e-c99bbaddd1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:06.359023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd8a9a-ca25-44b1-9c1e-c99bbaddd1d1 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:06.359304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd8a9a-ca25-44b1-9c1e-c99bbaddd1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:06.359545 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd8a9a-ca25-44b1-9c1e-c99bbaddd1d1 None admin] PATH_INFO: `/v3/projects/1e9cf0a5668746ce94f8f598719cb56b` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:06.360792 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd8a9a-ca25-44b1-9c1e-c99bbaddd1d1 None admin] RBAC: Authorizing `identity:get_project(project_id=1e9cf0a5668746ce94f8f598719cb56b)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:06.362228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd8a9a-ca25-44b1-9c1e-c99bbaddd1d1 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:06.364124 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 556/1099] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:38:06 2022] GET /identity/v3/projects/1e9cf0a5668746ce94f8f598719cb56b => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:38:06.370074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd8a9a-ca25-44b1-9c1e-c99bbaddd1d1 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:06.376131 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3289f57-512e-4911-8319-75fde95fe344 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:06.411710 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcd364-0cd8-4830-ba9e-a240c2291c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:06.412358 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcd364-0cd8-4830-ba9e-a240c2291c1c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:06.412642 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcd364-0cd8-4830-ba9e-a240c2291c1c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:06.412919 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcd364-0cd8-4830-ba9e-a240c2291c1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:06.469690 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd8a9a-ca25-44b1-9c1e-c99bbaddd1d1 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:06.479574 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcd364-0cd8-4830-ba9e-a240c2291c1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:06.481052 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcd364-0cd8-4830-ba9e-a240c2291c1c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:06.503400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c4d3a-38aa-4ae1-b264-a877de7152a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:06.504047 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c4d3a-38aa-4ae1-b264-a877de7152a0 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:06.504323 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c4d3a-38aa-4ae1-b264-a877de7152a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:06.504594 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c4d3a-38aa-4ae1-b264-a877de7152a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:06.505425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c4d3a-38aa-4ae1-b264-a877de7152a0 None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=1e9cf0a5668746ce94f8f598719cb56b, user.id=d5313b0a299d48ccba2fa1f3ab16b99d) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:06.505732 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c4d3a-38aa-4ae1-b264-a877de7152a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:06.507181 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c4d3a-38aa-4ae1-b264-a877de7152a0 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:06.513060 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 557/1100] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:38:06 2022] GET /identity/v3/role_assignments?user.id=d5313b0a299d48ccba2fa1f3ab16b99d&scope.project.id=1e9cf0a5668746ce94f8f598719cb56b&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:38:06.570954 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:06.588855 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 544/1101] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:38:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:38:06.595060 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c4d3a-38aa-4ae1-b264-a877de7152a0 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:06.624103 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:06.708374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c4d3a-38aa-4ae1-b264-a877de7152a0 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:06.744770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77eaa4-921d-4389-9529-67d74cc2e857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:06.745486 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77eaa4-921d-4389-9529-67d74cc2e857 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:06.745759 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77eaa4-921d-4389-9529-67d74cc2e857 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:06.746024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77eaa4-921d-4389-9529-67d74cc2e857 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:06.756246 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77eaa4-921d-4389-9529-67d74cc2e857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:06.757691 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77eaa4-921d-4389-9529-67d74cc2e857 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:06.774972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-de77eaa4-921d-4389-9529-67d74cc2e857 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:38:06.774972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:06.774972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:06.774972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:06.774972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:06.774972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:06.774972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:06.774972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:06.774972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:06.774972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:06.774972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:06.774972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:06.774972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:38:06.774972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:38:06.774972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:06.775985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:06.777051 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:38:06.777051 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:38:06.777051 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:38:06.777051 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:38:06.777051 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 558/1102] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:06.782740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcd364-0cd8-4830-ba9e-a240c2291c1c None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:06.888390 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcd364-0cd8-4830-ba9e-a240c2291c1c None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:06.925153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b642219-5b6e-40c1-a826-32d47bd3c20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:06.925808 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b642219-5b6e-40c1-a826-32d47bd3c20b None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:06.926091 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b642219-5b6e-40c1-a826-32d47bd3c20b None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:06.926393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b642219-5b6e-40c1-a826-32d47bd3c20b None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:06.927113 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b642219-5b6e-40c1-a826-32d47bd3c20b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:06.927414 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b642219-5b6e-40c1-a826-32d47bd3c20b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:06.928790 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b642219-5b6e-40c1-a826-32d47bd3c20b None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:06.941428 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 545/1103] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:06.947488 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77eaa4-921d-4389-9529-67d74cc2e857 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:07.033128 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:07.058241 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77eaa4-921d-4389-9529-67d74cc2e857 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:07.094284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23787a6b-1dfb-4dbe-8edc-8ae0c2a49c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:07.095101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23787a6b-1dfb-4dbe-8edc-8ae0c2a49c3b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:07.095378 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23787a6b-1dfb-4dbe-8edc-8ae0c2a49c3b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:07.095644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23787a6b-1dfb-4dbe-8edc-8ae0c2a49c3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:07.102884 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23787a6b-1dfb-4dbe-8edc-8ae0c2a49c3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:07.104338 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23787a6b-1dfb-4dbe-8edc-8ae0c2a49c3b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:07.118506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-23787a6b-1dfb-4dbe-8edc-8ae0c2a49c3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:38:07.118506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:07.118506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:07.118506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:07.118506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:07.118506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:07.118506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:07.118506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:07.118506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:07.118506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:07.118506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:07.118506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:07.118506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:38:07.118506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:38:07.118506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:07.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:07.120287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:38:07.120287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:38:07.120287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:38:07.120287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:38:07.120287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:38:07.120287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:38:07.120287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:38:07.120287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:38:07.120287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 559/1104] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:38:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:07.125787 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b642219-5b6e-40c1-a826-32d47bd3c20b None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:07.175409 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:07.232766 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b642219-5b6e-40c1-a826-32d47bd3c20b None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:07.268010 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa73ec9-8121-4aaf-b69e-21da5511435d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:07.268674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa73ec9-8121-4aaf-b69e-21da5511435d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:07.268950 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa73ec9-8121-4aaf-b69e-21da5511435d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:07.269218 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa73ec9-8121-4aaf-b69e-21da5511435d None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:07.269996 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa73ec9-8121-4aaf-b69e-21da5511435d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:07.270301 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa73ec9-8121-4aaf-b69e-21da5511435d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:07.271764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa73ec9-8121-4aaf-b69e-21da5511435d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:07.284567 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 546/1105] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:38:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:07.290655 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23787a6b-1dfb-4dbe-8edc-8ae0c2a49c3b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:07.394000 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23787a6b-1dfb-4dbe-8edc-8ae0c2a49c3b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:07.428368 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ca89b-fc30-483d-9551-25172200e747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:07.429015 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ca89b-fc30-483d-9551-25172200e747 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:07.429289 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ca89b-fc30-483d-9551-25172200e747 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:07.429555 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ca89b-fc30-483d-9551-25172200e747 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:07.430426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ca89b-fc30-483d-9551-25172200e747 None admin] RBAC: Adding query filter params (role.id=de6fe15685d647c69e91e3772074a8b1, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=abde7cc02fae4d9bb561a6642c5252a6) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:07.430741 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ca89b-fc30-483d-9551-25172200e747 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:07.432157 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ca89b-fc30-483d-9551-25172200e747 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:07.435183 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:07.439142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 560/1106] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:38:07 2022] GET /identity/v3/role_assignments?user.id=abde7cc02fae4d9bb561a6642c5252a6&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=de6fe15685d647c69e91e3772074a8b1 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:08.736428 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70089ac1-0c2c-4edf-9737-bfd4ca6baa43 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:08.737296 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70089ac1-0c2c-4edf-9737-bfd4ca6baa43 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:08.737296 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70089ac1-0c2c-4edf-9737-bfd4ca6baa43 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:08.739660 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 547/1107] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:08.747772 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f0ff0-72d5-46b8-87e9-c825cdc5f459 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:08.748227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f0ff0-72d5-46b8-87e9-c825cdc5f459 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:08.748319 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f0ff0-72d5-46b8-87e9-c825cdc5f459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:08.765883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-403f0ff0-72d5-46b8-87e9-c825cdc5f459 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:08.767079 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-403f0ff0-72d5-46b8-87e9-c825cdc5f459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:08.835037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 561/1108] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:38:08.860586 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70089ac1-0c2c-4edf-9737-bfd4ca6baa43 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:08.970629 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70089ac1-0c2c-4edf-9737-bfd4ca6baa43 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:08.978138 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:09.019942 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeae958-72fb-4ecb-90f2-71942e1c9a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:09.020735 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeae958-72fb-4ecb-90f2-71942e1c9a17 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:09.021021 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeae958-72fb-4ecb-90f2-71942e1c9a17 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:09.021275 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeae958-72fb-4ecb-90f2-71942e1c9a17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:09.022684 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeae958-72fb-4ecb-90f2-71942e1c9a17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:09.024127 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeae958-72fb-4ecb-90f2-71942e1c9a17 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:09.025752 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 548/1109] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:38:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:09.032910 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f0ff0-72d5-46b8-87e9-c825cdc5f459 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:09.141536 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f0ff0-72d5-46b8-87e9-c825cdc5f459 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:09.170895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:09.182785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70531f-72bb-424a-8ef1-bc061b0832ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:09.183817 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70531f-72bb-424a-8ef1-bc061b0832ef None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:09.184244 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70531f-72bb-424a-8ef1-bc061b0832ef None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:09.184669 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70531f-72bb-424a-8ef1-bc061b0832ef None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:09.185791 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70531f-72bb-424a-8ef1-bc061b0832ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:09.188631 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70531f-72bb-424a-8ef1-bc061b0832ef None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:09.219448 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 562/1110] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:38:09 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:38:11.526145 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319efe04-45cd-4c68-819f-79ec5f1e80d8 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:11.527305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319efe04-45cd-4c68-819f-79ec5f1e80d8 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:11.527305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319efe04-45cd-4c68-819f-79ec5f1e80d8 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:11.529267 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 549/1111] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:11.535080 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675436d0-6892-4118-9975-9b5e3730a023 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:11.535499 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675436d0-6892-4118-9975-9b5e3730a023 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:11.535606 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675436d0-6892-4118-9975-9b5e3730a023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:11.554057 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-675436d0-6892-4118-9975-9b5e3730a023 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:11.555262 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-675436d0-6892-4118-9975-9b5e3730a023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:11.626246 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 563/1112] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:11.650008 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319efe04-45cd-4c68-819f-79ec5f1e80d8 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:11.756389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319efe04-45cd-4c68-819f-79ec5f1e80d8 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:11.791876 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700ee85-5076-4cfe-b2b8-984eb5a8fb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:11.792721 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700ee85-5076-4cfe-b2b8-984eb5a8fb66 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:11.792999 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700ee85-5076-4cfe-b2b8-984eb5a8fb66 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:11.793268 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700ee85-5076-4cfe-b2b8-984eb5a8fb66 None admin] PATH_INFO: `/v3/roles/4a241925b6cc4218b4f2af580b151877` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:11.794797 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700ee85-5076-4cfe-b2b8-984eb5a8fb66 None admin] RBAC: Authorizing `identity:get_role(role_id=4a241925b6cc4218b4f2af580b151877)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:11.796208 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700ee85-5076-4cfe-b2b8-984eb5a8fb66 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:11.797552 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 550/1113] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:38:11 2022] GET /identity/v3/roles/4a241925b6cc4218b4f2af580b151877 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:11.803840 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675436d0-6892-4118-9975-9b5e3730a023 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:11.917351 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675436d0-6892-4118-9975-9b5e3730a023 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:11.939052 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:11.965683 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33580f1a-8218-4ec9-b539-e9e0586c2ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:11.966594 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33580f1a-8218-4ec9-b539-e9e0586c2ae6 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:11.966871 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33580f1a-8218-4ec9-b539-e9e0586c2ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:11.967143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33580f1a-8218-4ec9-b539-e9e0586c2ae6 None admin] PATH_INFO: `/v3/users/44087cd990274e51b692df6d747a0b87` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:11.983391 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33580f1a-8218-4ec9-b539-e9e0586c2ae6 None admin] RBAC: Authorizing `identity:get_user(user_id=44087cd990274e51b692df6d747a0b87)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:11.984900 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33580f1a-8218-4ec9-b539-e9e0586c2ae6 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:11.986604 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 564/1114] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:38:11 2022] GET /identity/v3/users/44087cd990274e51b692df6d747a0b87 => generated 286 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:11.992842 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700ee85-5076-4cfe-b2b8-984eb5a8fb66 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:12.050769 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:12.110058 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700ee85-5076-4cfe-b2b8-984eb5a8fb66 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:12.144671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbaa90-54ab-4886-bb8c-57d1d2078425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:12.145302 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbaa90-54ab-4886-bb8c-57d1d2078425 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:12.145575 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbaa90-54ab-4886-bb8c-57d1d2078425 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:12.145844 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbaa90-54ab-4886-bb8c-57d1d2078425 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:12.213800 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbaa90-54ab-4886-bb8c-57d1d2078425 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:12.215275 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbaa90-54ab-4886-bb8c-57d1d2078425 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:12.257945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:12.323804 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 551/1115] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:38:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:38:12.330204 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33580f1a-8218-4ec9-b539-e9e0586c2ae6 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:12.437879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33580f1a-8218-4ec9-b539-e9e0586c2ae6 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:12.473988 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b796c-d73d-4fb8-bc14-01c84c419972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:12.474881 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b796c-d73d-4fb8-bc14-01c84c419972 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:12.475119 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b796c-d73d-4fb8-bc14-01c84c419972 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:12.475762 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b796c-d73d-4fb8-bc14-01c84c419972 None admin] PATH_INFO: `/v3/projects/1e9cf0a5668746ce94f8f598719cb56b` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:12.477007 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b796c-d73d-4fb8-bc14-01c84c419972 None admin] RBAC: Authorizing `identity:get_project(project_id=1e9cf0a5668746ce94f8f598719cb56b)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:12.478500 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b796c-d73d-4fb8-bc14-01c84c419972 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:12.480173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 565/1116] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:38:12 2022] GET /identity/v3/projects/1e9cf0a5668746ce94f8f598719cb56b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:12.486585 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbaa90-54ab-4886-bb8c-57d1d2078425 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:12.595928 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbaa90-54ab-4886-bb8c-57d1d2078425 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:12.634751 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9081a-33ac-48c8-9253-a769ce0f52cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:12.635411 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9081a-33ac-48c8-9253-a769ce0f52cb None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:12.635688 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9081a-33ac-48c8-9253-a769ce0f52cb None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:12.635957 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9081a-33ac-48c8-9253-a769ce0f52cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:12.636784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9081a-33ac-48c8-9253-a769ce0f52cb None admin] RBAC: Adding query filter params (role.id=4a241925b6cc4218b4f2af580b151877, scope.project.id=1e9cf0a5668746ce94f8f598719cb56b, user.id=44087cd990274e51b692df6d747a0b87) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:12.637091 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9081a-33ac-48c8-9253-a769ce0f52cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:12.638550 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9081a-33ac-48c8-9253-a769ce0f52cb None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:12.644349 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 552/1117] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:38:12 2022] GET /identity/v3/role_assignments?user.id=44087cd990274e51b692df6d747a0b87&scope.project.id=1e9cf0a5668746ce94f8f598719cb56b&role.id=4a241925b6cc4218b4f2af580b151877 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:14.822724 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067a55d-c424-4d82-8bb8-40f40c74815e None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:14.823597 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067a55d-c424-4d82-8bb8-40f40c74815e None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:14.823597 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067a55d-c424-4d82-8bb8-40f40c74815e None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:14.825775 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 566/1118] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:14.834928 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41a529-c650-4177-8439-8762c1b7bf0f None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:14.835342 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41a529-c650-4177-8439-8762c1b7bf0f None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:14.835528 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41a529-c650-4177-8439-8762c1b7bf0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:14.851739 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:14.857591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-ab41a529-c650-4177-8439-8762c1b7bf0f None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:14.859401 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-ab41a529-c650-4177-8439-8762c1b7bf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:14.932872 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 553/1119] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:14.956839 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067a55d-c424-4d82-8bb8-40f40c74815e None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:15.009875 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:15.075242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067a55d-c424-4d82-8bb8-40f40c74815e None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:15.111984 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2699801-c1fd-4e7d-89bd-3ae9e58712e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:15.112838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2699801-c1fd-4e7d-89bd-3ae9e58712e1 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:15.113113 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2699801-c1fd-4e7d-89bd-3ae9e58712e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:15.113379 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2699801-c1fd-4e7d-89bd-3ae9e58712e1 None admin] PATH_INFO: `/v3/roles/4a241925b6cc4218b4f2af580b151877` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:15.114999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2699801-c1fd-4e7d-89bd-3ae9e58712e1 None admin] RBAC: Authorizing `identity:get_role(role_id=4a241925b6cc4218b4f2af580b151877)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:15.116383 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2699801-c1fd-4e7d-89bd-3ae9e58712e1 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:15.117763 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 567/1120] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:38:14 2022] GET /identity/v3/roles/4a241925b6cc4218b4f2af580b151877 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:15.124306 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41a529-c650-4177-8439-8762c1b7bf0f None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:15.236755 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41a529-c650-4177-8439-8762c1b7bf0f None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:15.273419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e259ebc-37a5-4e66-b377-2ded61212d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:15.274307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e259ebc-37a5-4e66-b377-2ded61212d43 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:15.274572 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e259ebc-37a5-4e66-b377-2ded61212d43 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:15.274844 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e259ebc-37a5-4e66-b377-2ded61212d43 None admin] PATH_INFO: `/v3/users/44087cd990274e51b692df6d747a0b87` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:15.276235 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e259ebc-37a5-4e66-b377-2ded61212d43 None admin] RBAC: Authorizing `identity:get_user(user_id=44087cd990274e51b692df6d747a0b87)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:15.277688 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e259ebc-37a5-4e66-b377-2ded61212d43 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:15.279405 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 554/1121] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:38:15 2022] GET /identity/v3/users/44087cd990274e51b692df6d747a0b87 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:15.288492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2699801-c1fd-4e7d-89bd-3ae9e58712e1 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:15.404363 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2699801-c1fd-4e7d-89bd-3ae9e58712e1 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:15.416400 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:15.444347 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3f1eb-df87-492c-a259-20f9ed2c3a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:15.445206 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3f1eb-df87-492c-a259-20f9ed2c3a1f None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:15.445486 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3f1eb-df87-492c-a259-20f9ed2c3a1f None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:15.445750 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3f1eb-df87-492c-a259-20f9ed2c3a1f None admin] PATH_INFO: `/v3/projects/1e9cf0a5668746ce94f8f598719cb56b` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:15.447000 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3f1eb-df87-492c-a259-20f9ed2c3a1f None admin] RBAC: Authorizing `identity:get_project(project_id=1e9cf0a5668746ce94f8f598719cb56b)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:15.448434 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3f1eb-df87-492c-a259-20f9ed2c3a1f None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:15.450116 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 568/1122] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:38:15 2022] GET /identity/v3/projects/1e9cf0a5668746ce94f8f598719cb56b => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:15.456229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e259ebc-37a5-4e66-b377-2ded61212d43 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:15.460431 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:15.564288 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e259ebc-37a5-4e66-b377-2ded61212d43 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:15.599613 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e53fbc-d460-4d79-908b-088e380d2e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:15.600815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e53fbc-d460-4d79-908b-088e380d2e20 None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:15.601096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e53fbc-d460-4d79-908b-088e380d2e20 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:15.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e53fbc-d460-4d79-908b-088e380d2e20 None admin] PATH_INFO: `/v3/projects/1e9cf0a5668746ce94f8f598719cb56b/users/44087cd990274e51b692df6d747a0b87/roles/4a241925b6cc4218b4f2af580b151877` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:15.603948 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e53fbc-d460-4d79-908b-088e380d2e20 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e9cf0a5668746ce94f8f598719cb56b, user_id=44087cd990274e51b692df6d747a0b87, role_id=4a241925b6cc4218b4f2af580b151877)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:15.605426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e53fbc-d460-4d79-908b-088e380d2e20 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:15.624752 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 555/1123] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:38:15 2022] PUT /identity/v3/projects/1e9cf0a5668746ce94f8f598719cb56b/users/44087cd990274e51b692df6d747a0b87/roles/4a241925b6cc4218b4f2af580b151877 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:38:17.808384 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fa87f-a9a7-4192-b18a-c54c5346ea3f None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:17.809257 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fa87f-a9a7-4192-b18a-c54c5346ea3f None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:17.809257 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fa87f-a9a7-4192-b18a-c54c5346ea3f None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:17.811519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 569/1124] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:17.818093 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230e678-d5ee-42d8-8d73-07701fe86779 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:17.818877 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230e678-d5ee-42d8-8d73-07701fe86779 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:17.819023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230e678-d5ee-42d8-8d73-07701fe86779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:17.847367 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-c230e678-d5ee-42d8-8d73-07701fe86779 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:17.848554 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-c230e678-d5ee-42d8-8d73-07701fe86779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:17.922601 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:17.932731 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 556/1125] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:17.957710 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fa87f-a9a7-4192-b18a-c54c5346ea3f None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:18.072354 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fa87f-a9a7-4192-b18a-c54c5346ea3f None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:18.107808 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede8d74b-715a-404e-9ce0-c2eff83f283b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:18.108755 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8d74b-715a-404e-9ce0-c2eff83f283b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:18.109015 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8d74b-715a-404e-9ce0-c2eff83f283b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:18.109280 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8d74b-715a-404e-9ce0-c2eff83f283b None admin] PATH_INFO: `/v3/roles/4a241925b6cc4218b4f2af580b151877` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:18.110887 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede8d74b-715a-404e-9ce0-c2eff83f283b None admin] RBAC: Authorizing `identity:get_role(role_id=4a241925b6cc4218b4f2af580b151877)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:18.112263 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede8d74b-715a-404e-9ce0-c2eff83f283b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:18.113625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 570/1126] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:38:17 2022] GET /identity/v3/roles/4a241925b6cc4218b4f2af580b151877 => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:18.123145 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230e678-d5ee-42d8-8d73-07701fe86779 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:18.244021 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230e678-d5ee-42d8-8d73-07701fe86779 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:18.288017 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c05a1-21d9-4e17-8e45-1f3d30f8af6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:18.289373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c05a1-21d9-4e17-8e45-1f3d30f8af6e None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:18.289782 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c05a1-21d9-4e17-8e45-1f3d30f8af6e None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:18.290202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c05a1-21d9-4e17-8e45-1f3d30f8af6e None admin] PATH_INFO: `/v3/users/44087cd990274e51b692df6d747a0b87` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:18.292217 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c05a1-21d9-4e17-8e45-1f3d30f8af6e None admin] RBAC: Authorizing `identity:get_user(user_id=44087cd990274e51b692df6d747a0b87)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:18.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c05a1-21d9-4e17-8e45-1f3d30f8af6e None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:18.297082 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 557/1127] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:38:18 2022] GET /identity/v3/users/44087cd990274e51b692df6d747a0b87 => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:18.303067 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede8d74b-715a-404e-9ce0-c2eff83f283b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:18.352449 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:18.408179 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede8d74b-715a-404e-9ce0-c2eff83f283b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:18.443516 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86209b1d-1a41-408f-b168-3b5376288b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:18.444143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86209b1d-1a41-408f-b168-3b5376288b96 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:18.444418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86209b1d-1a41-408f-b168-3b5376288b96 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:18.444679 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86209b1d-1a41-408f-b168-3b5376288b96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:18.513058 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86209b1d-1a41-408f-b168-3b5376288b96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:18.514539 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86209b1d-1a41-408f-b168-3b5376288b96 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:18.562968 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:18.634601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 571/1128] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:38:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:38:18.641242 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c05a1-21d9-4e17-8e45-1f3d30f8af6e None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:18.707508 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:18.748084 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c05a1-21d9-4e17-8e45-1f3d30f8af6e None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:18.783005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756517c4-bfbd-4492-861a-d45aa027db42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:18.783863 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756517c4-bfbd-4492-861a-d45aa027db42 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:18.784133 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756517c4-bfbd-4492-861a-d45aa027db42 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:18.784399 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756517c4-bfbd-4492-861a-d45aa027db42 None admin] PATH_INFO: `/v3/projects/1e9cf0a5668746ce94f8f598719cb56b` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:18.785595 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756517c4-bfbd-4492-861a-d45aa027db42 None admin] RBAC: Authorizing `identity:get_project(project_id=1e9cf0a5668746ce94f8f598719cb56b)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:18.787058 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756517c4-bfbd-4492-861a-d45aa027db42 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:18.788714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 558/1129] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:38:18 2022] GET /identity/v3/projects/1e9cf0a5668746ce94f8f598719cb56b => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:18.795676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86209b1d-1a41-408f-b168-3b5376288b96 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:18.929447 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86209b1d-1a41-408f-b168-3b5376288b96 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:18.970381 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef2798-280b-450a-a70e-091be9f70173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:18.971022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef2798-280b-450a-a70e-091be9f70173 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:18.971302 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef2798-280b-450a-a70e-091be9f70173 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:18.971571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef2798-280b-450a-a70e-091be9f70173 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:18.972401 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef2798-280b-450a-a70e-091be9f70173 None admin] RBAC: Adding query filter params (role.id=4a241925b6cc4218b4f2af580b151877, scope.project.id=1e9cf0a5668746ce94f8f598719cb56b, user.id=44087cd990274e51b692df6d747a0b87) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:18.972708 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef2798-280b-450a-a70e-091be9f70173 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:18.974147 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef2798-280b-450a-a70e-091be9f70173 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:18.980633 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 572/1130] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:38:18 2022] GET /identity/v3/role_assignments?user.id=44087cd990274e51b692df6d747a0b87&scope.project.id=1e9cf0a5668746ce94f8f598719cb56b&role.id=4a241925b6cc4218b4f2af580b151877 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:21.270593 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826517d3-e270-4ef2-978d-d59d4b328ec1 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:21.271158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826517d3-e270-4ef2-978d-d59d4b328ec1 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:21.271158 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826517d3-e270-4ef2-978d-d59d4b328ec1 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:21.273234 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 559/1131] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:21.281370 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448619e6-d0de-4a64-a431-e7caa25f6d84 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:21.281834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448619e6-d0de-4a64-a431-e7caa25f6d84 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:21.281938 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448619e6-d0de-4a64-a431-e7caa25f6d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:21.300532 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-448619e6-d0de-4a64-a431-e7caa25f6d84 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:21.301714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-448619e6-d0de-4a64-a431-e7caa25f6d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:21.304589 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:21.372874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 573/1132] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:38:21.396528 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826517d3-e270-4ef2-978d-d59d4b328ec1 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:21.511472 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826517d3-e270-4ef2-978d-d59d4b328ec1 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:21.542275 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:21.549727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f635c8a-2b5f-44f1-84fd-b1921086c7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:21.550576 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f635c8a-2b5f-44f1-84fd-b1921086c7f1 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:21.550852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f635c8a-2b5f-44f1-84fd-b1921086c7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:21.551129 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f635c8a-2b5f-44f1-84fd-b1921086c7f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:21.552443 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f635c8a-2b5f-44f1-84fd-b1921086c7f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:21.553895 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f635c8a-2b5f-44f1-84fd-b1921086c7f1 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:21.555555 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 560/1133] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:38:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:21.562155 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448619e6-d0de-4a64-a431-e7caa25f6d84 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:21.670177 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448619e6-d0de-4a64-a431-e7caa25f6d84 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:21.706042 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005fb41-a4b5-4a8e-a4fc-9368648c5bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:21.706717 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005fb41-a4b5-4a8e-a4fc-9368648c5bfd None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:21.706995 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005fb41-a4b5-4a8e-a4fc-9368648c5bfd None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:21.707265 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005fb41-a4b5-4a8e-a4fc-9368648c5bfd None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:21.707968 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005fb41-a4b5-4a8e-a4fc-9368648c5bfd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:21.709404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005fb41-a4b5-4a8e-a4fc-9368648c5bfd None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:21.719042 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:21.742415 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 574/1134] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:38:21 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:38:23.953476 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6157c-6649-479b-8927-f472e14eb8d3 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:23.954383 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6157c-6649-479b-8927-f472e14eb8d3 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:23.954383 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6157c-6649-479b-8927-f472e14eb8d3 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:23.956421 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 561/1135] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:23.964455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c8f72-a8d5-4ddb-a5fb-426734dff003 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:23.964889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c8f72-a8d5-4ddb-a5fb-426734dff003 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:23.965002 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c8f72-a8d5-4ddb-a5fb-426734dff003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:23.983332 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-761c8f72-a8d5-4ddb-a5fb-426734dff003 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:23.984487 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-761c8f72-a8d5-4ddb-a5fb-426734dff003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:24.059692 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 575/1136] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:38:24.102351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6157c-6649-479b-8927-f472e14eb8d3 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:24.219266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6157c-6649-479b-8927-f472e14eb8d3 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:24.254737 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac8cee-2d8a-4a18-8b07-c076fe156816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:24.255519 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac8cee-2d8a-4a18-8b07-c076fe156816 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:24.255791 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac8cee-2d8a-4a18-8b07-c076fe156816 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:24.256056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac8cee-2d8a-4a18-8b07-c076fe156816 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:24.257277 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac8cee-2d8a-4a18-8b07-c076fe156816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:24.258742 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac8cee-2d8a-4a18-8b07-c076fe156816 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:24.260325 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 562/1137] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:38:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:24.266530 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c8f72-a8d5-4ddb-a5fb-426734dff003 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:24.388200 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c8f72-a8d5-4ddb-a5fb-426734dff003 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:24.411261 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:24.424907 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88654c3c-1213-4aa2-b4f6-14ecdb11f3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:24.425569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88654c3c-1213-4aa2-b4f6-14ecdb11f3ac None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:24.425853 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88654c3c-1213-4aa2-b4f6-14ecdb11f3ac None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:24.426117 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88654c3c-1213-4aa2-b4f6-14ecdb11f3ac None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:24.426968 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88654c3c-1213-4aa2-b4f6-14ecdb11f3ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:24.428410 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88654c3c-1213-4aa2-b4f6-14ecdb11f3ac None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:24.451158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 576/1138] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:38:24 2022] POST /identity/v3/users => generated 286 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:38:26.683075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7671184-0602-45f9-a7d1-982082498111 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:26.683873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7671184-0602-45f9-a7d1-982082498111 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:26.683873 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7671184-0602-45f9-a7d1-982082498111 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:26.686289 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 563/1139] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:26.691489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780faa9e-47d0-4b9a-9c1d-169db5bad6b7 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:26.691908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780faa9e-47d0-4b9a-9c1d-169db5bad6b7 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:26.691996 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780faa9e-47d0-4b9a-9c1d-169db5bad6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:26.710884 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-780faa9e-47d0-4b9a-9c1d-169db5bad6b7 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:26.712043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-780faa9e-47d0-4b9a-9c1d-169db5bad6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:26.786559 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 577/1140] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:26.810169 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7671184-0602-45f9-a7d1-982082498111 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:26.889787 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:26.933322 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7671184-0602-45f9-a7d1-982082498111 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:26.972360 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c4cef-1481-408f-a760-30c2a8718f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:26.973151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c4cef-1481-408f-a760-30c2a8718f4f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:26.973459 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c4cef-1481-408f-a760-30c2a8718f4f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:26.973731 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c4cef-1481-408f-a760-30c2a8718f4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:26.985918 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c4cef-1481-408f-a760-30c2a8718f4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-f97c4cef-1481-408f-a760-30c2a8718f4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:38:26.993446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:26.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:26.995505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:26.995505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:26.995505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:26.995505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:26.995505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:38:26.995505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:38:26.995505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:38:26.995505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:38:26.995505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:38:26.995505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:38:26.995505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:26.995505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:38:26.995505 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 564/1141] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:38:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:27.001916 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780faa9e-47d0-4b9a-9c1d-169db5bad6b7 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:27.119542 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780faa9e-47d0-4b9a-9c1d-169db5bad6b7 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:27.129111 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:27.156847 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711252fa-362e-4dbd-a5ef-61e7f9c16b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:27.157443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711252fa-362e-4dbd-a5ef-61e7f9c16b00 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:27.157723 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711252fa-362e-4dbd-a5ef-61e7f9c16b00 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:27.157992 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711252fa-362e-4dbd-a5ef-61e7f9c16b00 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:27.158782 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711252fa-362e-4dbd-a5ef-61e7f9c16b00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:27.159088 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711252fa-362e-4dbd-a5ef-61e7f9c16b00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:27.160465 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711252fa-362e-4dbd-a5ef-61e7f9c16b00 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:27.168689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 578/1142] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:38:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:27.175474 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c4cef-1481-408f-a760-30c2a8718f4f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:27.297743 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c4cef-1481-408f-a760-30c2a8718f4f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:27.323251 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:27.332187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a34b8-d636-4fc0-a57b-252d995bad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:27.333066 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a34b8-d636-4fc0-a57b-252d995bad24 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:27.333345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a34b8-d636-4fc0-a57b-252d995bad24 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:27.333623 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a34b8-d636-4fc0-a57b-252d995bad24 None admin] PATH_INFO: `/v3/users/1b97c263a8e54b1d963eb4e967cd42a5` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:27.346614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a34b8-d636-4fc0-a57b-252d995bad24 None admin] RBAC: Authorizing `identity:get_user(user_id=1b97c263a8e54b1d963eb4e967cd42a5)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:27.348058 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a34b8-d636-4fc0-a57b-252d995bad24 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:27.349706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 565/1143] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:38:27 2022] GET /identity/v3/users/1b97c263a8e54b1d963eb4e967cd42a5 => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:27.356443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711252fa-362e-4dbd-a5ef-61e7f9c16b00 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:27.463298 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711252fa-362e-4dbd-a5ef-61e7f9c16b00 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:27.500021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd229c8f-6bfa-4571-bfed-f9fc3f6b1d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:27.500652 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd229c8f-6bfa-4571-bfed-f9fc3f6b1d3b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:27.500926 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd229c8f-6bfa-4571-bfed-f9fc3f6b1d3b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:27.501192 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd229c8f-6bfa-4571-bfed-f9fc3f6b1d3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:27.543764 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:27.579038 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd229c8f-6bfa-4571-bfed-f9fc3f6b1d3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:27.580916 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd229c8f-6bfa-4571-bfed-f9fc3f6b1d3b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:27.691063 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 579/1144] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:38:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:38:27.697225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a34b8-d636-4fc0-a57b-252d995bad24 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:27.803740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a34b8-d636-4fc0-a57b-252d995bad24 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:27.840077 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260231f7-7d4f-468e-b6b0-ccbb9ab4d3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:27.840935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260231f7-7d4f-468e-b6b0-ccbb9ab4d3cb None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:27.841216 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260231f7-7d4f-468e-b6b0-ccbb9ab4d3cb None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:27.841473 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260231f7-7d4f-468e-b6b0-ccbb9ab4d3cb None admin] PATH_INFO: `/v3/projects/ca00c668888f4c1a93ac844c901608cc` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:27.842832 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260231f7-7d4f-468e-b6b0-ccbb9ab4d3cb None admin] RBAC: Authorizing `identity:get_project(project_id=ca00c668888f4c1a93ac844c901608cc)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:27.844576 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260231f7-7d4f-468e-b6b0-ccbb9ab4d3cb None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:27.846239 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 566/1145] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:38:27 2022] GET /identity/v3/projects/ca00c668888f4c1a93ac844c901608cc => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:27.856365 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd229c8f-6bfa-4571-bfed-f9fc3f6b1d3b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:27.930722 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:27.965450 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd229c8f-6bfa-4571-bfed-f9fc3f6b1d3b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:28.002966 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2be998-80e2-4b9e-b35d-51e48e935146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:28.003617 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2be998-80e2-4b9e-b35d-51e48e935146 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:28.003890 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2be998-80e2-4b9e-b35d-51e48e935146 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:28.004155 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2be998-80e2-4b9e-b35d-51e48e935146 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:28.004989 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2be998-80e2-4b9e-b35d-51e48e935146 None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=ca00c668888f4c1a93ac844c901608cc, user.id=1b97c263a8e54b1d963eb4e967cd42a5) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:28.005295 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2be998-80e2-4b9e-b35d-51e48e935146 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:28.006768 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2be998-80e2-4b9e-b35d-51e48e935146 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:28.012336 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 580/1146] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:38:27 2022] GET /identity/v3/role_assignments?user.id=1b97c263a8e54b1d963eb4e967cd42a5&scope.project.id=ca00c668888f4c1a93ac844c901608cc&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:30.229450 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703e8ed-1b84-41e8-97e9-c72f20be4019 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:30.230385 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703e8ed-1b84-41e8-97e9-c72f20be4019 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:30.230385 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703e8ed-1b84-41e8-97e9-c72f20be4019 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:30.232538 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 567/1147] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:30.238346 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e890aa4-bab2-47bf-ad9d-be5bbf16f818 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:30.238764 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e890aa4-bab2-47bf-ad9d-be5bbf16f818 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:30.238881 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e890aa4-bab2-47bf-ad9d-be5bbf16f818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:30.257562 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-7e890aa4-bab2-47bf-ad9d-be5bbf16f818 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:30.258777 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-7e890aa4-bab2-47bf-ad9d-be5bbf16f818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:30.328913 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 581/1148] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:30.352042 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703e8ed-1b84-41e8-97e9-c72f20be4019 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:30.413659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:30.466119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703e8ed-1b84-41e8-97e9-c72f20be4019 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:30.505132 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb319908-b2e3-49f4-b3f1-202f5198a0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:30.505850 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb319908-b2e3-49f4-b3f1-202f5198a0cf None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:30.506125 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb319908-b2e3-49f4-b3f1-202f5198a0cf None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:30.506421 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb319908-b2e3-49f4-b3f1-202f5198a0cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:30.517944 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb319908-b2e3-49f4-b3f1-202f5198a0cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-fb319908-b2e3-49f4-b3f1-202f5198a0cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:38:30.525485 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:30.526602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:30.527490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:30.527490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:30.527490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:30.527490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:30.527490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:38:30.527490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:38:30.527490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:38:30.527490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:38:30.527490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:38:30.527490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:38:30.527490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:30.527490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:38:30.527490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 568/1149] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:38:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:30.533623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e890aa4-bab2-47bf-ad9d-be5bbf16f818 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:30.608099 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:30.652843 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e890aa4-bab2-47bf-ad9d-be5bbf16f818 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:30.687988 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a2ddc-6060-4fe4-9e08-bd88af48d14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:30.688645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a2ddc-6060-4fe4-9e08-bd88af48d14f None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:30.688917 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a2ddc-6060-4fe4-9e08-bd88af48d14f None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:30.689182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a2ddc-6060-4fe4-9e08-bd88af48d14f None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:30.689930 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a2ddc-6060-4fe4-9e08-bd88af48d14f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:30.690227 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a2ddc-6060-4fe4-9e08-bd88af48d14f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:30.691641 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a2ddc-6060-4fe4-9e08-bd88af48d14f None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:30.699670 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 582/1150] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:38:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:30.705975 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb319908-b2e3-49f4-b3f1-202f5198a0cf None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:30.812839 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb319908-b2e3-49f4-b3f1-202f5198a0cf None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:30.825543 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:30.848810 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f5f25-32a6-43bd-893b-543ee4bd19fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:30.849677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f5f25-32a6-43bd-893b-543ee4bd19fc None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:30.849955 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f5f25-32a6-43bd-893b-543ee4bd19fc None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:30.850578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f5f25-32a6-43bd-893b-543ee4bd19fc None admin] PATH_INFO: `/v3/users/1b97c263a8e54b1d963eb4e967cd42a5` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:30.851875 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f5f25-32a6-43bd-893b-543ee4bd19fc None admin] RBAC: Authorizing `identity:get_user(user_id=1b97c263a8e54b1d963eb4e967cd42a5)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:30.853317 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f5f25-32a6-43bd-893b-543ee4bd19fc None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:30.854958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 569/1151] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:38:30 2022] GET /identity/v3/users/1b97c263a8e54b1d963eb4e967cd42a5 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:30.862058 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a2ddc-6060-4fe4-9e08-bd88af48d14f None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:30.964643 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a2ddc-6060-4fe4-9e08-bd88af48d14f None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:30.991451 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:31.005982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec6591-a559-489c-9ef0-2b0af6ad31f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:31.007290 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec6591-a559-489c-9ef0-2b0af6ad31f2 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:31.007657 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec6591-a559-489c-9ef0-2b0af6ad31f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:31.008028 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec6591-a559-489c-9ef0-2b0af6ad31f2 None admin] PATH_INFO: `/v3/projects/ca00c668888f4c1a93ac844c901608cc` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:31.010065 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec6591-a559-489c-9ef0-2b0af6ad31f2 None admin] RBAC: Authorizing `identity:get_project(project_id=ca00c668888f4c1a93ac844c901608cc)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:31.012396 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec6591-a559-489c-9ef0-2b0af6ad31f2 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:31.015011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 583/1152] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:38:30 2022] GET /identity/v3/projects/ca00c668888f4c1a93ac844c901608cc => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:31.021865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f5f25-32a6-43bd-893b-543ee4bd19fc None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:31.138291 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f5f25-32a6-43bd-893b-543ee4bd19fc None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:31.174988 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cda865-1c05-4a27-be83-ecf6f19118d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:31.175864 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cda865-1c05-4a27-be83-ecf6f19118d3 None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:31.176140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cda865-1c05-4a27-be83-ecf6f19118d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:31.176409 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cda865-1c05-4a27-be83-ecf6f19118d3 None admin] PATH_INFO: `/v3/projects/ca00c668888f4c1a93ac844c901608cc/users/1b97c263a8e54b1d963eb4e967cd42a5/roles/a33caf2301fb44db90304eb3d9afc3a5` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:31.178627 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cda865-1c05-4a27-be83-ecf6f19118d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca00c668888f4c1a93ac844c901608cc, user_id=1b97c263a8e54b1d963eb4e967cd42a5, role_id=a33caf2301fb44db90304eb3d9afc3a5)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:31.180058 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cda865-1c05-4a27-be83-ecf6f19118d3 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:31.201298 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 570/1153] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:38:31 2022] PUT /identity/v3/projects/ca00c668888f4c1a93ac844c901608cc/users/1b97c263a8e54b1d963eb4e967cd42a5/roles/a33caf2301fb44db90304eb3d9afc3a5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:38:33.452136 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a74a16-8e52-405e-aa6e-0347465c8f11 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:33.453144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a74a16-8e52-405e-aa6e-0347465c8f11 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:33.453144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a74a16-8e52-405e-aa6e-0347465c8f11 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:33.455390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 584/1154] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:33.464756 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849372d-1e67-44fa-8739-a5a982bc8125 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:33.465478 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849372d-1e67-44fa-8739-a5a982bc8125 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:33.465645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849372d-1e67-44fa-8739-a5a982bc8125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:33.486016 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-8849372d-1e67-44fa-8739-a5a982bc8125 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:33.487237 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-8849372d-1e67-44fa-8739-a5a982bc8125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:33.527351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:33.565187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 571/1155] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:33.589754 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a74a16-8e52-405e-aa6e-0347465c8f11 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:33.702434 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a74a16-8e52-405e-aa6e-0347465c8f11 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:33.738833 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45e8dc-3f98-49ab-b4b3-513aeb8ab901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:33.739626 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45e8dc-3f98-49ab-b4b3-513aeb8ab901 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:33.739909 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45e8dc-3f98-49ab-b4b3-513aeb8ab901 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:33.740177 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45e8dc-3f98-49ab-b4b3-513aeb8ab901 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:33.751795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45e8dc-3f98-49ab-b4b3-513aeb8ab901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-fd45e8dc-3f98-49ab-b4b3-513aeb8ab901 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:38:33.759515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:33.760540 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:33.761475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:33.761475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:33.761475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:33.761475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:33.761475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:38:33.761475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:38:33.761475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:38:33.761475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:38:33.761475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:38:33.761475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:38:33.761475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:33.761475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:38:33.761475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 585/1156] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:38:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:33.767329 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8849372d-1e67-44fa-8739-a5a982bc8125 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:33.872535 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8849372d-1e67-44fa-8739-a5a982bc8125 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:33.910148 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee07e2-ee12-4a2f-afaa-ba3c41f42491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:33.910840 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee07e2-ee12-4a2f-afaa-ba3c41f42491 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:33.911128 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee07e2-ee12-4a2f-afaa-ba3c41f42491 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:33.911401 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee07e2-ee12-4a2f-afaa-ba3c41f42491 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:33.912142 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee07e2-ee12-4a2f-afaa-ba3c41f42491 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:33.912449 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee07e2-ee12-4a2f-afaa-ba3c41f42491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:33.913829 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee07e2-ee12-4a2f-afaa-ba3c41f42491 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:33.922317 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 572/1157] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:38:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:33.928539 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45e8dc-3f98-49ab-b4b3-513aeb8ab901 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:33.988631 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:34.035114 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45e8dc-3f98-49ab-b4b3-513aeb8ab901 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:34.070781 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4552533-76b9-4717-95a2-63ceb6c38ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:34.071650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4552533-76b9-4717-95a2-63ceb6c38ec7 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:34.071926 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4552533-76b9-4717-95a2-63ceb6c38ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:34.072194 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4552533-76b9-4717-95a2-63ceb6c38ec7 None admin] PATH_INFO: `/v3/users/1b97c263a8e54b1d963eb4e967cd42a5` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:34.073506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4552533-76b9-4717-95a2-63ceb6c38ec7 None admin] RBAC: Authorizing `identity:get_user(user_id=1b97c263a8e54b1d963eb4e967cd42a5)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:34.074989 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4552533-76b9-4717-95a2-63ceb6c38ec7 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:34.076728 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 586/1158] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:38:33 2022] GET /identity/v3/users/1b97c263a8e54b1d963eb4e967cd42a5 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:34.083023 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee07e2-ee12-4a2f-afaa-ba3c41f42491 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:34.119717 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:34.189768 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee07e2-ee12-4a2f-afaa-ba3c41f42491 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:34.226002 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cb081-66c2-4412-9ac2-da58cf34d4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:34.226683 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cb081-66c2-4412-9ac2-da58cf34d4e5 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:34.226964 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cb081-66c2-4412-9ac2-da58cf34d4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:34.227236 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cb081-66c2-4412-9ac2-da58cf34d4e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:34.298356 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cb081-66c2-4412-9ac2-da58cf34d4e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:34.299814 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cb081-66c2-4412-9ac2-da58cf34d4e5 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:34.330469 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:34.402251 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 573/1159] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:38:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:38:34.409089 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4552533-76b9-4717-95a2-63ceb6c38ec7 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:34.528270 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4552533-76b9-4717-95a2-63ceb6c38ec7 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:34.552509 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:34.565609 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa9b85-c93c-48da-b972-7437b44b4876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:34.566490 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa9b85-c93c-48da-b972-7437b44b4876 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:34.566759 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa9b85-c93c-48da-b972-7437b44b4876 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:34.567018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa9b85-c93c-48da-b972-7437b44b4876 None admin] PATH_INFO: `/v3/projects/ca00c668888f4c1a93ac844c901608cc` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:34.568299 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa9b85-c93c-48da-b972-7437b44b4876 None admin] RBAC: Authorizing `identity:get_project(project_id=ca00c668888f4c1a93ac844c901608cc)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:34.569757 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa9b85-c93c-48da-b972-7437b44b4876 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:34.571471 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 587/1160] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:38:34 2022] GET /identity/v3/projects/ca00c668888f4c1a93ac844c901608cc => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:34.578118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cb081-66c2-4412-9ac2-da58cf34d4e5 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:34.687546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cb081-66c2-4412-9ac2-da58cf34d4e5 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:34.721385 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:34.731500 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aca3fa-2713-4170-9149-bb5c052563ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:34.732652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aca3fa-2713-4170-9149-bb5c052563ec None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:34.733175 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aca3fa-2713-4170-9149-bb5c052563ec None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:34.733603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aca3fa-2713-4170-9149-bb5c052563ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:34.734341 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aca3fa-2713-4170-9149-bb5c052563ec None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=ca00c668888f4c1a93ac844c901608cc, user.id=1b97c263a8e54b1d963eb4e967cd42a5) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:34.734654 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aca3fa-2713-4170-9149-bb5c052563ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:34.736090 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aca3fa-2713-4170-9149-bb5c052563ec None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:34.743025 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 574/1161] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:38:34 2022] GET /identity/v3/role_assignments?user.id=1b97c263a8e54b1d963eb4e967cd42a5&scope.project.id=ca00c668888f4c1a93ac844c901608cc&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:36.978038 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c128255-43a8-4112-8012-cb59905b67cc None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:36.978943 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c128255-43a8-4112-8012-cb59905b67cc None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:36.978943 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c128255-43a8-4112-8012-cb59905b67cc None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:36.981331 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 588/1162] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:36.990629 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946450e-cfe4-4071-afb8-166afa1c19cc None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:36.991094 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946450e-cfe4-4071-afb8-166afa1c19cc None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:36.991195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946450e-cfe4-4071-afb8-166afa1c19cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:37.017709 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-6946450e-cfe4-4071-afb8-166afa1c19cc None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:37.019663 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-6946450e-cfe4-4071-afb8-166afa1c19cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:37.099482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 575/1163] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:37.124993 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c128255-43a8-4112-8012-cb59905b67cc None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:37.239483 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c128255-43a8-4112-8012-cb59905b67cc None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:37.274480 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd5a51-83fa-4b3e-8569-b5cdb0290934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:37.275107 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd5a51-83fa-4b3e-8569-b5cdb0290934 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:37.275382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd5a51-83fa-4b3e-8569-b5cdb0290934 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:37.275646 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd5a51-83fa-4b3e-8569-b5cdb0290934 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:37.334636 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:37.346956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd5a51-83fa-4b3e-8569-b5cdb0290934 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:37.348424 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd5a51-83fa-4b3e-8569-b5cdb0290934 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:37.463043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 589/1164] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:38:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:38:37.471557 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946450e-cfe4-4071-afb8-166afa1c19cc None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:37.584106 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946450e-cfe4-4071-afb8-166afa1c19cc None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:37.611457 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:37.624714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ef45d-ff33-4073-a671-92eaf0a76982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:37.625974 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ef45d-ff33-4073-a671-92eaf0a76982 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:37.626479 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ef45d-ff33-4073-a671-92eaf0a76982 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:37.626926 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ef45d-ff33-4073-a671-92eaf0a76982 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:37.639470 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ef45d-ff33-4073-a671-92eaf0a76982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:37.641215 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ef45d-ff33-4073-a671-92eaf0a76982 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-d22ef45d-ff33-4073-a671-92eaf0a76982 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:37.656705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:37.657958 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:37.659164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:37.659164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:38:37.659164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:37.659164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:37.659164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:37.659164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:37.659164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:37.659164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:37.659164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:37.659164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:37.659164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 19:38:37.659164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:38:37.659164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 19:38:37.659164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:38:37.662446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 576/1165] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:38:37 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:37.664456 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd5a51-83fa-4b3e-8569-b5cdb0290934 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:37.768106 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:37.779544 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd5a51-83fa-4b3e-8569-b5cdb0290934 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:37.816233 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5700ae-0dab-4c7c-ab58-fb200cbf166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:37.816879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5700ae-0dab-4c7c-ab58-fb200cbf166b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:37.817158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5700ae-0dab-4c7c-ab58-fb200cbf166b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:37.817424 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5700ae-0dab-4c7c-ab58-fb200cbf166b None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:37.818121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5700ae-0dab-4c7c-ab58-fb200cbf166b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:37.818447 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5700ae-0dab-4c7c-ab58-fb200cbf166b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:37.819812 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5700ae-0dab-4c7c-ab58-fb200cbf166b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:37.826965 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 590/1166] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 19:38:37 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:40.043600 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0875b3-726d-4c93-b4af-3bd9cfb23942 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:40.044174 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0875b3-726d-4c93-b4af-3bd9cfb23942 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:40.044174 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0875b3-726d-4c93-b4af-3bd9cfb23942 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:40.045762 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 577/1167] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:40.052203 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335655c9-a207-408f-b5b9-371d00071aa2 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:40.052882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335655c9-a207-408f-b5b9-371d00071aa2 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:40.053055 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335655c9-a207-408f-b5b9-371d00071aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:40.080045 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-335655c9-a207-408f-b5b9-371d00071aa2 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:40.081243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-335655c9-a207-408f-b5b9-371d00071aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:40.154014 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 591/1168] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:40.178451 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0875b3-726d-4c93-b4af-3bd9cfb23942 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:40.289015 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0875b3-726d-4c93-b4af-3bd9cfb23942 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:40.325640 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c278d-952b-4def-bf83-4aa2025a15bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:40.326242 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c278d-952b-4def-bf83-4aa2025a15bb None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:40.326557 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c278d-952b-4def-bf83-4aa2025a15bb None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:40.326827 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c278d-952b-4def-bf83-4aa2025a15bb None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:40.327550 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c278d-952b-4def-bf83-4aa2025a15bb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:40.328923 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c278d-952b-4def-bf83-4aa2025a15bb None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:40.349673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 578/1169] 10.222.0.84 () {60 vars in 1138 bytes} [Thu May 19 19:38:40 2022] POST /identity/v3/domains => generated 267 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:38:42.566425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9da91-3407-46b7-8f20-a0d177b50a3f None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:42.567280 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9da91-3407-46b7-8f20-a0d177b50a3f None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:42.567280 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9da91-3407-46b7-8f20-a0d177b50a3f None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:42.569493 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 592/1170] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:42.575239 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017302b2-a1a7-4ec9-ab79-5750331ace7b None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:42.575667 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017302b2-a1a7-4ec9-ab79-5750331ace7b None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:42.575774 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017302b2-a1a7-4ec9-ab79-5750331ace7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:42.594463 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-017302b2-a1a7-4ec9-ab79-5750331ace7b None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:42.595631 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-017302b2-a1a7-4ec9-ab79-5750331ace7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:42.642681 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:42.671854 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 579/1171] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:42.696151 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9da91-3407-46b7-8f20-a0d177b50a3f None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:42.794831 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:42.814165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9da91-3407-46b7-8f20-a0d177b50a3f None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:42.848898 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1850947-13b1-428d-a8db-1e95680b67cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:42.849693 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1850947-13b1-428d-a8db-1e95680b67cf None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:42.850288 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1850947-13b1-428d-a8db-1e95680b67cf None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:42.850595 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1850947-13b1-428d-a8db-1e95680b67cf None admin] PATH_INFO: `/v3/domains/4b966e3d78b14eb981b52a571702d449` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:42.861698 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1850947-13b1-428d-a8db-1e95680b67cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=4b966e3d78b14eb981b52a571702d449)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:42.863195 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1850947-13b1-428d-a8db-1e95680b67cf None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:42.864847 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 593/1172] 10.222.0.84 () {56 vars in 1183 bytes} [Thu May 19 19:38:42 2022] GET /identity/v3/domains/4b966e3d78b14eb981b52a571702d449 => generated 267 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:42.871674 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017302b2-a1a7-4ec9-ab79-5750331ace7b None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:42.976719 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017302b2-a1a7-4ec9-ab79-5750331ace7b None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:43.027402 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06052810-b8fc-4f3e-9303-5e00c92b2c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:43.028056 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06052810-b8fc-4f3e-9303-5e00c92b2c07 None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:43.028333 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06052810-b8fc-4f3e-9303-5e00c92b2c07 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:43.028602 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06052810-b8fc-4f3e-9303-5e00c92b2c07 None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:43.029464 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06052810-b8fc-4f3e-9303-5e00c92b2c07 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:43.030938 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06052810-b8fc-4f3e-9303-5e00c92b2c07 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:43.058818 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 580/1173] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:38:42 2022] POST /identity/v3/projects => generated 361 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:38:45.351787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53c60a-fb26-4cf2-a855-4293747dcd54 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:45.352351 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53c60a-fb26-4cf2-a855-4293747dcd54 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:45.352351 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53c60a-fb26-4cf2-a855-4293747dcd54 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:45.353722 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 594/1174] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:45.360715 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e96665-dcdf-485a-848b-c6db0a4acbaa None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:45.361418 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e96665-dcdf-485a-848b-c6db0a4acbaa None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:45.361566 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e96665-dcdf-485a-848b-c6db0a4acbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:45.384757 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-52e96665-dcdf-485a-848b-c6db0a4acbaa None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:45.385970 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-52e96665-dcdf-485a-848b-c6db0a4acbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:45.410902 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:45.460299 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 581/1175] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:45.487075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53c60a-fb26-4cf2-a855-4293747dcd54 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:45.601357 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53c60a-fb26-4cf2-a855-4293747dcd54 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:45.637618 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16149ba1-287c-4f78-a7e6-0c09131e773f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:45.638513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16149ba1-287c-4f78-a7e6-0c09131e773f None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:45.638778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16149ba1-287c-4f78-a7e6-0c09131e773f None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:45.639044 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16149ba1-287c-4f78-a7e6-0c09131e773f None admin] PATH_INFO: `/v3/domains/4b966e3d78b14eb981b52a571702d449` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:45.640404 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16149ba1-287c-4f78-a7e6-0c09131e773f None admin] RBAC: Authorizing `identity:get_domain(domain_id=4b966e3d78b14eb981b52a571702d449)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:45.641853 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16149ba1-287c-4f78-a7e6-0c09131e773f None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:45.643522 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 595/1176] 10.222.0.84 () {56 vars in 1183 bytes} [Thu May 19 19:38:45 2022] GET /identity/v3/domains/4b966e3d78b14eb981b52a571702d449 => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:45.650184 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e96665-dcdf-485a-848b-c6db0a4acbaa None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:45.760202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e96665-dcdf-485a-848b-c6db0a4acbaa None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:45.796858 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871ef11-128e-4ab6-87e5-59f2cdaa78db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:45.797522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871ef11-128e-4ab6-87e5-59f2cdaa78db None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:45.797799 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871ef11-128e-4ab6-87e5-59f2cdaa78db None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:45.798070 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871ef11-128e-4ab6-87e5-59f2cdaa78db None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:45.798799 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871ef11-128e-4ab6-87e5-59f2cdaa78db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:45.800230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871ef11-128e-4ab6-87e5-59f2cdaa78db None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:45.827872 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 582/1177] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:38:45 2022] POST /identity/v3/users => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:38:48.072698 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43eeb9-2ffc-417b-b302-dccd36a496f5 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:48.073590 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43eeb9-2ffc-417b-b302-dccd36a496f5 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:48.073590 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43eeb9-2ffc-417b-b302-dccd36a496f5 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:48.075794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 596/1178] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:48.082223 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30ee2b-0dc5-4187-ba5c-a949546526f7 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:48.082704 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30ee2b-0dc5-4187-ba5c-a949546526f7 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:48.082808 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30ee2b-0dc5-4187-ba5c-a949546526f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:48.109726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-7d30ee2b-0dc5-4187-ba5c-a949546526f7 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:48.110935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-7d30ee2b-0dc5-4187-ba5c-a949546526f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:48.120409 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:48.181093 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 583/1179] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:48.213253 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43eeb9-2ffc-417b-b302-dccd36a496f5 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:48.237292 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:48.348731 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43eeb9-2ffc-417b-b302-dccd36a496f5 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:48.390005 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ce572-d726-47af-aca1-42ed7a1ca99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:48.391054 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ce572-d726-47af-aca1-42ed7a1ca99f None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:48.391328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ce572-d726-47af-aca1-42ed7a1ca99f None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:48.391587 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ce572-d726-47af-aca1-42ed7a1ca99f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:48.402781 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ce572-d726-47af-aca1-42ed7a1ca99f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-102ce572-d726-47af-aca1-42ed7a1ca99f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:38:48.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:48.411685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:48.412944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:48.412944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:48.412944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:48.412944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:48.412944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:38:48.412944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:38:48.412944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:38:48.412944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:38:48.412944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:38:48.412944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:38:48.412944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:48.412944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:38:48.412944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 597/1180] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:38:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:48.420811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30ee2b-0dc5-4187-ba5c-a949546526f7 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:48.532059 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30ee2b-0dc5-4187-ba5c-a949546526f7 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:48.567747 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed64d6f-8d8c-452e-80cf-c758eb0f055b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:48.568406 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed64d6f-8d8c-452e-80cf-c758eb0f055b None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:48.568681 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed64d6f-8d8c-452e-80cf-c758eb0f055b None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:48.568954 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed64d6f-8d8c-452e-80cf-c758eb0f055b None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:48.569696 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed64d6f-8d8c-452e-80cf-c758eb0f055b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:48.570005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed64d6f-8d8c-452e-80cf-c758eb0f055b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:48.571425 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed64d6f-8d8c-452e-80cf-c758eb0f055b None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:48.580060 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 584/1181] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:38:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:48.589277 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ce572-d726-47af-aca1-42ed7a1ca99f None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:48.663226 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:48.702225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ce572-d726-47af-aca1-42ed7a1ca99f None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:48.740814 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9021f25-5bee-482a-bcba-1f7bdbf4fb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:48.741685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9021f25-5bee-482a-bcba-1f7bdbf4fb2d None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:48.741964 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9021f25-5bee-482a-bcba-1f7bdbf4fb2d None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:48.742238 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9021f25-5bee-482a-bcba-1f7bdbf4fb2d None admin] PATH_INFO: `/v3/users/9c6490b3450541fca16a1a5276b6e8ea` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:48.756667 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9021f25-5bee-482a-bcba-1f7bdbf4fb2d None admin] RBAC: Authorizing `identity:get_user(user_id=9c6490b3450541fca16a1a5276b6e8ea)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:48.758132 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9021f25-5bee-482a-bcba-1f7bdbf4fb2d None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:48.759838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 598/1182] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:38:48 2022] GET /identity/v3/users/9c6490b3450541fca16a1a5276b6e8ea => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:48.767405 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed64d6f-8d8c-452e-80cf-c758eb0f055b None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:48.777215 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:48.903574 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed64d6f-8d8c-452e-80cf-c758eb0f055b None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:48.947057 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d718c7a-3373-42e4-b209-bd0811bafb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:48.947706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d718c7a-3373-42e4-b209-bd0811bafb7d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:48.947992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d718c7a-3373-42e4-b209-bd0811bafb7d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:48.948266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d718c7a-3373-42e4-b209-bd0811bafb7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:49.024149 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d718c7a-3373-42e4-b209-bd0811bafb7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:49.025604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d718c7a-3373-42e4-b209-bd0811bafb7d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:49.034322 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:49.133103 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 585/1183] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:38:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:38:49.139370 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9021f25-5bee-482a-bcba-1f7bdbf4fb2d None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:49.241411 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9021f25-5bee-482a-bcba-1f7bdbf4fb2d None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:49.269918 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:49.276801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30ec28-ff68-4c71-983a-b330f082700b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:49.277659 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30ec28-ff68-4c71-983a-b330f082700b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:49.277935 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30ec28-ff68-4c71-983a-b330f082700b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:49.278200 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30ec28-ff68-4c71-983a-b330f082700b None admin] PATH_INFO: `/v3/projects/720726022338496a88e91c775e17cce9` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:49.279597 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30ec28-ff68-4c71-983a-b330f082700b None admin] RBAC: Authorizing `identity:get_project(project_id=720726022338496a88e91c775e17cce9)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:49.281148 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30ec28-ff68-4c71-983a-b330f082700b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:49.282864 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 599/1184] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:38:49 2022] GET /identity/v3/projects/720726022338496a88e91c775e17cce9 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:49.292536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d718c7a-3373-42e4-b209-bd0811bafb7d None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:49.404835 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d718c7a-3373-42e4-b209-bd0811bafb7d None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:49.412261 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:49.440422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67fdd4-a2bc-4ffd-a4ea-d4370a20df5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:49.441063 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67fdd4-a2bc-4ffd-a4ea-d4370a20df5f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:49.441339 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67fdd4-a2bc-4ffd-a4ea-d4370a20df5f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:49.441605 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67fdd4-a2bc-4ffd-a4ea-d4370a20df5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:49.442466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67fdd4-a2bc-4ffd-a4ea-d4370a20df5f None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=720726022338496a88e91c775e17cce9, user.id=9c6490b3450541fca16a1a5276b6e8ea) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:49.442777 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67fdd4-a2bc-4ffd-a4ea-d4370a20df5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:49.444229 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67fdd4-a2bc-4ffd-a4ea-d4370a20df5f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:49.449762 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 586/1185] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:38:49 2022] GET /identity/v3/role_assignments?user.id=9c6490b3450541fca16a1a5276b6e8ea&scope.project.id=720726022338496a88e91c775e17cce9&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:51.756975 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8fe3b-7241-47f1-ae73-d0ccce72ff20 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:51.757527 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8fe3b-7241-47f1-ae73-d0ccce72ff20 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:51.757527 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8fe3b-7241-47f1-ae73-d0ccce72ff20 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:51.759633 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 600/1186] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:51.765916 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeacbb-1b8a-45e5-8a15-ce7347d0e175 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:51.766393 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeacbb-1b8a-45e5-8a15-ce7347d0e175 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:51.766539 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeacbb-1b8a-45e5-8a15-ce7347d0e175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:51.792307 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-f3eeacbb-1b8a-45e5-8a15-ce7347d0e175 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:51.793562 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-f3eeacbb-1b8a-45e5-8a15-ce7347d0e175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:51.870140 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 587/1187] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:51.899042 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8fe3b-7241-47f1-ae73-d0ccce72ff20 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:52.015327 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8fe3b-7241-47f1-ae73-d0ccce72ff20 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:52.049971 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75553b2-e6cc-4120-a854-3b5c6554aaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:52.050795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75553b2-e6cc-4120-a854-3b5c6554aaee None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:52.051070 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75553b2-e6cc-4120-a854-3b5c6554aaee None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:52.051323 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75553b2-e6cc-4120-a854-3b5c6554aaee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:52.062396 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75553b2-e6cc-4120-a854-3b5c6554aaee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-f75553b2-e6cc-4120-a854-3b5c6554aaee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:38:52.069982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:52.071033 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:52.072040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:52.072040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:52.072040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:52.072040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:52.072040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:38:52.072040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:38:52.072040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:38:52.072040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:38:52.072040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:38:52.072040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:38:52.072040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:52.072040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:38:52.072040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 601/1188] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:38:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:52.077473 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eeacbb-1b8a-45e5-8a15-ce7347d0e175 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:52.181512 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:52.184428 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eeacbb-1b8a-45e5-8a15-ce7347d0e175 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:52.220504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff4f45-5a85-42d0-b23b-38d0d9bcdca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:52.221169 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff4f45-5a85-42d0-b23b-38d0d9bcdca7 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:52.221443 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff4f45-5a85-42d0-b23b-38d0d9bcdca7 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:52.221714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff4f45-5a85-42d0-b23b-38d0d9bcdca7 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:52.222484 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff4f45-5a85-42d0-b23b-38d0d9bcdca7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:52.222792 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff4f45-5a85-42d0-b23b-38d0d9bcdca7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:52.224162 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff4f45-5a85-42d0-b23b-38d0d9bcdca7 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:52.232910 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 588/1189] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:38:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:52.239753 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75553b2-e6cc-4120-a854-3b5c6554aaee None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:52.303313 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:52.354729 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75553b2-e6cc-4120-a854-3b5c6554aaee None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:52.390711 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9413d-2fb5-4e20-bffb-dc1437d8e342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:52.391868 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9413d-2fb5-4e20-bffb-dc1437d8e342 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:52.392141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9413d-2fb5-4e20-bffb-dc1437d8e342 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:52.392409 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9413d-2fb5-4e20-bffb-dc1437d8e342 None admin] PATH_INFO: `/v3/users/9c6490b3450541fca16a1a5276b6e8ea` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:52.393604 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b9413d-2fb5-4e20-bffb-dc1437d8e342 None admin] RBAC: Authorizing `identity:get_user(user_id=9c6490b3450541fca16a1a5276b6e8ea)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:52.395570 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b9413d-2fb5-4e20-bffb-dc1437d8e342 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:52.397230 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 602/1190] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:38:52 2022] GET /identity/v3/users/9c6490b3450541fca16a1a5276b6e8ea => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:52.403992 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff4f45-5a85-42d0-b23b-38d0d9bcdca7 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:52.516050 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff4f45-5a85-42d0-b23b-38d0d9bcdca7 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:52.551193 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2085ff-5190-491c-aa61-17d12acedb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:52.552054 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2085ff-5190-491c-aa61-17d12acedb80 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:52.552330 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2085ff-5190-491c-aa61-17d12acedb80 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:52.552597 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2085ff-5190-491c-aa61-17d12acedb80 None admin] PATH_INFO: `/v3/projects/720726022338496a88e91c775e17cce9` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:52.553874 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2085ff-5190-491c-aa61-17d12acedb80 None admin] RBAC: Authorizing `identity:get_project(project_id=720726022338496a88e91c775e17cce9)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:52.555339 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2085ff-5190-491c-aa61-17d12acedb80 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:52.557024 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 589/1191] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:38:52 2022] GET /identity/v3/projects/720726022338496a88e91c775e17cce9 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:52.566541 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9413d-2fb5-4e20-bffb-dc1437d8e342 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:52.674830 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9413d-2fb5-4e20-bffb-dc1437d8e342 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:52.699996 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:52.710489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a6af6-d2d1-4eee-9664-370cd1b43eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:52.711375 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a6af6-d2d1-4eee-9664-370cd1b43eaf None admin] REQUEST_METHOD: `PUT` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:52.711648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a6af6-d2d1-4eee-9664-370cd1b43eaf None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:52.711914 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a6af6-d2d1-4eee-9664-370cd1b43eaf None admin] PATH_INFO: `/v3/projects/720726022338496a88e91c775e17cce9/users/9c6490b3450541fca16a1a5276b6e8ea/roles/a33caf2301fb44db90304eb3d9afc3a5` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:52.714078 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a6af6-d2d1-4eee-9664-370cd1b43eaf None admin] RBAC: Authorizing `identity:create_grant(project_id=720726022338496a88e91c775e17cce9, user_id=9c6490b3450541fca16a1a5276b6e8ea, role_id=a33caf2301fb44db90304eb3d9afc3a5)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:52.715553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a6af6-d2d1-4eee-9664-370cd1b43eaf None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:52.730706 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 603/1192] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:38:52 2022] PUT /identity/v3/projects/720726022338496a88e91c775e17cce9/users/9c6490b3450541fca16a1a5276b6e8ea/roles/a33caf2301fb44db90304eb3d9afc3a5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:38:54.933343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbcf90-f13f-4c4d-a781-48ba43da1856 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:54.934185 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbcf90-f13f-4c4d-a781-48ba43da1856 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:54.934185 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbcf90-f13f-4c4d-a781-48ba43da1856 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:54.936412 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 590/1193] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:38:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:38:54.942814 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f188e2-66ac-4548-b603-13666fd4cb9b None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:54.943590 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f188e2-66ac-4548-b603-13666fd4cb9b None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:54.943740 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f188e2-66ac-4548-b603-13666fd4cb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:54.964228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-d8f188e2-66ac-4548-b603-13666fd4cb9b None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:38:54.965415 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-d8f188e2-66ac-4548-b603-13666fd4cb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:38:55.037215 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 604/1194] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:38:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:38:55.063937 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbcf90-f13f-4c4d-a781-48ba43da1856 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:55.099339 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:55.176657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbcf90-f13f-4c4d-a781-48ba43da1856 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:55.225266 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fa67c-eb87-4113-8009-5c808895e8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:55.225998 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fa67c-eb87-4113-8009-5c808895e8cb None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:55.226304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fa67c-eb87-4113-8009-5c808895e8cb None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:55.226559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fa67c-eb87-4113-8009-5c808895e8cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:55.237875 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fa67c-eb87-4113-8009-5c808895e8cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-ad8fa67c-eb87-4113-8009-5c808895e8cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:38:55.245520 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:38:55.246645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:38:55.247581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:38:55.247581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:38:55.247581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:38:55.247581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:38:55.247581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:38:55.247581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:38:55.247581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:38:55.247581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:38:55.247581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:38:55.247581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:38:55.247581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:38:55.247581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:38:55.247581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 591/1195] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:38:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:38:55.255204 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f188e2-66ac-4548-b603-13666fd4cb9b None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:55.374236 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f188e2-66ac-4548-b603-13666fd4cb9b None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:55.392339 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:55.412811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32860b04-e56b-4736-bc6c-37719c1a6b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:55.413475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32860b04-e56b-4736-bc6c-37719c1a6b07 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:55.413750 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32860b04-e56b-4736-bc6c-37719c1a6b07 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:55.414018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32860b04-e56b-4736-bc6c-37719c1a6b07 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:55.414801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32860b04-e56b-4736-bc6c-37719c1a6b07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:55.415175 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32860b04-e56b-4736-bc6c-37719c1a6b07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:55.416544 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32860b04-e56b-4736-bc6c-37719c1a6b07 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:55.424763 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 605/1196] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:38:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:55.431367 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fa67c-eb87-4113-8009-5c808895e8cb None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:55.524283 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:55.534410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fa67c-eb87-4113-8009-5c808895e8cb None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:55.568633 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc9d28-4bc3-4992-8856-fbdd250eb27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:55.569497 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc9d28-4bc3-4992-8856-fbdd250eb27d None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:55.569772 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc9d28-4bc3-4992-8856-fbdd250eb27d None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:55.570043 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc9d28-4bc3-4992-8856-fbdd250eb27d None admin] PATH_INFO: `/v3/users/9c6490b3450541fca16a1a5276b6e8ea` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:55.571290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc9d28-4bc3-4992-8856-fbdd250eb27d None admin] RBAC: Authorizing `identity:get_user(user_id=9c6490b3450541fca16a1a5276b6e8ea)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:55.572749 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc9d28-4bc3-4992-8856-fbdd250eb27d None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:55.574388 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 592/1197] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:38:55 2022] GET /identity/v3/users/9c6490b3450541fca16a1a5276b6e8ea => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:38:55.584058 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32860b04-e56b-4736-bc6c-37719c1a6b07 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:55.700297 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32860b04-e56b-4736-bc6c-37719c1a6b07 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:55.736243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13219320-69c6-413e-845d-427ea6baaf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:55.736895 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13219320-69c6-413e-845d-427ea6baaf71 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:55.737173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13219320-69c6-413e-845d-427ea6baaf71 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:55.737441 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13219320-69c6-413e-845d-427ea6baaf71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:55.785159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:55.808704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13219320-69c6-413e-845d-427ea6baaf71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:55.810169 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13219320-69c6-413e-845d-427ea6baaf71 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:55.915448 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 606/1198] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:38:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:38:55.922332 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc9d28-4bc3-4992-8856-fbdd250eb27d None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:56.041171 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc9d28-4bc3-4992-8856-fbdd250eb27d None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:56.076614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07589c73-bcd2-4e37-8215-2b7b276861c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:56.077568 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07589c73-bcd2-4e37-8215-2b7b276861c7 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:56.077837 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07589c73-bcd2-4e37-8215-2b7b276861c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:56.078106 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07589c73-bcd2-4e37-8215-2b7b276861c7 None admin] PATH_INFO: `/v3/projects/720726022338496a88e91c775e17cce9` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:56.079416 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07589c73-bcd2-4e37-8215-2b7b276861c7 None admin] RBAC: Authorizing `identity:get_project(project_id=720726022338496a88e91c775e17cce9)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:56.080843 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07589c73-bcd2-4e37-8215-2b7b276861c7 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:56.082526 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 593/1199] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:38:55 2022] GET /identity/v3/projects/720726022338496a88e91c775e17cce9 => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:38:56.088837 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13219320-69c6-413e-845d-427ea6baaf71 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:38:56.174074 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:38:56.197476 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13219320-69c6-413e-845d-427ea6baaf71 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:38:56.235339 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de11de-dfd0-4d13-a6c2-3aaa225fecec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:38:56.235984 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de11de-dfd0-4d13-a6c2-3aaa225fecec None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:38:56.236262 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de11de-dfd0-4d13-a6c2-3aaa225fecec None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:38:56.236530 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de11de-dfd0-4d13-a6c2-3aaa225fecec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:38:56.237362 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de11de-dfd0-4d13-a6c2-3aaa225fecec None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=720726022338496a88e91c775e17cce9, user.id=9c6490b3450541fca16a1a5276b6e8ea) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:38:56.237666 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de11de-dfd0-4d13-a6c2-3aaa225fecec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:38:56.239160 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de11de-dfd0-4d13-a6c2-3aaa225fecec None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:38:56.245071 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 607/1200] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:38:56 2022] GET /identity/v3/role_assignments?user.id=9c6490b3450541fca16a1a5276b6e8ea&scope.project.id=720726022338496a88e91c775e17cce9&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:39:06.071777 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=87 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:39:06.242715 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=29 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:39:56.830506 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265db8c-f1e3-4d04-8245-c6fcbb577833 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:39:56.831112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265db8c-f1e3-4d04-8245-c6fcbb577833 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:39:56.831112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265db8c-f1e3-4d04-8245-c6fcbb577833 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:39:56.832614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 594/1201] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:39:56.838493 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07981e0e-3b47-4b9e-a575-0b45daef145e None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:39:56.838940 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07981e0e-3b47-4b9e-a575-0b45daef145e None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:39:56.839025 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07981e0e-3b47-4b9e-a575-0b45daef145e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:39:56.863908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-07981e0e-3b47-4b9e-a575-0b45daef145e None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:39:56.865267 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-07981e0e-3b47-4b9e-a575-0b45daef145e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:39:56.970372 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 608/1202] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:39:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:39:56.992440 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265db8c-f1e3-4d04-8245-c6fcbb577833 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:39:57.121746 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265db8c-f1e3-4d04-8245-c6fcbb577833 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:39:57.161672 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466c978-3591-4adf-bfd5-ea841f0fafa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:39:57.162569 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466c978-3591-4adf-bfd5-ea841f0fafa0 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:39:57.162929 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466c978-3591-4adf-bfd5-ea841f0fafa0 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:39:57.163148 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466c978-3591-4adf-bfd5-ea841f0fafa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:39:57.176200 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466c978-3591-4adf-bfd5-ea841f0fafa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:39:57.177827 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466c978-3591-4adf-bfd5-ea841f0fafa0 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:39:57.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-8466c978-3591-4adf-bfd5-ea841f0fafa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:39:57.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:39:57.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:39:57.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:39:57.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:39:57.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:39:57.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:39:57.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:39:57.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:39:57.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:39:57.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:39:57.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:39:57.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:39:57.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:39:57.197784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:39:57.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:39:57.200192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:39:57.200192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:39:57.200192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:39:57.200192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:39:57.200192 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 595/1203] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:39:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:39:57.204850 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07981e0e-3b47-4b9e-a575-0b45daef145e None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:39:57.325650 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07981e0e-3b47-4b9e-a575-0b45daef145e None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:39:57.365414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:39:57.377519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68eac8-a555-4bee-b28c-9496aeb3c33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:39:57.378444 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68eac8-a555-4bee-b28c-9496aeb3c33c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:39:57.379487 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68eac8-a555-4bee-b28c-9496aeb3c33c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:39:57.379736 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68eac8-a555-4bee-b28c-9496aeb3c33c None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:39:57.380633 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68eac8-a555-4bee-b28c-9496aeb3c33c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:39:57.381013 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68eac8-a555-4bee-b28c-9496aeb3c33c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:39:57.382812 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68eac8-a555-4bee-b28c-9496aeb3c33c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:39:57.395730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 609/1204] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:39:57.401650 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466c978-3591-4adf-bfd5-ea841f0fafa0 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:39:57.472304 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:39:57.529548 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466c978-3591-4adf-bfd5-ea841f0fafa0 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:39:57.576971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205cb98-350e-4f08-94a4-729da105b02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:39:57.577620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205cb98-350e-4f08-94a4-729da105b02e None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:39:57.577902 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205cb98-350e-4f08-94a4-729da105b02e None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:39:57.578176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205cb98-350e-4f08-94a4-729da105b02e None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:39:57.578971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205cb98-350e-4f08-94a4-729da105b02e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:39:57.580415 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205cb98-350e-4f08-94a4-729da105b02e None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:39:57.600912 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 596/1205] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:39:57 2022] POST /identity/v3/users => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:40:00.111320 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7cdc0-9f6f-408b-8d09-cd2faae55d9c None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:00.111900 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7cdc0-9f6f-408b-8d09-cd2faae55d9c None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:00.111900 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7cdc0-9f6f-408b-8d09-cd2faae55d9c None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:00.113263 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 610/1206] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:40:00.121423 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d0516-af19-451d-9a12-2ef84e8ad73d None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:00.121423 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d0516-af19-451d-9a12-2ef84e8ad73d None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:00.121423 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d0516-af19-451d-9a12-2ef84e8ad73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:00.138935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-7a5d0516-af19-451d-9a12-2ef84e8ad73d None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:40:00.140293 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-7a5d0516-af19-451d-9a12-2ef84e8ad73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:40:00.212883 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 597/1207] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:40:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:40:00.237052 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7cdc0-9f6f-408b-8d09-cd2faae55d9c None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:00.356816 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7cdc0-9f6f-408b-8d09-cd2faae55d9c None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:00.400524 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60ba04-91e6-4dbf-bda4-27066362cf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:00.401330 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60ba04-91e6-4dbf-bda4-27066362cf9d None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:00.401623 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60ba04-91e6-4dbf-bda4-27066362cf9d None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:00.401987 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60ba04-91e6-4dbf-bda4-27066362cf9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:00.415427 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60ba04-91e6-4dbf-bda4-27066362cf9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-3c60ba04-91e6-4dbf-bda4-27066362cf9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:40:00.424811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:00.425802 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:00.431584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:00.431584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:00.431584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:00.431584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:00.431584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:40:00.431584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:40:00.431584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:40:00.431584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:40:00.431584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:40:00.431584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:40:00.431584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:40:00.431584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:00.431584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 611/1208] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:40:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:00.432515 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d0516-af19-451d-9a12-2ef84e8ad73d None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:00.481645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:00.550749 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d0516-af19-451d-9a12-2ef84e8ad73d None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:00.590659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75948b98-f914-4436-8168-f6cfe91e25f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:00.591338 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75948b98-f914-4436-8168-f6cfe91e25f0 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:00.591669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75948b98-f914-4436-8168-f6cfe91e25f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:00.591910 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75948b98-f914-4436-8168-f6cfe91e25f0 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:00.592640 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75948b98-f914-4436-8168-f6cfe91e25f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:00.592976 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75948b98-f914-4436-8168-f6cfe91e25f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:00.594405 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75948b98-f914-4436-8168-f6cfe91e25f0 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:00.603406 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 598/1209] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:40:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:00.609439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60ba04-91e6-4dbf-bda4-27066362cf9d None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:00.663932 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:00.726754 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60ba04-91e6-4dbf-bda4-27066362cf9d None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:00.769841 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483acf98-da0e-4dd9-8eda-c315601e88fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:00.771011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483acf98-da0e-4dd9-8eda-c315601e88fc None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:00.771288 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483acf98-da0e-4dd9-8eda-c315601e88fc None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:00.771568 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483acf98-da0e-4dd9-8eda-c315601e88fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:00.783484 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483acf98-da0e-4dd9-8eda-c315601e88fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:00.784971 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483acf98-da0e-4dd9-8eda-c315601e88fc None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:00.806584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-483acf98-da0e-4dd9-8eda-c315601e88fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:00.806584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:00.806584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:00.806584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:00.806584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:00.806584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:00.806584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:00.806584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:00.806584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:00.806584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:00.806584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:00.806584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:00.806584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:40:00.806584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:40:00.806584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:00.807547 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:00.809222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:40:00.809222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:40:00.809222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:00.809222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:00.809222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 612/1210] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:00.812313 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75948b98-f914-4436-8168-f6cfe91e25f0 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:00.936002 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:00.939372 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75948b98-f914-4436-8168-f6cfe91e25f0 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:00.982388 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fee30a-a7d2-4ddc-a08d-36895b9b3c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:00.983062 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fee30a-a7d2-4ddc-a08d-36895b9b3c34 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:00.983345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fee30a-a7d2-4ddc-a08d-36895b9b3c34 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:00.983619 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fee30a-a7d2-4ddc-a08d-36895b9b3c34 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:00.984351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fee30a-a7d2-4ddc-a08d-36895b9b3c34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:00.984657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fee30a-a7d2-4ddc-a08d-36895b9b3c34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:00.986049 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fee30a-a7d2-4ddc-a08d-36895b9b3c34 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:00.999042 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 599/1211] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:40:01.004770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483acf98-da0e-4dd9-8eda-c315601e88fc None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:01.109213 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483acf98-da0e-4dd9-8eda-c315601e88fc None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:01.111228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:01.147412 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5856d4-0e1d-4736-a63b-81f9b9d2d6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:01.148225 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5856d4-0e1d-4736-a63b-81f9b9d2d6a1 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:01.148490 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5856d4-0e1d-4736-a63b-81f9b9d2d6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:01.148760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5856d4-0e1d-4736-a63b-81f9b9d2d6a1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:01.157754 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5856d4-0e1d-4736-a63b-81f9b9d2d6a1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:01.159310 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5856d4-0e1d-4736-a63b-81f9b9d2d6a1 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-7a5856d4-0e1d-4736-a63b-81f9b9d2d6a1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:40:01.176165 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:01.177455 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:01.183614 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:01.183614 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:01.183614 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:40:01.183614 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:40:01.183614 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:40:01.183614 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:40:01.183614 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:40:01.183614 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:40:01.183614 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:40:01.183614 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:01.183614 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 613/1212] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:01.184762 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fee30a-a7d2-4ddc-a08d-36895b9b3c34 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:01.293385 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fee30a-a7d2-4ddc-a08d-36895b9b3c34 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:01.330703 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee128e-c802-4140-ae47-ed649bf55e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:01.331383 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee128e-c802-4140-ae47-ed649bf55e2c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:01.331657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee128e-c802-4140-ae47-ed649bf55e2c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:01.331927 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee128e-c802-4140-ae47-ed649bf55e2c None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:01.332749 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee128e-c802-4140-ae47-ed649bf55e2c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:01.333083 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee128e-c802-4140-ae47-ed649bf55e2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:01.334510 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee128e-c802-4140-ae47-ed649bf55e2c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:01.353383 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 600/1213] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 19:40:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:40:01.359000 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5856d4-0e1d-4736-a63b-81f9b9d2d6a1 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:01.469538 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5856d4-0e1d-4736-a63b-81f9b9d2d6a1 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:01.509518 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdf1fc-3be2-43f2-bcb8-f741fc0761e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:01.510168 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdf1fc-3be2-43f2-bcb8-f741fc0761e8 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:01.510472 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdf1fc-3be2-43f2-bcb8-f741fc0761e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:01.510787 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdf1fc-3be2-43f2-bcb8-f741fc0761e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:01.543682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:01.589273 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbdf1fc-3be2-43f2-bcb8-f741fc0761e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:01.590789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbdf1fc-3be2-43f2-bcb8-f741fc0761e8 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:01.713587 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 614/1214] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:40:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:40:01.719571 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee128e-c802-4140-ae47-ed649bf55e2c None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:01.745852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:01.840931 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee128e-c802-4140-ae47-ed649bf55e2c None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:01.883776 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8725e-d048-4b37-8689-765c3a976651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:01.884972 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8725e-d048-4b37-8689-765c3a976651 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:01.885400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8725e-d048-4b37-8689-765c3a976651 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:01.885756 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8725e-d048-4b37-8689-765c3a976651 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:01.902391 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8725e-d048-4b37-8689-765c3a976651 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:01.904091 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8725e-d048-4b37-8689-765c3a976651 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:01.931295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-1fd8725e-d048-4b37-8689-765c3a976651 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:01.931295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:01.931295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:01.931295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:01.931295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:01.931295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:01.931295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:01.931295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:01.931295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:01.931295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:01.931295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:01.931295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:01.931295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:40:01.931295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:40:01.931295 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:01.932348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:01.933313 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:40:01.933313 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:40:01.933313 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:01.933313 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:40:01.938353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 601/1215] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:01.938513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdf1fc-3be2-43f2-bcb8-f741fc0761e8 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:02.003978 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:02.045576 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdf1fc-3be2-43f2-bcb8-f741fc0761e8 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:02.085715 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0735e-eee3-4279-a75c-2b4a6a2bd75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:02.086411 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0735e-eee3-4279-a75c-2b4a6a2bd75c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:02.086685 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0735e-eee3-4279-a75c-2b4a6a2bd75c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:02.086950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0735e-eee3-4279-a75c-2b4a6a2bd75c None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:02.087669 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0735e-eee3-4279-a75c-2b4a6a2bd75c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:02.087970 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0735e-eee3-4279-a75c-2b4a6a2bd75c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:02.089876 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0735e-eee3-4279-a75c-2b4a6a2bd75c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:02.104315 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 615/1216] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:02.110111 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8725e-d048-4b37-8689-765c3a976651 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:02.193718 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:02.237408 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8725e-d048-4b37-8689-765c3a976651 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:02.274809 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd9f09-1de4-45f6-8c31-45165dcfd2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:02.275614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd9f09-1de4-45f6-8c31-45165dcfd2db None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:02.275893 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd9f09-1de4-45f6-8c31-45165dcfd2db None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:02.276160 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd9f09-1de4-45f6-8c31-45165dcfd2db None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:02.283774 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd9f09-1de4-45f6-8c31-45165dcfd2db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:02.285241 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd9f09-1de4-45f6-8c31-45165dcfd2db None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:02.300894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-10bd9f09-1de4-45f6-8c31-45165dcfd2db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:40:02.300894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:02.300894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:02.300894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:02.300894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:02.300894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:02.300894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:02.300894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:02.300894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:02.300894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:02.300894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:02.300894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:02.300894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:40:02.300894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:40:02.300894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:02.301982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:02.302891 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:40:02.302891 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:40:02.302891 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:40:02.302891 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:40:02.302891 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:40:02.302891 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:40:02.302891 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:40:02.302891 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:40:02.302891 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 602/1217] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:40:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:02.308306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0735e-eee3-4279-a75c-2b4a6a2bd75c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:02.416359 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0735e-eee3-4279-a75c-2b4a6a2bd75c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:02.444772 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:02.458793 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f3949-a762-4d8c-a935-b1032164450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:02.459533 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f3949-a762-4d8c-a935-b1032164450a None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:02.459737 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f3949-a762-4d8c-a935-b1032164450a None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:02.460045 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f3949-a762-4d8c-a935-b1032164450a None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:02.460811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f3949-a762-4d8c-a935-b1032164450a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:02.461130 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f3949-a762-4d8c-a935-b1032164450a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:02.462778 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f3949-a762-4d8c-a935-b1032164450a None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:02.476748 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 616/1218] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:40:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:02.483799 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd9f09-1de4-45f6-8c31-45165dcfd2db None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:02.601397 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd9f09-1de4-45f6-8c31-45165dcfd2db None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:02.640671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:02.647765 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759921a6-5504-49de-982e-cd0c9074a9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:02.648458 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759921a6-5504-49de-982e-cd0c9074a9a0 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:02.648748 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759921a6-5504-49de-982e-cd0c9074a9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:02.649064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759921a6-5504-49de-982e-cd0c9074a9a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:02.649982 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759921a6-5504-49de-982e-cd0c9074a9a0 None admin] RBAC: Adding query filter params (role.id=cefdc5c73ec347438e5b4687f191382c, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=673a9bbc7c194dafb25fb007aa66b30a) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:02.650352 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759921a6-5504-49de-982e-cd0c9074a9a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:02.651988 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759921a6-5504-49de-982e-cd0c9074a9a0 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:02.658954 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 603/1219] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:40:02 2022] GET /identity/v3/role_assignments?user.id=673a9bbc7c194dafb25fb007aa66b30a&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=cefdc5c73ec347438e5b4687f191382c => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:05.077942 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6c013-8c98-440b-8bc6-c67cbe633fd1 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:05.078513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6c013-8c98-440b-8bc6-c67cbe633fd1 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:05.078513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6c013-8c98-440b-8bc6-c67cbe633fd1 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:05.079888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 617/1220] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:40:05.085559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20631230-10eb-4f50-8c96-0e363db44e45 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:05.085996 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20631230-10eb-4f50-8c96-0e363db44e45 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:05.086108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20631230-10eb-4f50-8c96-0e363db44e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:05.105319 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-20631230-10eb-4f50-8c96-0e363db44e45 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:40:05.106530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-20631230-10eb-4f50-8c96-0e363db44e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:40:05.182995 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 604/1221] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:40:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:40:05.209497 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6c013-8c98-440b-8bc6-c67cbe633fd1 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:05.318301 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6c013-8c98-440b-8bc6-c67cbe633fd1 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:05.360533 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08bd2d-9aca-4218-9485-5b5332337930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:05.361578 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08bd2d-9aca-4218-9485-5b5332337930 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:05.362091 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08bd2d-9aca-4218-9485-5b5332337930 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:05.362505 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08bd2d-9aca-4218-9485-5b5332337930 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:05.379032 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08bd2d-9aca-4218-9485-5b5332337930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-9e08bd2d-9aca-4218-9485-5b5332337930 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:40:05.392767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:05.394040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:05.398374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:05.398374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:05.398374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:05.398374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:05.398374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:40:05.398374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:40:05.398374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:40:05.398374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:40:05.398374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:40:05.398374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:40:05.398374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:40:05.398374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:05.398374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 618/1222] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:40:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:05.403407 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20631230-10eb-4f50-8c96-0e363db44e45 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:05.516271 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20631230-10eb-4f50-8c96-0e363db44e45 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:05.546983 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:05.558688 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aaa34-cb84-4955-b60d-45a583111309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:05.559383 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aaa34-cb84-4955-b60d-45a583111309 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:05.559657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aaa34-cb84-4955-b60d-45a583111309 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:05.559922 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aaa34-cb84-4955-b60d-45a583111309 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:05.560666 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454aaa34-cb84-4955-b60d-45a583111309 None admin] RBAC: Adding query filter params (name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:05.560965 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454aaa34-cb84-4955-b60d-45a583111309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:05.562351 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454aaa34-cb84-4955-b60d-45a583111309 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:05.571113 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 605/1223] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:40:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:05.577766 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08bd2d-9aca-4218-9485-5b5332337930 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:05.625974 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:05.689224 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08bd2d-9aca-4218-9485-5b5332337930 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:05.727571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5861d-1709-44e2-afab-901baa6b57eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:05.728432 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5861d-1709-44e2-afab-901baa6b57eb None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:05.728724 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5861d-1709-44e2-afab-901baa6b57eb None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:05.728995 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5861d-1709-44e2-afab-901baa6b57eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:05.742767 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5861d-1709-44e2-afab-901baa6b57eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:05.744317 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5861d-1709-44e2-afab-901baa6b57eb None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:05.761924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-05f5861d-1709-44e2-afab-901baa6b57eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:05.761924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:05.761924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:05.761924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:05.761924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:05.761924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:05.761924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:05.761924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:05.761924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:05.761924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:05.761924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:05.761924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:05.761924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:40:05.761924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:40:05.761924 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:05.763264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:05.764439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:40:05.764439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:40:05.764439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:05.764439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:05.764439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 619/1224] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:05.769235 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aaa34-cb84-4955-b60d-45a583111309 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:05.874590 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aaa34-cb84-4955-b60d-45a583111309 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:05.911227 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cef0f1-7ef8-4179-b9dc-08c28b8c3050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:05.912010 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cef0f1-7ef8-4179-b9dc-08c28b8c3050 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:05.912345 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cef0f1-7ef8-4179-b9dc-08c28b8c3050 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:05.912657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cef0f1-7ef8-4179-b9dc-08c28b8c3050 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:05.913619 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cef0f1-7ef8-4179-b9dc-08c28b8c3050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:05.913965 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cef0f1-7ef8-4179-b9dc-08c28b8c3050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:05.915549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cef0f1-7ef8-4179-b9dc-08c28b8c3050 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:05.929440 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 606/1225] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:05.935543 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5861d-1709-44e2-afab-901baa6b57eb None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:06.038824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:06.046630 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5861d-1709-44e2-afab-901baa6b57eb None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:06.082197 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c0b1c9-2f8f-427d-b196-41240b64f9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:06.083045 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0b1c9-2f8f-427d-b196-41240b64f9ab None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:06.083382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0b1c9-2f8f-427d-b196-41240b64f9ab None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:06.083678 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0b1c9-2f8f-427d-b196-41240b64f9ab None admin] PATH_INFO: `/v3/users/placement` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:06.091408 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0b1c9-2f8f-427d-b196-41240b64f9ab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:06.092873 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0b1c9-2f8f-427d-b196-41240b64f9ab None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-89c0b1c9-2f8f-427d-b196-41240b64f9ab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:40:06.109674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:06.110707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:06.111981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:06.111981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:06.111981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:40:06.111981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:40:06.111981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:40:06.111981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:40:06.111981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:40:06.111981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:40:06.111981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:40:06.111981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:06.111981 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 620/1226] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:06.116966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cef0f1-7ef8-4179-b9dc-08c28b8c3050 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:06.169119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:06.226111 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cef0f1-7ef8-4179-b9dc-08c28b8c3050 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:06.261886 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bef11d-1c2b-4f90-b3a8-604ccf789a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:06.262589 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bef11d-1c2b-4f90-b3a8-604ccf789a3a None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:06.262865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bef11d-1c2b-4f90-b3a8-604ccf789a3a None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:06.263211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bef11d-1c2b-4f90-b3a8-604ccf789a3a None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:06.263956 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bef11d-1c2b-4f90-b3a8-604ccf789a3a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:06.264265 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bef11d-1c2b-4f90-b3a8-604ccf789a3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:06.265722 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bef11d-1c2b-4f90-b3a8-604ccf789a3a None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:06.283373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 607/1227] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 19:40:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:06.289392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c0b1c9-2f8f-427d-b196-41240b64f9ab None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:06.402956 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c0b1c9-2f8f-427d-b196-41240b64f9ab None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:06.441331 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc472372-e714-45f5-806d-94903a1bee10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:06.442042 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc472372-e714-45f5-806d-94903a1bee10 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:06.442425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc472372-e714-45f5-806d-94903a1bee10 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:06.442707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc472372-e714-45f5-806d-94903a1bee10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:06.455947 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc472372-e714-45f5-806d-94903a1bee10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:06.457808 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc472372-e714-45f5-806d-94903a1bee10 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:06.478496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-bc472372-e714-45f5-806d-94903a1bee10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:06.478496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:06.478496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:06.478496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:06.478496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:06.478496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:06.478496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:06.478496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:06.478496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:06.478496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:06.478496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:06.478496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:06.478496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:40:06.478496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:40:06.478496 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:06.479801 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:06.481298 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:40:06.481298 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:40:06.481298 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:06.481298 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:06.481298 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 621/1228] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:06.486546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bef11d-1c2b-4f90-b3a8-604ccf789a3a None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:06.596735 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bef11d-1c2b-4f90-b3a8-604ccf789a3a None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:06.599447 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:06.640096 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe320a3-f190-4030-ba3c-188b649f2a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:06.640764 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe320a3-f190-4030-ba3c-188b649f2a20 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:06.641040 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe320a3-f190-4030-ba3c-188b649f2a20 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:06.641308 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe320a3-f190-4030-ba3c-188b649f2a20 None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:06.642052 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe320a3-f190-4030-ba3c-188b649f2a20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:06.642379 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe320a3-f190-4030-ba3c-188b649f2a20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:06.643865 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe320a3-f190-4030-ba3c-188b649f2a20 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:06.665350 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 608/1229] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:06.666651 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc472372-e714-45f5-806d-94903a1bee10 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:06.677634 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:06.779054 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc472372-e714-45f5-806d-94903a1bee10 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:06.814761 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf47bef-b57c-4bb6-8ae8-b345c5c87b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:06.815556 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf47bef-b57c-4bb6-8ae8-b345c5c87b33 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:06.815832 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf47bef-b57c-4bb6-8ae8-b345c5c87b33 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:06.816109 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf47bef-b57c-4bb6-8ae8-b345c5c87b33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:06.822933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf47bef-b57c-4bb6-8ae8-b345c5c87b33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:06.824698 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf47bef-b57c-4bb6-8ae8-b345c5c87b33 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:06.840072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-acf47bef-b57c-4bb6-8ae8-b345c5c87b33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:40:06.840072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:06.840072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:06.840072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:06.840072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:06.840072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:06.840072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:06.840072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:06.840072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:06.840072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:06.840072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:06.840072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:06.840072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:40:06.840072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:40:06.840072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:06.841066 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:06.841944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:40:06.841944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:40:06.841944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:40:06.841944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:40:06.841944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:40:06.841944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:40:06.841944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:40:06.841944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:06.841944 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 622/1230] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:40:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:40:06.848012 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe320a3-f190-4030-ba3c-188b649f2a20 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:06.963512 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe320a3-f190-4030-ba3c-188b649f2a20 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:06.998607 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99b369-cb48-4cf3-98e0-cf11a2dac890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:06.999269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99b369-cb48-4cf3-98e0-cf11a2dac890 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:06.999546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99b369-cb48-4cf3-98e0-cf11a2dac890 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:06.999815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99b369-cb48-4cf3-98e0-cf11a2dac890 None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:07.000591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99b369-cb48-4cf3-98e0-cf11a2dac890 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:07.000893 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99b369-cb48-4cf3-98e0-cf11a2dac890 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:07.002354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99b369-cb48-4cf3-98e0-cf11a2dac890 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:07.014946 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 609/1231] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:40:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:40:07.021264 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf47bef-b57c-4bb6-8ae8-b345c5c87b33 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:07.091736 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:07.126797 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf47bef-b57c-4bb6-8ae8-b345c5c87b33 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:07.161618 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35be4cb-68dc-4e10-b497-1ec8b3e56ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:07.162564 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35be4cb-68dc-4e10-b497-1ec8b3e56ae6 None admin] REQUEST_METHOD: `PUT` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:07.162814 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35be4cb-68dc-4e10-b497-1ec8b3e56ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:07.163078 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35be4cb-68dc-4e10-b497-1ec8b3e56ae6 None admin] PATH_INFO: `/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/673a9bbc7c194dafb25fb007aa66b30a/roles/cefdc5c73ec347438e5b4687f191382c` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:07.177297 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35be4cb-68dc-4e10-b497-1ec8b3e56ae6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecf87429758641eeb462b3e409cadc9e, user_id=673a9bbc7c194dafb25fb007aa66b30a, role_id=cefdc5c73ec347438e5b4687f191382c)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:07.178791 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35be4cb-68dc-4e10-b497-1ec8b3e56ae6 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:07.193009 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 623/1232] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:40:07 2022] PUT /identity/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/673a9bbc7c194dafb25fb007aa66b30a/roles/cefdc5c73ec347438e5b4687f191382c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:40:09.402186 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83980c3-1979-405a-a38b-c61fbd51f0ee None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:09.403587 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83980c3-1979-405a-a38b-c61fbd51f0ee None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:09.403587 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83980c3-1979-405a-a38b-c61fbd51f0ee None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:09.404498 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 610/1233] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:40:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:40:09.410664 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb6b01-590d-433c-b157-feebbab350d7 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:09.411398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb6b01-590d-433c-b157-feebbab350d7 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:09.411398 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb6b01-590d-433c-b157-feebbab350d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:09.429281 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-2bbb6b01-590d-433c-b157-feebbab350d7 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:40:09.430522 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-2bbb6b01-590d-433c-b157-feebbab350d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:40:09.500789 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 624/1234] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:40:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:40:09.525352 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83980c3-1979-405a-a38b-c61fbd51f0ee None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:09.551635 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:09.639465 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83980c3-1979-405a-a38b-c61fbd51f0ee None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:09.674934 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec74d5-b7d2-464f-90f0-8b4010101c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:09.675658 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec74d5-b7d2-464f-90f0-8b4010101c0c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:09.675939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec74d5-b7d2-464f-90f0-8b4010101c0c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:09.676219 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec74d5-b7d2-464f-90f0-8b4010101c0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:09.686627 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec74d5-b7d2-464f-90f0-8b4010101c0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-1bec74d5-b7d2-464f-90f0-8b4010101c0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:40:09.694225 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:09.695353 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:09.696373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:09.696373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:09.696373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:09.696373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:09.696373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:40:09.696373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:40:09.696373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:40:09.696373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:40:09.696373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:40:09.696373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:40:09.696373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:40:09.696373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:40:09.696373 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 611/1235] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:40:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:09.701677 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb6b01-590d-433c-b157-feebbab350d7 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:09.759117 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:09.816214 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb6b01-590d-433c-b157-feebbab350d7 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:09.853013 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68fe569-fbee-4a93-ac07-f773ce3866a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:09.853933 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68fe569-fbee-4a93-ac07-f773ce3866a6 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:09.854439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68fe569-fbee-4a93-ac07-f773ce3866a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:09.854783 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68fe569-fbee-4a93-ac07-f773ce3866a6 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:09.855726 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68fe569-fbee-4a93-ac07-f773ce3866a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:09.856130 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68fe569-fbee-4a93-ac07-f773ce3866a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:09.857638 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68fe569-fbee-4a93-ac07-f773ce3866a6 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:09.867142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 625/1236] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:40:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:40:09.873828 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec74d5-b7d2-464f-90f0-8b4010101c0c None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:09.963359 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:09.981560 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec74d5-b7d2-464f-90f0-8b4010101c0c None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:10.019319 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c855fb-f86e-4104-8293-3c61c2c1061c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:10.020150 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c855fb-f86e-4104-8293-3c61c2c1061c None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:10.020434 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c855fb-f86e-4104-8293-3c61c2c1061c None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:10.020725 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c855fb-f86e-4104-8293-3c61c2c1061c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:10.033783 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c855fb-f86e-4104-8293-3c61c2c1061c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:10.035309 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c855fb-f86e-4104-8293-3c61c2c1061c None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:10.053559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-85c855fb-f86e-4104-8293-3c61c2c1061c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:10.053559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:10.053559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:10.053559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:10.053559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:10.053559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:10.053559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:10.053559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:10.053559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:10.053559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:10.053559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:10.053559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:10.053559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:40:10.053559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:40:10.053559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:10.054760 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:10.055879 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:40:10.055879 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:40:10.055879 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:10.055879 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:40:10.060230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 612/1237] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:40:10.061464 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68fe569-fbee-4a93-ac07-f773ce3866a6 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:10.178929 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68fe569-fbee-4a93-ac07-f773ce3866a6 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:10.189947 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:10.218967 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87d81a-ef9f-4b9c-b2f5-aa674e0a61fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:10.219560 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87d81a-ef9f-4b9c-b2f5-aa674e0a61fa None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:10.219840 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87d81a-ef9f-4b9c-b2f5-aa674e0a61fa None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:10.220111 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87d81a-ef9f-4b9c-b2f5-aa674e0a61fa None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:10.220834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87d81a-ef9f-4b9c-b2f5-aa674e0a61fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:10.221142 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87d81a-ef9f-4b9c-b2f5-aa674e0a61fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:10.222793 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87d81a-ef9f-4b9c-b2f5-aa674e0a61fa None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:10.237187 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 626/1238] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:10.244168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c855fb-f86e-4104-8293-3c61c2c1061c None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:10.357633 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c855fb-f86e-4104-8293-3c61c2c1061c None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:10.399177 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01a76d-ccf4-4e18-8868-3eebc4fb41ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:10.400068 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01a76d-ccf4-4e18-8868-3eebc4fb41ed None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:10.400348 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01a76d-ccf4-4e18-8868-3eebc4fb41ed None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:10.400616 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01a76d-ccf4-4e18-8868-3eebc4fb41ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:10.404090 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:10.410220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01a76d-ccf4-4e18-8868-3eebc4fb41ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:10.412338 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01a76d-ccf4-4e18-8868-3eebc4fb41ed None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-1a01a76d-ccf4-4e18-8868-3eebc4fb41ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:40:10.427707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:10.428784 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:10.429945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:10.429945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:10.429945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:40:10.429945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:40:10.429945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:40:10.429945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:40:10.429945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:40:10.429945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:40:10.429945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:40:10.429945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:40:10.429945 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 613/1239] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:10.435517 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87d81a-ef9f-4b9c-b2f5-aa674e0a61fa None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:10.550730 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87d81a-ef9f-4b9c-b2f5-aa674e0a61fa None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:10.586909 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0990a-0063-4df8-af56-a0a9692d93a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:10.587612 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0990a-0063-4df8-af56-a0a9692d93a5 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:10.587881 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0990a-0063-4df8-af56-a0a9692d93a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:10.588186 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0990a-0063-4df8-af56-a0a9692d93a5 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:10.589144 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0990a-0063-4df8-af56-a0a9692d93a5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:10.589515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0990a-0063-4df8-af56-a0a9692d93a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:10.591052 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0990a-0063-4df8-af56-a0a9692d93a5 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:10.608972 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 627/1240] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 19:40:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:10.615036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01a76d-ccf4-4e18-8868-3eebc4fb41ed None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:10.727935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01a76d-ccf4-4e18-8868-3eebc4fb41ed None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:10.766432 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f171216-3d13-4d68-8147-a3be7b26b06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:10.767054 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f171216-3d13-4d68-8147-a3be7b26b06f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:10.767331 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f171216-3d13-4d68-8147-a3be7b26b06f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:10.767604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f171216-3d13-4d68-8147-a3be7b26b06f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:10.831310 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:10.840794 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f171216-3d13-4d68-8147-a3be7b26b06f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:10.842792 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f171216-3d13-4d68-8147-a3be7b26b06f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:10.956794 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 614/1241] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:40:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:40:10.962934 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0990a-0063-4df8-af56-a0a9692d93a5 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:10.992072 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:11.070917 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0990a-0063-4df8-af56-a0a9692d93a5 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:11.108652 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fddf5-459f-42a2-a1a0-308785bd9e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:11.109676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fddf5-459f-42a2-a1a0-308785bd9e8c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:11.110028 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fddf5-459f-42a2-a1a0-308785bd9e8c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:11.110375 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fddf5-459f-42a2-a1a0-308785bd9e8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:11.122754 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fddf5-459f-42a2-a1a0-308785bd9e8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:11.124233 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fddf5-459f-42a2-a1a0-308785bd9e8c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:11.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-2e8fddf5-459f-42a2-a1a0-308785bd9e8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:11.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:11.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:11.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:11.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:11.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:11.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:11.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:11.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:11.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:11.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:11.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:11.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:40:11.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:40:11.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:11.143714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:11.144807 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:40:11.144807 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:40:11.144807 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:11.144807 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:11.144807 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 628/1242] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:11.149890 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f171216-3d13-4d68-8147-a3be7b26b06f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:11.255790 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:11.264746 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f171216-3d13-4d68-8147-a3be7b26b06f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:11.304984 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09222612-fcb5-48bd-a51e-2168fae64eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:11.305688 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09222612-fcb5-48bd-a51e-2168fae64eea None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:11.305941 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09222612-fcb5-48bd-a51e-2168fae64eea None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:11.306212 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09222612-fcb5-48bd-a51e-2168fae64eea None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:11.307017 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09222612-fcb5-48bd-a51e-2168fae64eea None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:11.307272 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09222612-fcb5-48bd-a51e-2168fae64eea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:11.308829 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09222612-fcb5-48bd-a51e-2168fae64eea None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:11.321168 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 615/1243] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:40:11.327241 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fddf5-459f-42a2-a1a0-308785bd9e8c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:11.419468 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:11.444565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fddf5-459f-42a2-a1a0-308785bd9e8c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:11.483534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ab3e1-22f0-41c8-995f-00c7be8eda05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:11.484340 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ab3e1-22f0-41c8-995f-00c7be8eda05 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:11.484625 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ab3e1-22f0-41c8-995f-00c7be8eda05 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:11.484885 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ab3e1-22f0-41c8-995f-00c7be8eda05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:11.491751 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ab3e1-22f0-41c8-995f-00c7be8eda05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:11.493199 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ab3e1-22f0-41c8-995f-00c7be8eda05 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:11.508382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-316ab3e1-22f0-41c8-995f-00c7be8eda05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:40:11.508382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:11.508382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:11.508382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:11.508382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:11.508382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:11.508382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:11.508382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:11.508382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:11.508382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:11.508382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:11.508382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:11.508382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:40:11.508382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:40:11.508382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:11.509519 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:11.510454 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:40:11.510454 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:40:11.510454 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:40:11.510454 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:40:11.510454 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:40:11.510454 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:40:11.510454 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:40:11.510454 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:11.510454 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 629/1244] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:40:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:11.516191 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09222612-fcb5-48bd-a51e-2168fae64eea None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:11.626120 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09222612-fcb5-48bd-a51e-2168fae64eea None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:11.664672 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66cb70-97dd-43c2-9e88-7cd14ee29dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:11.665366 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66cb70-97dd-43c2-9e88-7cd14ee29dd7 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:11.665648 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66cb70-97dd-43c2-9e88-7cd14ee29dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:11.665912 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66cb70-97dd-43c2-9e88-7cd14ee29dd7 None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:11.666726 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66cb70-97dd-43c2-9e88-7cd14ee29dd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:11.667099 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66cb70-97dd-43c2-9e88-7cd14ee29dd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:11.668631 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66cb70-97dd-43c2-9e88-7cd14ee29dd7 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:11.681798 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 616/1245] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:40:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:11.688243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ab3e1-22f0-41c8-995f-00c7be8eda05 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:11.794656 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ab3e1-22f0-41c8-995f-00c7be8eda05 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:11.836061 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bed55-7a8a-468e-b478-e3f5f1123e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:11.836737 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bed55-7a8a-468e-b478-e3f5f1123e3e None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:11.837038 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bed55-7a8a-468e-b478-e3f5f1123e3e None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:11.837299 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bed55-7a8a-468e-b478-e3f5f1123e3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:11.838214 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bed55-7a8a-468e-b478-e3f5f1123e3e None admin] RBAC: Adding query filter params (role.id=cefdc5c73ec347438e5b4687f191382c, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=673a9bbc7c194dafb25fb007aa66b30a) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:11.838604 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bed55-7a8a-468e-b478-e3f5f1123e3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:11.840474 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bed55-7a8a-468e-b478-e3f5f1123e3e None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:11.848644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 630/1246] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:40:11 2022] GET /identity/v3/role_assignments?user.id=673a9bbc7c194dafb25fb007aa66b30a&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=cefdc5c73ec347438e5b4687f191382c => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:14.180421 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc9012-3c5e-4f04-a460-c080cb55ff32 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:14.180971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc9012-3c5e-4f04-a460-c080cb55ff32 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:14.180971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc9012-3c5e-4f04-a460-c080cb55ff32 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:14.182423 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 617/1247] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:40:14.188828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41987a-9736-43d7-9bf8-a67f5eaa151b None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:14.189267 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41987a-9736-43d7-9bf8-a67f5eaa151b None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:14.189470 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41987a-9736-43d7-9bf8-a67f5eaa151b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:14.195452 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:14.210468 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-bb41987a-9736-43d7-9bf8-a67f5eaa151b None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:40:14.211970 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-bb41987a-9736-43d7-9bf8-a67f5eaa151b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:40:14.289520 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 631/1248] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:40:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:40:14.316346 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc9012-3c5e-4f04-a460-c080cb55ff32 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:14.336249 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:14.440286 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc9012-3c5e-4f04-a460-c080cb55ff32 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:14.478462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967cdba-238c-4c46-bf26-622f8f3a50a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:14.479451 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967cdba-238c-4c46-bf26-622f8f3a50a1 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:14.480428 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967cdba-238c-4c46-bf26-622f8f3a50a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:14.480708 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967cdba-238c-4c46-bf26-622f8f3a50a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:14.493506 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967cdba-238c-4c46-bf26-622f8f3a50a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-4967cdba-238c-4c46-bf26-622f8f3a50a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:40:14.502517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:14.503634 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:14.504651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:14.504651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:14.504651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:14.504651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:14.504651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:40:14.504651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:40:14.504651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:40:14.504651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:40:14.504651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:40:14.504651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:40:14.504651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:40:14.504651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:40:14.504651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 618/1249] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:40:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:14.509868 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41987a-9736-43d7-9bf8-a67f5eaa151b None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:14.630148 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41987a-9736-43d7-9bf8-a67f5eaa151b None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:14.663513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:14.670902 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcab9a9-fe33-4e3f-8e34-0e74950c8a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:14.671590 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcab9a9-fe33-4e3f-8e34-0e74950c8a04 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:14.671897 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcab9a9-fe33-4e3f-8e34-0e74950c8a04 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:14.672303 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcab9a9-fe33-4e3f-8e34-0e74950c8a04 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:14.673325 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcab9a9-fe33-4e3f-8e34-0e74950c8a04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:14.673668 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcab9a9-fe33-4e3f-8e34-0e74950c8a04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:14.675110 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcab9a9-fe33-4e3f-8e34-0e74950c8a04 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:14.684498 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 632/1250] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:40:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:14.691339 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967cdba-238c-4c46-bf26-622f8f3a50a1 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:14.786752 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:14.815310 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967cdba-238c-4c46-bf26-622f8f3a50a1 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:14.853454 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd314486-ea55-470c-a821-0edd15084be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:14.854275 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd314486-ea55-470c-a821-0edd15084be0 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:14.855100 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd314486-ea55-470c-a821-0edd15084be0 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:14.855451 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd314486-ea55-470c-a821-0edd15084be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:14.866768 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd314486-ea55-470c-a821-0edd15084be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:14.868218 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd314486-ea55-470c-a821-0edd15084be0 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:14.886534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-fd314486-ea55-470c-a821-0edd15084be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:14.886534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:14.886534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:14.886534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:14.886534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:14.886534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:14.886534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:14.886534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:14.886534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:14.886534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:14.886534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:14.886534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:14.886534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:40:14.886534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:40:14.886534 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:14.887614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:14.888671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:40:14.888671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:40:14.888671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:14.888671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:40:14.888671 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 619/1251] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:14.893882 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcab9a9-fe33-4e3f-8e34-0e74950c8a04 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:15.000120 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcab9a9-fe33-4e3f-8e34-0e74950c8a04 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:15.035867 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d09cd-1060-4097-856f-8b9898205077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:15.036630 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d09cd-1060-4097-856f-8b9898205077 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:15.037004 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d09cd-1060-4097-856f-8b9898205077 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:15.037237 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d09cd-1060-4097-856f-8b9898205077 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:15.038140 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d09cd-1060-4097-856f-8b9898205077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:15.038506 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d09cd-1060-4097-856f-8b9898205077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:15.040163 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d09cd-1060-4097-856f-8b9898205077 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:15.053907 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 633/1252] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:15.059847 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd314486-ea55-470c-a821-0edd15084be0 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:15.166046 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd314486-ea55-470c-a821-0edd15084be0 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:15.192555 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:15.201154 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554e1f9-5b73-42c6-9e66-9050bb424e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:15.201959 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554e1f9-5b73-42c6-9e66-9050bb424e98 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:15.202235 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554e1f9-5b73-42c6-9e66-9050bb424e98 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:15.202532 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554e1f9-5b73-42c6-9e66-9050bb424e98 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:15.210687 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554e1f9-5b73-42c6-9e66-9050bb424e98 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:15.212190 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554e1f9-5b73-42c6-9e66-9050bb424e98 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-d554e1f9-5b73-42c6-9e66-9050bb424e98 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:40:15.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:15.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:15.230653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:15.230653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:15.230653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:40:15.230653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:40:15.230653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:40:15.230653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:40:15.230653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:40:15.230653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:40:15.230653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:40:15.230653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:40:15.230653 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 620/1253] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:15.235213 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d09cd-1060-4097-856f-8b9898205077 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:15.298564 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:15.346818 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d09cd-1060-4097-856f-8b9898205077 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:15.384844 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbe865-8681-4b61-af6a-64ceecdf8b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:15.385557 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbe865-8681-4b61-af6a-64ceecdf8b09 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:15.385884 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbe865-8681-4b61-af6a-64ceecdf8b09 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:15.386174 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbe865-8681-4b61-af6a-64ceecdf8b09 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:15.387049 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbe865-8681-4b61-af6a-64ceecdf8b09 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:15.387371 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbe865-8681-4b61-af6a-64ceecdf8b09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:15.389039 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbe865-8681-4b61-af6a-64ceecdf8b09 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:15.414059 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 634/1254] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 19:40:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:15.415395 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554e1f9-5b73-42c6-9e66-9050bb424e98 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:15.533971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554e1f9-5b73-42c6-9e66-9050bb424e98 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:15.573206 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cfdde-d451-4e2b-9f5c-15fd15448872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:15.573905 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cfdde-d451-4e2b-9f5c-15fd15448872 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:15.574184 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cfdde-d451-4e2b-9f5c-15fd15448872 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:15.574503 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cfdde-d451-4e2b-9f5c-15fd15448872 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:15.637260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:15.652235 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cfdde-d451-4e2b-9f5c-15fd15448872 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:15.653696 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cfdde-d451-4e2b-9f5c-15fd15448872 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:15.764887 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 621/1255] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:40:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:40:15.771088 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbe865-8681-4b61-af6a-64ceecdf8b09 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:15.885733 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbe865-8681-4b61-af6a-64ceecdf8b09 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:15.900298 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:15.925467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d31d5-b9f1-459c-8d90-898697d844b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:15.926302 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d31d5-b9f1-459c-8d90-898697d844b5 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:15.926580 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d31d5-b9f1-459c-8d90-898697d844b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:15.926850 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d31d5-b9f1-459c-8d90-898697d844b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:15.938700 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d31d5-b9f1-459c-8d90-898697d844b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:15.940197 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d31d5-b9f1-459c-8d90-898697d844b5 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:15.959616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-005d31d5-b9f1-459c-8d90-898697d844b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:15.959616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:15.959616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:15.959616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:15.959616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:15.959616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:15.959616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:15.959616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:15.959616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:15.959616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:15.959616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:15.959616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:15.959616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:40:15.959616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:40:15.959616 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:15.960645 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:15.961533 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:40:15.961533 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:40:15.961533 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:15.961533 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:15.961533 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 635/1256] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:15.967389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cfdde-d451-4e2b-9f5c-15fd15448872 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:16.067832 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:16.082186 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cfdde-d451-4e2b-9f5c-15fd15448872 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:16.127696 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6ca3c-1659-4df9-84c3-dcb14f1e544b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:16.128355 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6ca3c-1659-4df9-84c3-dcb14f1e544b None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:16.128640 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6ca3c-1659-4df9-84c3-dcb14f1e544b None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:16.128917 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6ca3c-1659-4df9-84c3-dcb14f1e544b None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:16.129666 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6ca3c-1659-4df9-84c3-dcb14f1e544b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:16.129960 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6ca3c-1659-4df9-84c3-dcb14f1e544b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:16.131400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6ca3c-1659-4df9-84c3-dcb14f1e544b None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:16.145673 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 622/1257] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:16.152173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d31d5-b9f1-459c-8d90-898697d844b5 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:16.259655 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d31d5-b9f1-459c-8d90-898697d844b5 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:16.296486 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde86b0-6278-4608-867d-35b62f9c745b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:16.297287 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde86b0-6278-4608-867d-35b62f9c745b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:16.297565 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde86b0-6278-4608-867d-35b62f9c745b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:16.297831 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde86b0-6278-4608-867d-35b62f9c745b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:16.304863 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde86b0-6278-4608-867d-35b62f9c745b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:16.306397 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde86b0-6278-4608-867d-35b62f9c745b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:16.320640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-6bde86b0-6278-4608-867d-35b62f9c745b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:40:16.320640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:16.320640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:16.320640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:16.320640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:16.320640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:16.320640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:16.320640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:16.320640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:16.320640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:16.320640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:16.320640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:16.320640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:40:16.320640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:40:16.320640 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:16.321553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:16.322689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:40:16.322689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:40:16.322689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:40:16.322689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:40:16.322689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:40:16.322689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:40:16.322689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:40:16.322689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:16.322689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 636/1258] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:40:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:16.327817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6ca3c-1659-4df9-84c3-dcb14f1e544b None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:16.444428 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6ca3c-1659-4df9-84c3-dcb14f1e544b None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:16.477530 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e23410-7199-4047-89a7-821f2dab4c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:16.478185 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e23410-7199-4047-89a7-821f2dab4c67 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:16.478495 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e23410-7199-4047-89a7-821f2dab4c67 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:16.478771 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e23410-7199-4047-89a7-821f2dab4c67 None admin] PATH_INFO: `/v3/projects` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:16.479711 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e23410-7199-4047-89a7-821f2dab4c67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:16.479948 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e23410-7199-4047-89a7-821f2dab4c67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:16.481394 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e23410-7199-4047-89a7-821f2dab4c67 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:16.494176 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 623/1259] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:40:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:16.503357 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde86b0-6278-4608-867d-35b62f9c745b None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:16.532343 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:16.611921 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde86b0-6278-4608-867d-35b62f9c745b None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:16.649171 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e35fe7-8342-4a6e-a0ae-ce27a895e222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:16.649812 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e35fe7-8342-4a6e-a0ae-ce27a895e222 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:16.650083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e35fe7-8342-4a6e-a0ae-ce27a895e222 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:16.650380 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e35fe7-8342-4a6e-a0ae-ce27a895e222 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:16.651226 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e35fe7-8342-4a6e-a0ae-ce27a895e222 None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=673a9bbc7c194dafb25fb007aa66b30a) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:16.651530 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e35fe7-8342-4a6e-a0ae-ce27a895e222 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:16.652958 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e35fe7-8342-4a6e-a0ae-ce27a895e222 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:16.658521 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 637/1260] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:40:16 2022] GET /identity/v3/role_assignments?user.id=673a9bbc7c194dafb25fb007aa66b30a&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:18.810861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25230a12-34ef-465e-93da-b9ff7351992f None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:18.811740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25230a12-34ef-465e-93da-b9ff7351992f None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:18.811740 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25230a12-34ef-465e-93da-b9ff7351992f None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:18.813906 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 624/1261] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:40:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:40:18.821824 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf416a6a-6df0-4056-9f6a-20fb55af6b51 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:18.822255 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf416a6a-6df0-4056-9f6a-20fb55af6b51 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:18.822392 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf416a6a-6df0-4056-9f6a-20fb55af6b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:18.839821 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-cf416a6a-6df0-4056-9f6a-20fb55af6b51 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:40:18.840985 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-cf416a6a-6df0-4056-9f6a-20fb55af6b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:40:18.913595 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 638/1262] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:40:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:40:18.936156 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25230a12-34ef-465e-93da-b9ff7351992f None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:18.949063 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:19.058153 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25230a12-34ef-465e-93da-b9ff7351992f None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:19.092815 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957a1d7-eb16-4e33-9a5e-6189af0c5238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:19.093591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957a1d7-eb16-4e33-9a5e-6189af0c5238 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:19.093861 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957a1d7-eb16-4e33-9a5e-6189af0c5238 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:19.094098 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957a1d7-eb16-4e33-9a5e-6189af0c5238 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:19.105007 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957a1d7-eb16-4e33-9a5e-6189af0c5238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-a957a1d7-eb16-4e33-9a5e-6189af0c5238 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:40:19.112570 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:19.113603 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:19.115165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:19.115165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:19.115165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:19.115165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:19.115165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:40:19.115165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:40:19.115165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:40:19.115165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:40:19.115165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:40:19.115165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:40:19.115165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:40:19.115165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:40:19.115165 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 625/1263] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:40:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:19.124076 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf416a6a-6df0-4056-9f6a-20fb55af6b51 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:19.142431 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:19.231425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf416a6a-6df0-4056-9f6a-20fb55af6b51 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:19.268581 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd8eca-6a6e-457f-b57e-9cd9288a33f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:19.269242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd8eca-6a6e-457f-b57e-9cd9288a33f5 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:19.269521 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd8eca-6a6e-457f-b57e-9cd9288a33f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:19.269794 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd8eca-6a6e-457f-b57e-9cd9288a33f5 None admin] PATH_INFO: `/v3/roles` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:19.270573 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd8eca-6a6e-457f-b57e-9cd9288a33f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:19.270869 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd8eca-6a6e-457f-b57e-9cd9288a33f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:19.272247 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd8eca-6a6e-457f-b57e-9cd9288a33f5 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:19.280651 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 639/1264] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:40:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:19.289233 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957a1d7-eb16-4e33-9a5e-6189af0c5238 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:19.359231 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:19.396541 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957a1d7-eb16-4e33-9a5e-6189af0c5238 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:19.431591 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e14034-f3cd-4d31-9c54-0a2c906b3eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:19.432450 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e14034-f3cd-4d31-9c54-0a2c906b3eaf None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:19.432848 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e14034-f3cd-4d31-9c54-0a2c906b3eaf None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:19.433160 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e14034-f3cd-4d31-9c54-0a2c906b3eaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:19.445707 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e14034-f3cd-4d31-9c54-0a2c906b3eaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:19.447230 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e14034-f3cd-4d31-9c54-0a2c906b3eaf None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:19.465118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-f6e14034-f3cd-4d31-9c54-0a2c906b3eaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:19.465118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:19.465118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:19.465118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:19.465118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:19.465118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:19.465118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:19.465118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:19.465118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:19.465118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:19.465118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:19.465118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:19.465118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:40:19.465118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:40:19.465118 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:19.466112 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:19.470645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:40:19.470645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:40:19.470645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:19.470645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:40:19.470645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 626/1265] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:40:19.474898 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd8eca-6a6e-457f-b57e-9cd9288a33f5 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:19.573374 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:19.605549 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd8eca-6a6e-457f-b57e-9cd9288a33f5 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:19.646545 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fe70e-5a9b-4244-b3bd-d656378662e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:19.647207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fe70e-5a9b-4244-b3bd-d656378662e9 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:19.647485 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fe70e-5a9b-4244-b3bd-d656378662e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:19.647749 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fe70e-5a9b-4244-b3bd-d656378662e9 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:19.648467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fe70e-5a9b-4244-b3bd-d656378662e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:19.648768 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fe70e-5a9b-4244-b3bd-d656378662e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:19.650137 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fe70e-5a9b-4244-b3bd-d656378662e9 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:19.663023 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 640/1266] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:19.670251 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e14034-f3cd-4d31-9c54-0a2c906b3eaf None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:19.784729 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e14034-f3cd-4d31-9c54-0a2c906b3eaf None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:19.806628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:19.828912 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275626f-2050-42e6-b6ae-e9cce53936de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:19.829649 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275626f-2050-42e6-b6ae-e9cce53936de None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:19.829924 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275626f-2050-42e6-b6ae-e9cce53936de None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:19.830196 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275626f-2050-42e6-b6ae-e9cce53936de None admin] PATH_INFO: `/v3/users/placement` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:19.838143 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275626f-2050-42e6-b6ae-e9cce53936de None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:19.840231 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275626f-2050-42e6-b6ae-e9cce53936de None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-b275626f-2050-42e6-b6ae-e9cce53936de None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:40:19.863105 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:19.864269 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:19.865386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:19.865386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:19.865386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:40:19.865386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:40:19.865386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:40:19.865386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:40:19.865386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:40:19.865386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:40:19.865386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:40:19.865386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:40:19.865386 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 627/1267] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:19.871018 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fe70e-5a9b-4244-b3bd-d656378662e9 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:19.986785 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fe70e-5a9b-4244-b3bd-d656378662e9 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:20.022511 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34cea6-a252-4bb2-b193-eb7355ce9d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:20.023522 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34cea6-a252-4bb2-b193-eb7355ce9d05 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:20.023943 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34cea6-a252-4bb2-b193-eb7355ce9d05 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:20.024356 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34cea6-a252-4bb2-b193-eb7355ce9d05 None admin] PATH_INFO: `/v3/users` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:20.025701 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34cea6-a252-4bb2-b193-eb7355ce9d05 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:20.026246 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34cea6-a252-4bb2-b193-eb7355ce9d05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:20.027931 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34cea6-a252-4bb2-b193-eb7355ce9d05 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:20.037511 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:20.047131 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 641/1268] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 19:40:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:20.052829 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275626f-2050-42e6-b6ae-e9cce53936de None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:20.158218 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275626f-2050-42e6-b6ae-e9cce53936de None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:20.198689 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19e7d5-fd4c-41a0-b6f1-ad12b2a5fd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:20.199565 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19e7d5-fd4c-41a0-b6f1-ad12b2a5fd93 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:20.199565 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19e7d5-fd4c-41a0-b6f1-ad12b2a5fd93 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:20.199853 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19e7d5-fd4c-41a0-b6f1-ad12b2a5fd93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:20.212734 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19e7d5-fd4c-41a0-b6f1-ad12b2a5fd93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:20.214296 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19e7d5-fd4c-41a0-b6f1-ad12b2a5fd93 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:20.232677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-ab19e7d5-fd4c-41a0-b6f1-ad12b2a5fd93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:20.232677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:20.232677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:20.232677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:20.232677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:20.232677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:20.232677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:20.232677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:20.232677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:20.232677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:20.232677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:20.232677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:20.232677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:40:20.232677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:40:20.232677 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:20.233669 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:20.234644 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:40:20.234644 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:40:20.234644 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:20.234644 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:40:20.234644 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 628/1269] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:20.239687 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34cea6-a252-4bb2-b193-eb7355ce9d05 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:20.343579 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34cea6-a252-4bb2-b193-eb7355ce9d05 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:20.381297 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a058c-fe62-4a8f-85b2-b28d002ceaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:20.381997 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a058c-fe62-4a8f-85b2-b28d002ceaff None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:20.382306 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a058c-fe62-4a8f-85b2-b28d002ceaff None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:20.382597 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a058c-fe62-4a8f-85b2-b28d002ceaff None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:20.383333 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a058c-fe62-4a8f-85b2-b28d002ceaff None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:20.383617 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a058c-fe62-4a8f-85b2-b28d002ceaff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:20.385037 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a058c-fe62-4a8f-85b2-b28d002ceaff None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:20.397159 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 642/1270] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:20.403060 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19e7d5-fd4c-41a0-b6f1-ad12b2a5fd93 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:20.439232 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:20.513970 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19e7d5-fd4c-41a0-b6f1-ad12b2a5fd93 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:20.548075 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357c3f8-bc43-4a01-8e28-2ced712ccb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:20.548852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357c3f8-bc43-4a01-8e28-2ced712ccb91 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:20.549127 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357c3f8-bc43-4a01-8e28-2ced712ccb91 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:20.549399 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357c3f8-bc43-4a01-8e28-2ced712ccb91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:20.556198 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a357c3f8-bc43-4a01-8e28-2ced712ccb91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:20.557706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a357c3f8-bc43-4a01-8e28-2ced712ccb91 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:20.572064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-a357c3f8-bc43-4a01-8e28-2ced712ccb91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:40:20.572064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:20.572064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:20.572064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:20.572064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:20.572064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:20.572064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:20.572064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:20.572064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:20.572064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:20.572064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:20.572064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:20.572064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:40:20.572064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:40:20.572064 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:20.573211 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:20.574290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:40:20.574290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:40:20.574290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:40:20.574290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:40:20.574290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:40:20.574290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:40:20.574290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:40:20.574290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:40:20.574290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 629/1271] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:40:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:20.579648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a058c-fe62-4a8f-85b2-b28d002ceaff None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:20.644384 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:20.687763 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a058c-fe62-4a8f-85b2-b28d002ceaff None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:20.721930 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7507a54-59e1-4279-93c7-671a8bf3c1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:20.722569 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7507a54-59e1-4279-93c7-671a8bf3c1bd None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:20.722849 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7507a54-59e1-4279-93c7-671a8bf3c1bd None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:20.723115 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7507a54-59e1-4279-93c7-671a8bf3c1bd None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:20.724214 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7507a54-59e1-4279-93c7-671a8bf3c1bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:20.724514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7507a54-59e1-4279-93c7-671a8bf3c1bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:20.725942 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7507a54-59e1-4279-93c7-671a8bf3c1bd None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:20.737874 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 643/1272] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:40:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:40:20.744689 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357c3f8-bc43-4a01-8e28-2ced712ccb91 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:20.839102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:20.849682 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357c3f8-bc43-4a01-8e28-2ced712ccb91 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:20.889606 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64579719-1f74-40d0-9905-8fdfc3acf66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:20.890519 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64579719-1f74-40d0-9905-8fdfc3acf66f None admin] REQUEST_METHOD: `PUT` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:20.890795 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64579719-1f74-40d0-9905-8fdfc3acf66f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:20.891066 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64579719-1f74-40d0-9905-8fdfc3acf66f None admin] PATH_INFO: `/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/673a9bbc7c194dafb25fb007aa66b30a/roles/a33caf2301fb44db90304eb3d9afc3a5` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:20.893318 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64579719-1f74-40d0-9905-8fdfc3acf66f None admin] RBAC: Authorizing `identity:create_grant(project_id=ecf87429758641eeb462b3e409cadc9e, user_id=673a9bbc7c194dafb25fb007aa66b30a, role_id=a33caf2301fb44db90304eb3d9afc3a5)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:20.894792 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64579719-1f74-40d0-9905-8fdfc3acf66f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:20.914977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 630/1273] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:40:20 2022] PUT /identity/v3/projects/ecf87429758641eeb462b3e409cadc9e/users/673a9bbc7c194dafb25fb007aa66b30a/roles/a33caf2301fb44db90304eb3d9afc3a5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:40:23.087482 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406c7ac-fc0a-4a34-a774-18d76bbb127b None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:23.088024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406c7ac-fc0a-4a34-a774-18d76bbb127b None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:23.088024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406c7ac-fc0a-4a34-a774-18d76bbb127b None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:23.089805 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 644/1274] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:40:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:40:23.096260 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1e811-d215-4a2a-bac4-7f66feeaf6d7 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:23.097007 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1e811-d215-4a2a-bac4-7f66feeaf6d7 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:23.097151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1e811-d215-4a2a-bac4-7f66feeaf6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:23.124004 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-66e1e811-d215-4a2a-bac4-7f66feeaf6d7 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:40:23.125194 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-66e1e811-d215-4a2a-bac4-7f66feeaf6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:40:23.196539 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 631/1275] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:40:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:40:23.220684 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406c7ac-fc0a-4a34-a774-18d76bbb127b None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:23.330684 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406c7ac-fc0a-4a34-a774-18d76bbb127b None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:23.350999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:23.369786 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279a98c-0339-4692-88c5-7e78ed5193b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:23.371294 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279a98c-0339-4692-88c5-7e78ed5193b7 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:23.371571 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279a98c-0339-4692-88c5-7e78ed5193b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:23.371841 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279a98c-0339-4692-88c5-7e78ed5193b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:23.384888 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279a98c-0339-4692-88c5-7e78ed5193b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-0279a98c-0339-4692-88c5-7e78ed5193b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:40:23.396021 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:23.397534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:23.398513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:23.398513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:23.398513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:23.398513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:23.398513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:40:23.398513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:40:23.398513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:40:23.398513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:40:23.398513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:40:23.398513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:40:23.398513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:40:23.398513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:23.398513 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 645/1276] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:40:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:23.406000 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1e811-d215-4a2a-bac4-7f66feeaf6d7 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:23.493202 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:23.512564 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1e811-d215-4a2a-bac4-7f66feeaf6d7 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:23.545098 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7001c7e-1368-4d60-93a1-2557fe125190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:23.545753 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7001c7e-1368-4d60-93a1-2557fe125190 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:23.546029 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7001c7e-1368-4d60-93a1-2557fe125190 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:23.546296 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7001c7e-1368-4d60-93a1-2557fe125190 None admin] PATH_INFO: `/v3/roles` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:23.547049 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7001c7e-1368-4d60-93a1-2557fe125190 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:23.547355 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7001c7e-1368-4d60-93a1-2557fe125190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:23.548714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7001c7e-1368-4d60-93a1-2557fe125190 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:23.556508 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 632/1277] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:40:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:40:23.563221 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279a98c-0339-4692-88c5-7e78ed5193b7 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:23.671454 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279a98c-0339-4692-88c5-7e78ed5193b7 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:23.725382 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c9f5e-79af-446d-b7a4-d0efec8d6de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:23.726732 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c9f5e-79af-446d-b7a4-d0efec8d6de4 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:23.727200 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c9f5e-79af-446d-b7a4-d0efec8d6de4 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:23.727655 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c9f5e-79af-446d-b7a4-d0efec8d6de4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:23.746554 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c9f5e-79af-446d-b7a4-d0efec8d6de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:23.748040 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c9f5e-79af-446d-b7a4-d0efec8d6de4 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:23.765341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-741c9f5e-79af-446d-b7a4-d0efec8d6de4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:23.765341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:23.765341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:23.765341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:23.765341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:23.765341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:23.765341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:23.765341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:23.765341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:23.765341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:23.765341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:23.765341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:23.765341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:40:23.765341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:40:23.765341 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:23.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:23.767360 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:40:23.767360 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:40:23.767360 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:23.767360 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:23.767360 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 646/1278] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:23.773524 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7001c7e-1368-4d60-93a1-2557fe125190 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:23.882942 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7001c7e-1368-4d60-93a1-2557fe125190 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:23.920596 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b214ec-99b8-464e-a617-7829a279db3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:23.921244 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b214ec-99b8-464e-a617-7829a279db3f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:23.921526 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b214ec-99b8-464e-a617-7829a279db3f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:23.921793 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b214ec-99b8-464e-a617-7829a279db3f None admin] PATH_INFO: `/v3/domains` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:23.922538 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b214ec-99b8-464e-a617-7829a279db3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:23.922844 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b214ec-99b8-464e-a617-7829a279db3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:23.924224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b214ec-99b8-464e-a617-7829a279db3f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:23.937314 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 633/1279] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:23.943243 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c9f5e-79af-446d-b7a4-d0efec8d6de4 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:23.978011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:24.051738 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c9f5e-79af-446d-b7a4-d0efec8d6de4 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:24.087393 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db4cc5-0143-4eee-822d-054cdd8e419c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:24.088183 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db4cc5-0143-4eee-822d-054cdd8e419c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:24.088764 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db4cc5-0143-4eee-822d-054cdd8e419c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:24.088764 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db4cc5-0143-4eee-822d-054cdd8e419c None admin] PATH_INFO: `/v3/users/placement` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:24.097082 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db4cc5-0143-4eee-822d-054cdd8e419c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:24.098906 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db4cc5-0143-4eee-822d-054cdd8e419c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-41db4cc5-0143-4eee-822d-054cdd8e419c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:40:24.113795 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:24.114908 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:24.116141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:24.116141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:24.116141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:40:24.116141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:40:24.116141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:40:24.116141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:40:24.116141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:40:24.116141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:40:24.116141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:40:24.116141 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:24.118438 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 647/1280] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:24.120706 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b214ec-99b8-464e-a617-7829a279db3f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:24.124614 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:24.235396 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b214ec-99b8-464e-a617-7829a279db3f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:24.272969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61d13e-05c9-47bc-8cc3-e82f71e16a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:24.273628 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61d13e-05c9-47bc-8cc3-e82f71e16a0f None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:24.273913 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61d13e-05c9-47bc-8cc3-e82f71e16a0f None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:24.274182 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61d13e-05c9-47bc-8cc3-e82f71e16a0f None admin] PATH_INFO: `/v3/users` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:24.275048 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61d13e-05c9-47bc-8cc3-e82f71e16a0f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:24.275355 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61d13e-05c9-47bc-8cc3-e82f71e16a0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:24.276928 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61d13e-05c9-47bc-8cc3-e82f71e16a0f None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:24.294298 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 634/1281] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 19:40:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:24.300136 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db4cc5-0143-4eee-822d-054cdd8e419c None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:24.395548 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:24.405917 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db4cc5-0143-4eee-822d-054cdd8e419c None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:24.440459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868ebf7-d7f7-4094-a255-90f5294d5471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:24.441101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868ebf7-d7f7-4094-a255-90f5294d5471 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:24.441378 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868ebf7-d7f7-4094-a255-90f5294d5471 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:24.441652 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868ebf7-d7f7-4094-a255-90f5294d5471 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:24.511856 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868ebf7-d7f7-4094-a255-90f5294d5471 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:24.513263 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868ebf7-d7f7-4094-a255-90f5294d5471 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:24.605104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:24.620461 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 648/1282] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:40:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:40:24.626293 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61d13e-05c9-47bc-8cc3-e82f71e16a0f None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:24.689354 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:24.729745 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61d13e-05c9-47bc-8cc3-e82f71e16a0f None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:24.771343 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f49d4b-4cb8-43ab-b5d6-d57e1f4fc828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:24.772081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f49d4b-4cb8-43ab-b5d6-d57e1f4fc828 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:24.772363 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f49d4b-4cb8-43ab-b5d6-d57e1f4fc828 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:24.772638 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f49d4b-4cb8-43ab-b5d6-d57e1f4fc828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:24.783213 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f49d4b-4cb8-43ab-b5d6-d57e1f4fc828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:24.784665 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f49d4b-4cb8-43ab-b5d6-d57e1f4fc828 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:24.802482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-f8f49d4b-4cb8-43ab-b5d6-d57e1f4fc828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:24.802482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:24.802482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:24.802482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:24.802482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:24.802482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:24.802482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:24.802482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:24.802482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:24.802482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:24.802482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:24.802482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:24.802482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:40:24.802482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:40:24.802482 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:24.803410 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:24.804400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:40:24.804400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:40:24.804400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:40:24.804400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:40:24.804400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 635/1283] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:24.809883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868ebf7-d7f7-4094-a255-90f5294d5471 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:24.910013 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868ebf7-d7f7-4094-a255-90f5294d5471 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:24.950269 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f16d78-da6f-4147-a2fb-55ea8db6bba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:24.950952 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f16d78-da6f-4147-a2fb-55ea8db6bba2 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:24.951232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f16d78-da6f-4147-a2fb-55ea8db6bba2 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:24.951501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f16d78-da6f-4147-a2fb-55ea8db6bba2 None admin] PATH_INFO: `/v3/domains` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:24.952222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f16d78-da6f-4147-a2fb-55ea8db6bba2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:24.952525 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f16d78-da6f-4147-a2fb-55ea8db6bba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:24.953907 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f16d78-da6f-4147-a2fb-55ea8db6bba2 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:24.966517 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 649/1284] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:24.973008 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f49d4b-4cb8-43ab-b5d6-d57e1f4fc828 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:25.086732 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f49d4b-4cb8-43ab-b5d6-d57e1f4fc828 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:25.101233 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:25.124078 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b573cd-07a8-43ae-9cbc-79899359cbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:25.124860 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b573cd-07a8-43ae-9cbc-79899359cbc3 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:25.125134 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b573cd-07a8-43ae-9cbc-79899359cbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:25.125403 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b573cd-07a8-43ae-9cbc-79899359cbc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:25.132744 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b573cd-07a8-43ae-9cbc-79899359cbc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:25.134195 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b573cd-07a8-43ae-9cbc-79899359cbc3 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:25.148117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application [None req-52b573cd-07a8-43ae-9cbc-79899359cbc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:40:25.148117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:25.148117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:25.148117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:25.148117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:25.148117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:25.148117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:25.148117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:25.148117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:25.148117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:25.148117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:25.148117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:25.148117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:40:25.148117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:40:25.148117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application with Lock( May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:25.149110 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:25.150361 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:40:25.150361 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:40:25.150361 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:40:25.150361 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:40:25.150361 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:40:25.150361 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:40:25.150361 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:40:25.150361 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: ERROR keystone.server.flask.application May 19 19:40:25.150361 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 636/1285] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:40:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:25.155396 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f16d78-da6f-4147-a2fb-55ea8db6bba2 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:25.205205 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:25.267414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f16d78-da6f-4147-a2fb-55ea8db6bba2 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:25.302375 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8eeab3-80f0-4ddb-8424-2d6a85fa2896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:25.303027 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8eeab3-80f0-4ddb-8424-2d6a85fa2896 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:25.303308 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8eeab3-80f0-4ddb-8424-2d6a85fa2896 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:25.303577 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8eeab3-80f0-4ddb-8424-2d6a85fa2896 None admin] PATH_INFO: `/v3/projects` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:25.304367 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8eeab3-80f0-4ddb-8424-2d6a85fa2896 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:25.304674 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8eeab3-80f0-4ddb-8424-2d6a85fa2896 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:25.306126 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8eeab3-80f0-4ddb-8424-2d6a85fa2896 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:25.318442 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 650/1286] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:40:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:25.324872 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b573cd-07a8-43ae-9cbc-79899359cbc3 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:25.437512 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b573cd-07a8-43ae-9cbc-79899359cbc3 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:25.477030 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b40c0-f87f-409b-a8cd-78c3b78c9459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:25.477688 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b40c0-f87f-409b-a8cd-78c3b78c9459 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:25.477969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b40c0-f87f-409b-a8cd-78c3b78c9459 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:25.478240 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b40c0-f87f-409b-a8cd-78c3b78c9459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:25.479117 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b40c0-f87f-409b-a8cd-78c3b78c9459 None admin] RBAC: Adding query filter params (role.id=a33caf2301fb44db90304eb3d9afc3a5, scope.project.id=ecf87429758641eeb462b3e409cadc9e, user.id=673a9bbc7c194dafb25fb007aa66b30a) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:25.479522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b40c0-f87f-409b-a8cd-78c3b78c9459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:25.480976 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b40c0-f87f-409b-a8cd-78c3b78c9459 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:25.487199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 637/1287] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:40:25 2022] GET /identity/v3/role_assignments?user.id=673a9bbc7c194dafb25fb007aa66b30a&scope.project.id=ecf87429758641eeb462b3e409cadc9e&role.id=a33caf2301fb44db90304eb3d9afc3a5 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:27.688834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61632709-1723-408d-a616-1ea2cfd5b6f3 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:27.689648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61632709-1723-408d-a616-1ea2cfd5b6f3 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:27.689648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61632709-1723-408d-a616-1ea2cfd5b6f3 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:27.691946 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 651/1288] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:40:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:40:27.699826 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe8e38-cb76-4df4-8be9-63d94fcbba49 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:27.700258 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe8e38-cb76-4df4-8be9-63d94fcbba49 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:27.700376 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe8e38-cb76-4df4-8be9-63d94fcbba49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:27.718796 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-d0fe8e38-cb76-4df4-8be9-63d94fcbba49 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:40:27.719953 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-d0fe8e38-cb76-4df4-8be9-63d94fcbba49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:40:27.759456 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:27.798292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 638/1289] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:40:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:40:27.830080 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61632709-1723-408d-a616-1ea2cfd5b6f3 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:27.942594 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61632709-1723-408d-a616-1ea2cfd5b6f3 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:27.953232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:27.980665 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dff4d-3ecf-4035-9ff3-b94718067131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:27.981445 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dff4d-3ecf-4035-9ff3-b94718067131 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:27.981719 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dff4d-3ecf-4035-9ff3-b94718067131 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:27.981935 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dff4d-3ecf-4035-9ff3-b94718067131 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:27.982619 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91dff4d-3ecf-4035-9ff3-b94718067131 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:27.984015 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91dff4d-3ecf-4035-9ff3-b94718067131 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-d91dff4d-3ecf-4035-9ff3-b94718067131 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:27.996443 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:27.997644 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:27.998537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:27.998537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:27.998537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:27.998537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:27.998537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:27.998537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:27.998537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:27.998537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:27.998537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:27.998537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:27.998537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:40:27.998537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:40:27.998537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:40:27.998537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:27.998537 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 652/1290] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:40:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:28.003947 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe8e38-cb76-4df4-8be9-63d94fcbba49 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:28.114157 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe8e38-cb76-4df4-8be9-63d94fcbba49 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:28.153041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368c4a5-ece9-4067-ac1c-31b6fad50f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:28.153696 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368c4a5-ece9-4067-ac1c-31b6fad50f22 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:28.153971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368c4a5-ece9-4067-ac1c-31b6fad50f22 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:28.154242 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368c4a5-ece9-4067-ac1c-31b6fad50f22 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:28.154951 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368c4a5-ece9-4067-ac1c-31b6fad50f22 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:28.155282 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368c4a5-ece9-4067-ac1c-31b6fad50f22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:28.156648 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368c4a5-ece9-4067-ac1c-31b6fad50f22 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:28.161499 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 639/1291] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 19:40:28 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:40:28.167602 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dff4d-3ecf-4035-9ff3-b94718067131 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:28.282222 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dff4d-3ecf-4035-9ff3-b94718067131 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:28.315511 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0129712-9172-47d4-adb3-d0fbe8a4f18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:28.316164 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0129712-9172-47d4-adb3-d0fbe8a4f18c None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:28.316439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0129712-9172-47d4-adb3-d0fbe8a4f18c None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:28.316707 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0129712-9172-47d4-adb3-d0fbe8a4f18c None admin] PATH_INFO: `/v3/services` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:28.317415 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0129712-9172-47d4-adb3-d0fbe8a4f18c None admin] RBAC: Adding query filter params (type=placement) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:28.317719 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0129712-9172-47d4-adb3-d0fbe8a4f18c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:28.319116 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0129712-9172-47d4-adb3-d0fbe8a4f18c None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:28.324105 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 653/1292] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 19:40:28 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:30.498659 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbce39-0126-4893-ae9f-c2f51c6bb181 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:30.499551 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbce39-0126-4893-ae9f-c2f51c6bb181 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:30.499551 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbce39-0126-4893-ae9f-c2f51c6bb181 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:30.501753 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 640/1293] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:40:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:40:30.510891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ec3e1-d907-4727-aefd-bab6456f67ac None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:30.511414 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ec3e1-d907-4727-aefd-bab6456f67ac None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:30.511917 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ec3e1-d907-4727-aefd-bab6456f67ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:30.535597 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-f44ec3e1-d907-4727-aefd-bab6456f67ac None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:40:30.536793 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-f44ec3e1-d907-4727-aefd-bab6456f67ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:40:30.569604 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:30.616840 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 654/1294] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:40:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:40:30.649564 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbce39-0126-4893-ae9f-c2f51c6bb181 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:30.684422 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:30.760538 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbce39-0126-4893-ae9f-c2f51c6bb181 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:30.795727 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef692994-6090-4aa9-b3c1-6561d025670b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:30.796380 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef692994-6090-4aa9-b3c1-6561d025670b None admin] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:30.796665 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef692994-6090-4aa9-b3c1-6561d025670b None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:30.796938 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef692994-6090-4aa9-b3c1-6561d025670b None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:30.797651 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef692994-6090-4aa9-b3c1-6561d025670b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:30.799049 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef692994-6090-4aa9-b3c1-6561d025670b None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:30.821646 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 641/1295] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:40:30 2022] POST /identity/v3/services => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:40:32.995189 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f75ea-febd-465b-a5be-11c6d2ee79aa None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:32.996015 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f75ea-febd-465b-a5be-11c6d2ee79aa None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:32.996387 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f75ea-febd-465b-a5be-11c6d2ee79aa None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:32.998823 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 655/1296] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:40:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:40:33.007693 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cda024-ed0b-4b41-927f-005aea24eed8 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:33.008119 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cda024-ed0b-4b41-927f-005aea24eed8 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:33.008267 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cda024-ed0b-4b41-927f-005aea24eed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:33.027658 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-e3cda024-ed0b-4b41-927f-005aea24eed8 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:40:33.028843 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-e3cda024-ed0b-4b41-927f-005aea24eed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:40:33.117381 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 642/1297] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:40:33 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:40:33.140553 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f75ea-febd-465b-a5be-11c6d2ee79aa None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:33.249489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f75ea-febd-465b-a5be-11c6d2ee79aa None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:33.284502 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6b0a8-76b8-4b70-9879-f4c33d5e4dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:33.285281 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6b0a8-76b8-4b70-9879-f4c33d5e4dd4 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:33.285556 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6b0a8-76b8-4b70-9879-f4c33d5e4dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:33.285819 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6b0a8-76b8-4b70-9879-f4c33d5e4dd4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:33.286494 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6b0a8-76b8-4b70-9879-f4c33d5e4dd4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:33.287865 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6b0a8-76b8-4b70-9879-f4c33d5e4dd4 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-61e6b0a8-76b8-4b70-9879-f4c33d5e4dd4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:33.299883 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:33.300991 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:33.301999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:33.301999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:33.301999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:33.301999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:33.301999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:33.301999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:33.301999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:33.301999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:33.301999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:33.301999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:33.301999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:40:33.301999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:40:33.301999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:40:33.301999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:33.301999 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 656/1298] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:40:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:33.310759 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cda024-ed0b-4b41-927f-005aea24eed8 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:33.330689 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:33.413889 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cda024-ed0b-4b41-927f-005aea24eed8 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:33.449581 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6e242-6018-4f31-b9bc-26ac74302df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:33.450242 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e242-6018-4f31-b9bc-26ac74302df8 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:33.450549 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e242-6018-4f31-b9bc-26ac74302df8 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:33.450812 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e242-6018-4f31-b9bc-26ac74302df8 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:33.451501 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6e242-6018-4f31-b9bc-26ac74302df8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:33.451808 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6e242-6018-4f31-b9bc-26ac74302df8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:33.453172 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6e242-6018-4f31-b9bc-26ac74302df8 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:33.458392 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 643/1299] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 19:40:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:40:33.465006 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6b0a8-76b8-4b70-9879-f4c33d5e4dd4 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:33.515360 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:33.593704 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6b0a8-76b8-4b70-9879-f4c33d5e4dd4 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:33.631304 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eba998-61e4-456f-99d0-69433d836c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:33.632233 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eba998-61e4-456f-99d0-69433d836c7b None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:33.632597 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eba998-61e4-456f-99d0-69433d836c7b None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:33.632950 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eba998-61e4-456f-99d0-69433d836c7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:33.634138 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eba998-61e4-456f-99d0-69433d836c7b None admin] RBAC: Adding query filter params (interface=public, service_id=718525605de14c019dca607e894343dc, region_id=RegionOne) {{(pid=86981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:33.634590 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eba998-61e4-456f-99d0-69433d836c7b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:33.636379 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eba998-61e4-456f-99d0-69433d836c7b None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:33.644356 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 657/1300] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:40:33 2022] GET /identity/v3/endpoints?service_id=718525605de14c019dca607e894343dc&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:33.652902 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6e242-6018-4f31-b9bc-26ac74302df8 None admin] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:33.738952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:33.760698 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6e242-6018-4f31-b9bc-26ac74302df8 None admin] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:33.800724 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f53186-7be7-4cc4-a90e-b28071c63fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:33.801389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f53186-7be7-4cc4-a90e-b28071c63fc5 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:33.801668 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f53186-7be7-4cc4-a90e-b28071c63fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:33.801937 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f53186-7be7-4cc4-a90e-b28071c63fc5 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:33.802652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f53186-7be7-4cc4-a90e-b28071c63fc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:33.804036 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f53186-7be7-4cc4-a90e-b28071c63fc5 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:33.810962 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 644/1301] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:40:33 2022] GET /identity/v3/services => generated 2191 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:40:35.966221 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf1062-e2d2-4afa-8e38-7a0663ee8488 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:35.967131 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf1062-e2d2-4afa-8e38-7a0663ee8488 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:35.967131 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf1062-e2d2-4afa-8e38-7a0663ee8488 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:35.969846 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 658/1302] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:40:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:40:35.975620 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f10668-5296-4d94-a8dc-5b47fe8117fa None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:35.976039 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f10668-5296-4d94-a8dc-5b47fe8117fa None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:35.976151 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f10668-5296-4d94-a8dc-5b47fe8117fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:35.993328 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-20f10668-5296-4d94-a8dc-5b47fe8117fa None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:40:35.994521 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-20f10668-5296-4d94-a8dc-5b47fe8117fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:40:36.065599 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 645/1303] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:40:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:40:36.089084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf1062-e2d2-4afa-8e38-7a0663ee8488 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:36.201889 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:36.211067 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf1062-e2d2-4afa-8e38-7a0663ee8488 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:36.248751 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48310112-ff3a-4a8c-b460-bc35310058b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:36.249468 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48310112-ff3a-4a8c-b460-bc35310058b4 None admin] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:36.249742 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48310112-ff3a-4a8c-b460-bc35310058b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:36.250013 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48310112-ff3a-4a8c-b460-bc35310058b4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:36.250699 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48310112-ff3a-4a8c-b460-bc35310058b4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:36.252121 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48310112-ff3a-4a8c-b460-bc35310058b4 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application [None req-48310112-ff3a-4a8c-b460-bc35310058b4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:40:36.263854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:40:36.265182 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:40:36.266603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:40:36.266603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application with Lock( May 19 19:40:36.266603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:40:36.266603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self._enter() May 19 19:40:36.266603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:40:36.266603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:40:36.266603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:40:36.266603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application return self.creator() May 19 19:40:36.266603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:40:36.266603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application created_value = creator( May 19 19:40:36.266603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:40:36.266603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:40:36.266603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:40:36.266603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: ERROR keystone.server.flask.application May 19 19:40:36.266603 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 659/1304] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:40:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:40:36.274522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f10668-5296-4d94-a8dc-5b47fe8117fa None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:36.360907 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:36.386188 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f10668-5296-4d94-a8dc-5b47fe8117fa None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:36.430578 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e3d39-3fbd-4839-a5af-29105910aba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:36.431517 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e3d39-3fbd-4839-a5af-29105910aba0 None admin] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:36.431899 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e3d39-3fbd-4839-a5af-29105910aba0 None admin] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:36.432212 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e3d39-3fbd-4839-a5af-29105910aba0 None admin] PATH_INFO: `/v3/services` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:36.433241 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e3d39-3fbd-4839-a5af-29105910aba0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86982) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:40:36.433626 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e3d39-3fbd-4839-a5af-29105910aba0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:36.435349 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e3d39-3fbd-4839-a5af-29105910aba0 None admin] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:36.441302 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 646/1305] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 19:40:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:40:36.448008 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48310112-ff3a-4a8c-b460-bc35310058b4 None admin] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:40:36.564792 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48310112-ff3a-4a8c-b460-bc35310058b4 None admin] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:40:36.602982 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4de9a4-2d11-47c2-b428-f330af5cb506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5800b131869435eb3ae07f86d69929e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:40:36.603624 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4de9a4-2d11-47c2-b428-f330af5cb506 None admin] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:40:36.603897 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4de9a4-2d11-47c2-b428-f330af5cb506 None admin] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:40:36.604162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4de9a4-2d11-47c2-b428-f330af5cb506 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:40:36.604891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4de9a4-2d11-47c2-b428-f330af5cb506 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:40:36.606309 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4de9a4-2d11-47c2-b428-f330af5cb506 None admin] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:40:36.629136 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 660/1306] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:40:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:40:46.437738 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=30 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:40:46.624534 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:41.268977 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003d0fc-cb2c-4387-80f5-fdb1b367e30d None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:41.269563 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003d0fc-cb2c-4387-80f5-fdb1b367e30d None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:41.269835 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003d0fc-cb2c-4387-80f5-fdb1b367e30d None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:41.276584 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77e2d1-d6f9-4fd7-ab74-d8de2f524714 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:41.277028 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77e2d1-d6f9-4fd7-ab74-d8de2f524714 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:41.277343 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77e2d1-d6f9-4fd7-ab74-d8de2f524714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:41.281199 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 647/1307] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:41:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:41.370046 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-0c77e2d1-d6f9-4fd7-ab74-d8de2f524714 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:41.371726 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-0c77e2d1-d6f9-4fd7-ab74-d8de2f524714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:41.539328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 661/1308] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 19:41:41.608562 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21317af2-34a6-4a25-a742-3227168605cd None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:41.609142 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21317af2-34a6-4a25-a742-3227168605cd None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:41.609645 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21317af2-34a6-4a25-a742-3227168605cd None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:41.611305 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 648/1309] 10.222.0.84 () {54 vars in 964 bytes} [Thu May 19 19:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:41.617532 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783f9da-5b11-46ba-9db2-b8b683ced83d None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:41.618075 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783f9da-5b11-46ba-9db2-b8b683ced83d None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:41.618599 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783f9da-5b11-46ba-9db2-b8b683ced83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:41.742992 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-f783f9da-5b11-46ba-9db2-b8b683ced83d None None] MFA Rules not processed for user `d2b98b1e4024410a87addde112237f32`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:41.744514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-f783f9da-5b11-46ba-9db2-b8b683ced83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:41.773036 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:41.859467 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 662/1310] 10.222.0.84 () {58 vars in 1061 bytes} [Thu May 19 19:41:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:41:41.860522 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21317af2-34a6-4a25-a742-3227168605cd None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:41.862331 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-21317af2-34a6-4a25-a742-3227168605cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:41.989969 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21317af2-34a6-4a25-a742-3227168605cd None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:42.078166 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870edc7-1c7f-451d-a713-798541c65ffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b98b1e4024410a87addde112237f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:42.079183 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870edc7-1c7f-451d-a713-798541c65ffa service neutron] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:42.079609 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870edc7-1c7f-451d-a713-798541c65ffa service neutron] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:42.080021 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870edc7-1c7f-451d-a713-798541c65ffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:42.081939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-d870edc7-1c7f-451d-a713-798541c65ffa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:42.142224 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:42.210292 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870edc7-1c7f-451d-a713-798541c65ffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:42.212062 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870edc7-1c7f-451d-a713-798541c65ffa service neutron] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:42.248821 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 649/1311] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:41:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 19:41:44.821492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac05a5-c632-4015-b6f5-2c41980b072f None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:44.822057 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac05a5-c632-4015-b6f5-2c41980b072f None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:44.822057 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac05a5-c632-4015-b6f5-2c41980b072f None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:44.823660 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 663/1312] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:44.828852 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5635197-8517-4ffc-bad8-70a788cb4c2f None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:44.829251 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5635197-8517-4ffc-bad8-70a788cb4c2f None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:44.829397 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5635197-8517-4ffc-bad8-70a788cb4c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:44.913470 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-f5635197-8517-4ffc-bad8-70a788cb4c2f None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:44.914714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-f5635197-8517-4ffc-bad8-70a788cb4c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:45.013572 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 650/1313] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:41:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:41:45.051862 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac05a5-c632-4015-b6f5-2c41980b072f None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:45.053439 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-9cac05a5-c632-4015-b6f5-2c41980b072f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:45.164504 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac05a5-c632-4015-b6f5-2c41980b072f None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:45.240643 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:45.243462 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815e502-5c2d-42a5-a23c-170ac7940fa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b98b1e4024410a87addde112237f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:45.244188 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815e502-5c2d-42a5-a23c-170ac7940fa2 service neutron] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:45.244475 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815e502-5c2d-42a5-a23c-170ac7940fa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:45.244746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815e502-5c2d-42a5-a23c-170ac7940fa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:45.246847 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-f815e502-5c2d-42a5-a23c-170ac7940fa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:45.361694 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815e502-5c2d-42a5-a23c-170ac7940fa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:45.363180 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815e502-5c2d-42a5-a23c-170ac7940fa2 service neutron] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:45.381743 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 664/1314] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:41:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:41:48.005000 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47abff-64fd-4b78-97be-386a0ad4c467 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:48.005529 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47abff-64fd-4b78-97be-386a0ad4c467 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:48.005529 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47abff-64fd-4b78-97be-386a0ad4c467 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:48.006864 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 651/1315] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:48.013156 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471062ac-1481-49af-9893-99fae36edc08 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:48.013648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471062ac-1481-49af-9893-99fae36edc08 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:48.013648 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471062ac-1481-49af-9893-99fae36edc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:48.084593 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-471062ac-1481-49af-9893-99fae36edc08 None None] MFA Rules not processed for user `6c5e8aab0eaa42288cf4b090c1e64ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:48.085826 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-471062ac-1481-49af-9893-99fae36edc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:48.143393 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:48.190364 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 665/1316] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:41:48 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:41:48.232817 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47abff-64fd-4b78-97be-386a0ad4c467 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:48.234409 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-6a47abff-64fd-4b78-97be-386a0ad4c467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:48.244288 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:48.351497 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47abff-64fd-4b78-97be-386a0ad4c467 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:48.433811 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0913f-9e6f-4c95-815a-83aef87fd47e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b98b1e4024410a87addde112237f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:48.435593 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0913f-9e6f-4c95-815a-83aef87fd47e service neutron] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:48.435971 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0913f-9e6f-4c95-815a-83aef87fd47e service neutron] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:48.436426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0913f-9e6f-4c95-815a-83aef87fd47e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:48.438749 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-7ca0913f-9e6f-4c95-815a-83aef87fd47e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:48.521005 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:48.572298 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0913f-9e6f-4c95-815a-83aef87fd47e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:48.573752 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0913f-9e6f-4c95-815a-83aef87fd47e service neutron] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:48.594650 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 652/1317] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:41:48 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:41:51.320811 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ed3d1-82b8-48fa-9c9c-ca288080736d None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:51.321706 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ed3d1-82b8-48fa-9c9c-ca288080736d None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:51.321706 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ed3d1-82b8-48fa-9c9c-ca288080736d None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:51.324008 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 666/1318] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:41:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:51.330222 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff12f97-f53d-4ee9-be2f-8366ebe5aa82 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:51.330686 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff12f97-f53d-4ee9-be2f-8366ebe5aa82 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:51.330799 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff12f97-f53d-4ee9-be2f-8366ebe5aa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:51.404415 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-0ff12f97-f53d-4ee9-be2f-8366ebe5aa82 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:51.405618 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-0ff12f97-f53d-4ee9-be2f-8366ebe5aa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:51.506155 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 653/1319] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:41:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:41:51.547102 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ed3d1-82b8-48fa-9c9c-ca288080736d None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:51.548655 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-f80ed3d1-82b8-48fa-9c9c-ca288080736d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:51.663202 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ed3d1-82b8-48fa-9c9c-ca288080736d None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:51.745430 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb73e4e-ef89-4fa3-b177-b0d61240cf49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b98b1e4024410a87addde112237f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:51.746084 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb73e4e-ef89-4fa3-b177-b0d61240cf49 service neutron] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:51.746402 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb73e4e-ef89-4fa3-b177-b0d61240cf49 service neutron] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:51.746676 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb73e4e-ef89-4fa3-b177-b0d61240cf49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:51.748828 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-4fb73e4e-ef89-4fa3-b177-b0d61240cf49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:51.757746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:51.875485 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb73e4e-ef89-4fa3-b177-b0d61240cf49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:51.876923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb73e4e-ef89-4fa3-b177-b0d61240cf49 service neutron] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:51.897101 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 667/1320] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:41:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:41:54.936900 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2cd35-ec52-4c9c-9531-e6ea69c60986 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:54.937453 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2cd35-ec52-4c9c-9531-e6ea69c60986 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:54.937453 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2cd35-ec52-4c9c-9531-e6ea69c60986 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:54.938745 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 654/1321] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:54.944714 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca70ca-e838-47bf-a25b-c164c5f5ee9d None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:54.945153 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca70ca-e838-47bf-a25b-c164c5f5ee9d None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:54.945242 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca70ca-e838-47bf-a25b-c164c5f5ee9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:55.027308 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-d9ca70ca-e838-47bf-a25b-c164c5f5ee9d None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:55.028834 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-d9ca70ca-e838-47bf-a25b-c164c5f5ee9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:55.105838 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:55.147394 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 668/1322] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:41:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:41:55.198364 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69c6c0-18c1-4a3a-8efb-3d887fb73cbc None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:55.198597 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69c6c0-18c1-4a3a-8efb-3d887fb73cbc None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:55.198997 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69c6c0-18c1-4a3a-8efb-3d887fb73cbc None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:55.200564 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 655/1323] 10.222.0.84 () {54 vars in 964 bytes} [Thu May 19 19:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:55.206831 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8648e8-7405-4dab-a51f-1848ccd7d915 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:55.207120 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8648e8-7405-4dab-a51f-1848ccd7d915 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:55.207401 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8648e8-7405-4dab-a51f-1848ccd7d915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:55.286302 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-6d8648e8-7405-4dab-a51f-1848ccd7d915 None None] MFA Rules not processed for user `d2b98b1e4024410a87addde112237f32`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:55.287515 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-6d8648e8-7405-4dab-a51f-1848ccd7d915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:55.364796 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 669/1324] 10.222.0.84 () {58 vars in 1061 bytes} [Thu May 19 19:41:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:41:55.374108 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69c6c0-18c1-4a3a-8efb-3d887fb73cbc None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:55.375936 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-2e69c6c0-18c1-4a3a-8efb-3d887fb73cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:55.395617 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:55.482546 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69c6c0-18c1-4a3a-8efb-3d887fb73cbc None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:55.559457 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22719268-34ea-460f-ab09-dfda28bb2107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b98b1e4024410a87addde112237f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:55.560134 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22719268-34ea-460f-ab09-dfda28bb2107 service neutron] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:55.560406 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22719268-34ea-460f-ab09-dfda28bb2107 service neutron] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:55.560675 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22719268-34ea-460f-ab09-dfda28bb2107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:55.562303 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-22719268-34ea-460f-ab09-dfda28bb2107 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:55.636227 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:55.673823 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22719268-34ea-460f-ab09-dfda28bb2107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:55.675297 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22719268-34ea-460f-ab09-dfda28bb2107 service neutron] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:55.694311 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 656/1325] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:41:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:41:58.248459 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac4a3d-1bc6-4324-84f3-d420cc0f5a61 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:58.249007 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac4a3d-1bc6-4324-84f3-d420cc0f5a61 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:58.249098 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac4a3d-1bc6-4324-84f3-d420cc0f5a61 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:58.250466 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 670/1326] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:58.256070 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cac26b-489d-490e-9965-0de3196eb1f5 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:58.256490 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cac26b-489d-490e-9965-0de3196eb1f5 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:58.256652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cac26b-489d-490e-9965-0de3196eb1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:58.322695 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-b4cac26b-489d-490e-9965-0de3196eb1f5 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:58.323913 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-b4cac26b-489d-490e-9965-0de3196eb1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:58.432504 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 657/1327] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:41:58.467977 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac4a3d-1bc6-4324-84f3-d420cc0f5a61 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:58.469502 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-feac4a3d-1bc6-4324-84f3-d420cc0f5a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:58.533245 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:58.577077 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac4a3d-1bc6-4324-84f3-d420cc0f5a61 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:58.655574 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3ba35-56dc-4ba5-8028-18b8d8b849d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b98b1e4024410a87addde112237f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:58.656218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3ba35-56dc-4ba5-8028-18b8d8b849d8 service neutron] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:58.656489 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3ba35-56dc-4ba5-8028-18b8d8b849d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:58.656760 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3ba35-56dc-4ba5-8028-18b8d8b849d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:58.658410 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-31b3ba35-56dc-4ba5-8028-18b8d8b849d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:58.767011 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:58.769485 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3ba35-56dc-4ba5-8028-18b8d8b849d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:58.771405 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3ba35-56dc-4ba5-8028-18b8d8b849d8 service neutron] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:58.790232 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 671/1328] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:41:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 19:42:01.120127 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80bf53-d188-4093-8add-49fabc457e35 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:01.120688 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80bf53-d188-4093-8add-49fabc457e35 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:01.120688 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80bf53-d188-4093-8add-49fabc457e35 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:01.122306 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 658/1329] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:01.127833 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51a4b2-eb13-42f2-925f-3b17a274f80a None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:01.128280 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51a4b2-eb13-42f2-925f-3b17a274f80a None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:01.128386 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51a4b2-eb13-42f2-925f-3b17a274f80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:01.194278 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-df51a4b2-eb13-42f2-925f-3b17a274f80a None None] MFA Rules not processed for user `6c5e8aab0eaa42288cf4b090c1e64ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:01.195601 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-df51a4b2-eb13-42f2-925f-3b17a274f80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:01.295615 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 672/1330] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:42:01 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:42:01.334116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80bf53-d188-4093-8add-49fabc457e35 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:01.335684 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-6d80bf53-d188-4093-8add-49fabc457e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:01.369378 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:01.446034 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80bf53-d188-4093-8add-49fabc457e35 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:01.521514 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479ca9c-6d64-4b39-b0db-7579d7b92fd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b98b1e4024410a87addde112237f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:01.522156 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479ca9c-6d64-4b39-b0db-7579d7b92fd9 service neutron] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:01.522464 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479ca9c-6d64-4b39-b0db-7579d7b92fd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:01.522733 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479ca9c-6d64-4b39-b0db-7579d7b92fd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:01.524340 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-9479ca9c-6d64-4b39-b0db-7579d7b92fd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:01.605400 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:01.640095 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479ca9c-6d64-4b39-b0db-7579d7b92fd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:01.641572 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479ca9c-6d64-4b39-b0db-7579d7b92fd9 service neutron] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:01.663243 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 659/1331] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:42:01 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:42:04.150771 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8795eb-b9c7-4e93-96ab-b98cc259ab65 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:04.151280 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8795eb-b9c7-4e93-96ab-b98cc259ab65 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:04.151280 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8795eb-b9c7-4e93-96ab-b98cc259ab65 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:04.152690 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 673/1332] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:04.158590 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3a4b4-7950-41f5-b20f-9e603a645e52 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:04.159081 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3a4b4-7950-41f5-b20f-9e603a645e52 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:04.159164 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3a4b4-7950-41f5-b20f-9e603a645e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:04.231737 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-ffb3a4b4-7950-41f5-b20f-9e603a645e52 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:04.232912 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-ffb3a4b4-7950-41f5-b20f-9e603a645e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:04.329437 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 660/1333] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:42:04.365433 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8795eb-b9c7-4e93-96ab-b98cc259ab65 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:04.367048 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-0b8795eb-b9c7-4e93-96ab-b98cc259ab65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:04.452136 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:04.487668 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8795eb-b9c7-4e93-96ab-b98cc259ab65 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:04.572319 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb4942-e38a-4a8d-b382-73b4d9502064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b98b1e4024410a87addde112237f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:04.572959 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb4942-e38a-4a8d-b382-73b4d9502064 service neutron] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:04.573198 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb4942-e38a-4a8d-b382-73b4d9502064 service neutron] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:04.573492 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb4942-e38a-4a8d-b382-73b4d9502064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:04.575218 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-f7eb4942-e38a-4a8d-b382-73b4d9502064 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:04.690024 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb4942-e38a-4a8d-b382-73b4d9502064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:04.691514 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb4942-e38a-4a8d-b382-73b4d9502064 service neutron] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:04.701588 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:04.710994 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 674/1334] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:42:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:42:07.029939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1472cf4-243b-4189-a32d-b1e5cb6d7389 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:07.030502 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1472cf4-243b-4189-a32d-b1e5cb6d7389 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:07.030502 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1472cf4-243b-4189-a32d-b1e5cb6d7389 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:07.031826 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 661/1335] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:07.038417 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fe478-4b16-4991-96f6-03f541ce3493 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:07.038855 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fe478-4b16-4991-96f6-03f541ce3493 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:07.039028 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fe478-4b16-4991-96f6-03f541ce3493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:07.113544 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-380fe478-4b16-4991-96f6-03f541ce3493 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:07.114763 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-380fe478-4b16-4991-96f6-03f541ce3493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:07.237169 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 675/1336] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:42:07.278058 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1472cf4-243b-4189-a32d-b1e5cb6d7389 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:07.280952 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-b1472cf4-243b-4189-a32d-b1e5cb6d7389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:07.328598 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:07.395409 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1472cf4-243b-4189-a32d-b1e5cb6d7389 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:07.474040 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1fa33-11ec-4ca5-939c-7bc1465f9476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b98b1e4024410a87addde112237f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:07.475041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1fa33-11ec-4ca5-939c-7bc1465f9476 service neutron] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:07.475328 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1fa33-11ec-4ca5-939c-7bc1465f9476 service neutron] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:07.475599 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1fa33-11ec-4ca5-939c-7bc1465f9476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:07.477252 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-92d1fa33-11ec-4ca5-939c-7bc1465f9476 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:07.566301 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:07.594053 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1fa33-11ec-4ca5-939c-7bc1465f9476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:07.595544 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1fa33-11ec-4ca5-939c-7bc1465f9476 service neutron] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:07.615926 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 662/1337] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:42:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:42:10.518800 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475df40b-c978-4cf0-9cd7-9ecad7c2d4cd None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:10.519438 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475df40b-c978-4cf0-9cd7-9ecad7c2d4cd None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:10.519438 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475df40b-c978-4cf0-9cd7-9ecad7c2d4cd None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:10.520747 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 676/1338] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:10.527069 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade68172-8f38-46a1-9062-469a7ef70dc8 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:10.527966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade68172-8f38-46a1-9062-469a7ef70dc8 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:10.527966 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade68172-8f38-46a1-9062-469a7ef70dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:10.600652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-ade68172-8f38-46a1-9062-469a7ef70dc8 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:10.601835 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-ade68172-8f38-46a1-9062-469a7ef70dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:10.722973 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 663/1339] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:42:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:42:10.759425 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475df40b-c978-4cf0-9cd7-9ecad7c2d4cd None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:10.760970 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-475df40b-c978-4cf0-9cd7-9ecad7c2d4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:10.852112 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:10.882418 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475df40b-c978-4cf0-9cd7-9ecad7c2d4cd None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:10.957284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66df92d-e686-4304-9eda-8fd42d78f820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b98b1e4024410a87addde112237f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:10.957923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66df92d-e686-4304-9eda-8fd42d78f820 service neutron] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:10.958193 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66df92d-e686-4304-9eda-8fd42d78f820 service neutron] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:10.958501 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66df92d-e686-4304-9eda-8fd42d78f820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:10.960136 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-f66df92d-e686-4304-9eda-8fd42d78f820 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:11.074879 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66df92d-e686-4304-9eda-8fd42d78f820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:11.076356 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66df92d-e686-4304-9eda-8fd42d78f820 service neutron] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:11.095004 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 677/1340] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:42:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:42:15.549014 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcbcf0-d545-4828-b1b4-0d09b3e83fd5 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:15.549585 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcbcf0-d545-4828-b1b4-0d09b3e83fd5 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:15.549585 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcbcf0-d545-4828-b1b4-0d09b3e83fd5 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:15.551134 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 664/1341] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:15.556804 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b65fec-50f5-4ae7-865e-ac6dd64e9730 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:15.557247 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b65fec-50f5-4ae7-865e-ac6dd64e9730 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:15.557340 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b65fec-50f5-4ae7-865e-ac6dd64e9730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:15.564213 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=100 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:15.630994 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-d1b65fec-50f5-4ae7-865e-ac6dd64e9730 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:15.632193 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-d1b65fec-50f5-4ae7-865e-ac6dd64e9730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:15.737092 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 678/1342] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:42:15.778313 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcbcf0-d545-4828-b1b4-0d09b3e83fd5 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:15.781163 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-e3bcbcf0-d545-4828-b1b4-0d09b3e83fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:15.836876 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:15.888574 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcbcf0-d545-4828-b1b4-0d09b3e83fd5 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:15.892470 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4b1d9-c2b3-4fec-a09d-fefcb31aa005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b98b1e4024410a87addde112237f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:15.893148 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4b1d9-c2b3-4fec-a09d-fefcb31aa005 service neutron] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:15.893389 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4b1d9-c2b3-4fec-a09d-fefcb31aa005 service neutron] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:15.893676 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4b1d9-c2b3-4fec-a09d-fefcb31aa005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:15.904398 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4b1d9-c2b3-4fec-a09d-fefcb31aa005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:15.906346 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4b1d9-c2b3-4fec-a09d-fefcb31aa005 service neutron] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:15.916041 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 665/1343] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:42:18.788967 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99758ed8-327c-4993-9263-0dd0a24bd3e0 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:18.789541 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99758ed8-327c-4993-9263-0dd0a24bd3e0 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:18.789541 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99758ed8-327c-4993-9263-0dd0a24bd3e0 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:18.790935 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 679/1344] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:18.796856 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5382614-f67b-4cb6-ae76-7e3b5c05c0c1 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:18.797344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5382614-f67b-4cb6-ae76-7e3b5c05c0c1 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:18.797344 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5382614-f67b-4cb6-ae76-7e3b5c05c0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:18.826939 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-a5382614-f67b-4cb6-ae76-7e3b5c05c0c1 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:18.828116 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-a5382614-f67b-4cb6-ae76-7e3b5c05c0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:18.912097 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 666/1345] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:42:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:42:18.949361 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99758ed8-327c-4993-9263-0dd0a24bd3e0 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:18.952068 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-99758ed8-327c-4993-9263-0dd0a24bd3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:18.966328 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99758ed8-327c-4993-9263-0dd0a24bd3e0 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:18.969965 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42414b-ef68-449d-b298-6d725de97ad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b98b1e4024410a87addde112237f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:18.970647 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42414b-ef68-449d-b298-6d725de97ad2 service neutron] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:18.970923 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42414b-ef68-449d-b298-6d725de97ad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:18.971193 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42414b-ef68-449d-b298-6d725de97ad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:18.980845 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42414b-ef68-449d-b298-6d725de97ad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:18.982837 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42414b-ef68-449d-b298-6d725de97ad2 service neutron] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:18.992618 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 680/1346] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:42:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:42:23.382694 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383f302-2f77-40a5-b2b6-7ae620e1046f None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:23.383296 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383f302-2f77-40a5-b2b6-7ae620e1046f None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:23.383296 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383f302-2f77-40a5-b2b6-7ae620e1046f None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:23.384953 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 667/1347] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:42:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:23.391057 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb155a-26f1-4403-9108-efdf53c77954 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:23.391799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb155a-26f1-4403-9108-efdf53c77954 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:23.391799 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb155a-26f1-4403-9108-efdf53c77954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:23.410786 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-b2bb155a-26f1-4403-9108-efdf53c77954 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:23.411986 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-b2bb155a-26f1-4403-9108-efdf53c77954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:23.422671 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 681/1348] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:42:23.457935 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383f302-2f77-40a5-b2b6-7ae620e1046f None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:23.469800 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383f302-2f77-40a5-b2b6-7ae620e1046f None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:23.473777 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4f7b1-765f-4c01-8510-ad3581d89ae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b98b1e4024410a87addde112237f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:23.474038 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4f7b1-765f-4c01-8510-ad3581d89ae2 service neutron] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:23.474378 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4f7b1-765f-4c01-8510-ad3581d89ae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:23.474657 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4f7b1-765f-4c01-8510-ad3581d89ae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:23.485065 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4f7b1-765f-4c01-8510-ad3581d89ae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:23.486360 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4f7b1-765f-4c01-8510-ad3581d89ae2 service neutron] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:23.496078 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 668/1349] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:42:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:42:28.766433 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292e52d-87a2-4d5f-912b-db13f6a9bd60 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:28.767015 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292e52d-87a2-4d5f-912b-db13f6a9bd60 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:28.767015 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292e52d-87a2-4d5f-912b-db13f6a9bd60 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:28.768544 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 682/1350] 10.222.0.84 () {54 vars in 921 bytes} [Thu May 19 19:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:28.777466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2b6dc-a640-4dc0-9034-69631d32b285 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:28.777466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2b6dc-a640-4dc0-9034-69631d32b285 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:28.777466 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2b6dc-a640-4dc0-9034-69631d32b285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:28.871487 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-01d2b6dc-a640-4dc0-9034-69631d32b285 None None] MFA Rules not processed for user `673a9bbc7c194dafb25fb007aa66b30a`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:28.873436 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-01d2b6dc-a640-4dc0-9034-69631d32b285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:28.902925 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=100 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:28.957102 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 669/1351] 10.222.0.84 () {58 vars in 1018 bytes} [Thu May 19 19:42:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 19:42:33.405219 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=72 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:33.421901 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9f929-cdc5-4708-93d6-fb11a43c3d32 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:33.422577 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9f929-cdc5-4708-93d6-fb11a43c3d32 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:33.422945 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9f929-cdc5-4708-93d6-fb11a43c3d32 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:33.424505 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 683/1352] 10.222.0.84 () {54 vars in 921 bytes} [Thu May 19 19:42:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:33.430253 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed88a08-559a-455a-a948-1181978028ec None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:33.430808 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed88a08-559a-455a-a948-1181978028ec None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:33.430808 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed88a08-559a-455a-a948-1181978028ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:33.450705 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-fed88a08-559a-455a-a948-1181978028ec None None] MFA Rules not processed for user `673a9bbc7c194dafb25fb007aa66b30a`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:33.451889 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-fed88a08-559a-455a-a948-1181978028ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:33.461038 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 670/1353] 10.222.0.84 () {58 vars in 1018 bytes} [Thu May 19 19:42:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:42:34.897211 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e159527-27ee-4b61-a069-f031ad5190bb None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:34.897746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e159527-27ee-4b61-a069-f031ad5190bb None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:34.897746 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e159527-27ee-4b61-a069-f031ad5190bb None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:34.899078 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 684/1354] 10.222.0.84 () {54 vars in 921 bytes} [Thu May 19 19:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:34.904536 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d62f2-d2db-46d4-8424-f85878acbe3d None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:34.904940 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d62f2-d2db-46d4-8424-f85878acbe3d None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:34.905088 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d62f2-d2db-46d4-8424-f85878acbe3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:34.926426 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-9c6d62f2-d2db-46d4-8424-f85878acbe3d None None] MFA Rules not processed for user `673a9bbc7c194dafb25fb007aa66b30a`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:34.927626 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-9c6d62f2-d2db-46d4-8424-f85878acbe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:34.938220 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 671/1355] 10.222.0.84 () {58 vars in 1018 bytes} [Thu May 19 19:42:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:42:39.024512 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813db74-7b69-41c3-888a-ce1307711096 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:39.025160 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813db74-7b69-41c3-888a-ce1307711096 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:39.025160 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813db74-7b69-41c3-888a-ce1307711096 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:39.026735 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 685/1356] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:39.032446 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ea86b-66d9-4733-ae59-4d0830d1833d None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:39.032902 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ea86b-66d9-4733-ae59-4d0830d1833d None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:39.033013 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ea86b-66d9-4733-ae59-4d0830d1833d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:39.051735 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-a09ea86b-66d9-4733-ae59-4d0830d1833d None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:39.052915 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-a09ea86b-66d9-4733-ae59-4d0830d1833d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:39.061334 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 672/1357] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:42:39.114143 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f723e-be30-4ed1-85f2-7c4436a13d7b None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:39.114575 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f723e-be30-4ed1-85f2-7c4436a13d7b None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:39.114698 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f723e-be30-4ed1-85f2-7c4436a13d7b None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:39.116083 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 686/1358] 10.222.0.84 () {54 vars in 961 bytes} [Thu May 19 19:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:39.129187 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422fab0-aeb9-419e-926b-41b1ff4a4237 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:39.129830 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422fab0-aeb9-419e-926b-41b1ff4a4237 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:39.130371 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422fab0-aeb9-419e-926b-41b1ff4a4237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:39.186995 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-a422fab0-aeb9-419e-926b-41b1ff4a4237 None None] MFA Rules not processed for user `abde7cc02fae4d9bb561a6642c5252a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:39.188322 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-a422fab0-aeb9-419e-926b-41b1ff4a4237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:39.276928 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 673/1359] 10.222.0.84 () {58 vars in 1058 bytes} [Thu May 19 19:42:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:42:39.285146 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f723e-be30-4ed1-85f2-7c4436a13d7b None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:39.302104 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f723e-be30-4ed1-85f2-7c4436a13d7b None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:39.308682 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5d96d-7ef8-49a3-abc7-fe625e2d9586 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abde7cc02fae4d9bb561a6642c5252a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:39.309606 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5d96d-7ef8-49a3-abc7-fe625e2d9586 service nova] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:39.310043 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5d96d-7ef8-49a3-abc7-fe625e2d9586 service nova] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:39.310498 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5d96d-7ef8-49a3-abc7-fe625e2d9586 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:39.320270 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5d96d-7ef8-49a3-abc7-fe625e2d9586 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:39.322076 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5d96d-7ef8-49a3-abc7-fe625e2d9586 service nova] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:39.333385 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 687/1360] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:42:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:42:41.808279 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25eeaae-9c53-4620-8e21-ef4a8d038d08 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:41.808759 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25eeaae-9c53-4620-8e21-ef4a8d038d08 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:41.808869 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25eeaae-9c53-4620-8e21-ef4a8d038d08 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:41.810685 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 674/1361] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:42:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:41.816436 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60236627-b5b8-47e6-a2ff-e9a0b5acd87a None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:41.816841 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60236627-b5b8-47e6-a2ff-e9a0b5acd87a None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:41.817228 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60236627-b5b8-47e6-a2ff-e9a0b5acd87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:41.837162 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-60236627-b5b8-47e6-a2ff-e9a0b5acd87a None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:41.838423 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-60236627-b5b8-47e6-a2ff-e9a0b5acd87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:41.847390 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 688/1362] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:42:41.892658 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25eeaae-9c53-4620-8e21-ef4a8d038d08 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:41.907280 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25eeaae-9c53-4620-8e21-ef4a8d038d08 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:41.912652 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a0fe2-ea22-409a-aa6d-8fb9d02368c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abde7cc02fae4d9bb561a6642c5252a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:41.913324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a0fe2-ea22-409a-aa6d-8fb9d02368c1 service nova] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:41.913604 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a0fe2-ea22-409a-aa6d-8fb9d02368c1 service nova] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:41.913879 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a0fe2-ea22-409a-aa6d-8fb9d02368c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:41.922841 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a0fe2-ea22-409a-aa6d-8fb9d02368c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:41.924457 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a0fe2-ea22-409a-aa6d-8fb9d02368c1 service nova] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:41.934598 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 675/1363] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:42:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:42:44.499284 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a059e9-8355-45b3-a1aa-6af304d587f9 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.499866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a059e9-8355-45b3-a1aa-6af304d587f9 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.499866 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a059e9-8355-45b3-a1aa-6af304d587f9 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.501207 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 689/1364] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:44.506880 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb40e6-ea61-43c8-b4bf-00967f00911b None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.507303 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb40e6-ea61-43c8-b4bf-00967f00911b None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.507486 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb40e6-ea61-43c8-b4bf-00967f00911b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.528837 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-e1bb40e6-ea61-43c8-b4bf-00967f00911b None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:44.530290 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-e1bb40e6-ea61-43c8-b4bf-00967f00911b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:44.546288 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 676/1365] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:42:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:42:44.591891 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f02074-643d-4bd4-8a2a-a4917812c865 None None] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.592197 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f02074-643d-4bd4-8a2a-a4917812c865 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.592438 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f02074-643d-4bd4-8a2a-a4917812c865 None None] PATH_INFO: `/` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.594173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 690/1366] 10.222.0.84 () {54 vars in 961 bytes} [Thu May 19 19:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:44.599714 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900d48e-1733-48c0-9989-5f3b9d8fd076 None None] REQUEST_METHOD: `POST` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.600388 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900d48e-1733-48c0-9989-5f3b9d8fd076 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.600845 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900d48e-1733-48c0-9989-5f3b9d8fd076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.626280 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.auth.core [None req-0900d48e-1733-48c0-9989-5f3b9d8fd076 None None] MFA Rules not processed for user `abde7cc02fae4d9bb561a6642c5252a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86982) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:44.629072 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.fernet_utils [None req-0900d48e-1733-48c0-9989-5f3b9d8fd076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86982) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:44.646720 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 677/1367] 10.222.0.84 () {58 vars in 1058 bytes} [Thu May 19 19:42:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 19:42:44.654826 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f02074-643d-4bd4-8a2a-a4917812c865 None None] Authenticating user token {{(pid=86981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:44.670535 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f02074-643d-4bd4-8a2a-a4917812c865 None None] Validating token access rules against request {{(pid=86981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:44.676119 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c73b9-a7d1-4be6-9e5e-365f0312979e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abde7cc02fae4d9bb561a6642c5252a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:44.676770 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c73b9-a7d1-4be6-9e5e-365f0312979e service nova] REQUEST_METHOD: `GET` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.677158 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c73b9-a7d1-4be6-9e5e-365f0312979e service nova] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.677428 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c73b9-a7d1-4be6-9e5e-365f0312979e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.686393 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c73b9-a7d1-4be6-9e5e-365f0312979e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:44.688022 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c73b9-a7d1-4be6-9e5e-365f0312979e service nova] RBAC: Authorization granted {{(pid=86981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:44.697967 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 691/1368] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:42:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:42:45.374324 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128db9df-c286-491a-826b-78e07c114fe9 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:45.374579 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128db9df-c286-491a-826b-78e07c114fe9 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:45.374894 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128db9df-c286-491a-826b-78e07c114fe9 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:45.376462 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 678/1369] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:45.381715 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d59f52-e4a8-4d8f-a3c2-840a53115752 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:45.381967 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d59f52-e4a8-4d8f-a3c2-840a53115752 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:45.382510 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d59f52-e4a8-4d8f-a3c2-840a53115752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:45.401337 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-e9d59f52-e4a8-4d8f-a3c2-840a53115752 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:45.404329 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-e9d59f52-e4a8-4d8f-a3c2-840a53115752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:45.413583 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 692/1370] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 19:42:47.071559 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ed0c6-cde5-4d8b-a947-173b4ae73cb9 None None] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:47.072135 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ed0c6-cde5-4d8b-a947-173b4ae73cb9 None None] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:47.072135 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ed0c6-cde5-4d8b-a947-173b4ae73cb9 None None] PATH_INFO: `/` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:47.073529 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 679/1371] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:47.079426 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eb5c7-bdfc-4f99-98a5-86f113eaaa67 None None] REQUEST_METHOD: `POST` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:47.079842 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eb5c7-bdfc-4f99-98a5-86f113eaaa67 None None] SCRIPT_NAME: `/identity` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:47.079960 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eb5c7-bdfc-4f99-98a5-86f113eaaa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:47.099173 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.auth.core [None req-e37eb5c7-bdfc-4f99-98a5-86f113eaaa67 None None] MFA Rules not processed for user `a5800b131869435eb3ae07f86d69929e`. Rule list: `[]` (Enabled: `True`). {{(pid=86981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:47.100689 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG keystone.common.fernet_utils [None req-e37eb5c7-bdfc-4f99-98a5-86f113eaaa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:47.110016 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: [pid: 86981|app: 0|req: 693/1372] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:42:47.154403 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ed0c6-cde5-4d8b-a947-173b4ae73cb9 None None] Authenticating user token {{(pid=86982) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:47.169617 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ed0c6-cde5-4d8b-a947-173b4ae73cb9 None None] Validating token access rules against request {{(pid=86982) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:47.175535 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6144b774-109a-46a3-bd0e-5f30364fbbc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abde7cc02fae4d9bb561a6642c5252a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecf87429758641eeb462b3e409cadc9e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86982) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:47.176175 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144b774-109a-46a3-bd0e-5f30364fbbc8 service nova] REQUEST_METHOD: `GET` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:47.176419 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144b774-109a-46a3-bd0e-5f30364fbbc8 service nova] SCRIPT_NAME: `/identity` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:47.176719 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144b774-109a-46a3-bd0e-5f30364fbbc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86982) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:47.185472 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6144b774-109a-46a3-bd0e-5f30364fbbc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:47.187082 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6144b774-109a-46a3-bd0e-5f30364fbbc8 service nova] RBAC: Authorization granted {{(pid=86982) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:47.196447 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: [pid: 86982|app: 0|req: 680/1373] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:42:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:42:57.093854 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: DEBUG dbcounter [-] [86981] Writing DB stats keystone:SELECT=21 {{(pid=86981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:57.192311 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: DEBUG dbcounter [-] [86982] Writing DB stats keystone:SELECT=95 {{(pid=86982) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:09.028372 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: Thu May 19 19:43:09 2022 - graceful shutdown triggered... May 19 19:43:09.028498 dv-cinder-835881-13-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 19:43:09.029106 dv-cinder-835881-13-iscsi devstack@keystone.service[86981]: Gracefully killing worker 1 (pid: 86981)... May 19 19:43:09.029496 dv-cinder-835881-13-iscsi devstack@keystone.service[86982]: Gracefully killing worker 2 (pid: 86982)... May 19 19:43:10.030460 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: worker 1 buried after 1 seconds May 19 19:43:10.030460 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: worker 2 buried after 1 seconds May 19 19:43:10.030460 dv-cinder-835881-13-iscsi devstack@keystone.service[86977]: goodbye to uWSGI. May 19 19:43:10.032761 dv-cinder-835881-13-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 19:43:10.033633 dv-cinder-835881-13-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 19 19:49:10.802583 dv-cinder-835881-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 19:49:10.809873 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 19:49:10.811885 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 19:49:10 2022] *** May 19 19:49:10.811885 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 19:49:10.811885 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 19:49:10.811885 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: nodename: dv-cinder-835881-13-iscsi May 19 19:49:10.811885 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: machine: x86_64 May 19 19:49:10.811885 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: clock source: unix May 19 19:49:10.811885 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: pcre jit disabled May 19 19:49:10.812421 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: detected number of CPU cores: 4 May 19 19:49:10.812421 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: current working directory: / May 19 19:49:10.812421 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: detected binary path: /usr/bin/uwsgi-core May 19 19:49:10.812421 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: your processes number limit is 47680 May 19 19:49:10.812421 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: your memory page size is 4096 bytes May 19 19:49:10.812421 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: detected max file descriptor number: 2048 May 19 19:49:10.812421 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: lock engine: pthread robust mutexes May 19 19:49:10.812421 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: thunder lock: enabled May 19 19:49:10.812421 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 19:49:10.812421 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 19:49:10.858721 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: Python main interpreter initialized at 0x5587b542f9c0 May 19 19:49:10.858721 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: python threads support enabled May 19 19:49:10.858721 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: your server socket listen backlog is limited to 100 connections May 19 19:49:10.858721 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: your mercy for graceful operations on workers is 90 seconds May 19 19:49:10.859167 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: mapped 403077 bytes (393 KB) for 2 cores May 19 19:49:10.859167 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: *** Operational MODE: preforking *** May 19 19:49:10.859450 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: *** uWSGI is running in multiple interpreter mode *** May 19 19:49:10.859450 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: spawned uWSGI master process (pid: 150536) May 19 19:49:10.859706 dv-cinder-835881-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 19:49:10.865466 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: spawned uWSGI worker 1 (pid: 150548, cores: 1) May 19 19:49:10.865466 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: spawned uWSGI worker 2 (pid: 150549, cores: 1) May 19 19:49:10.865466 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 19:49:16.479568 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:49:16.479568 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: from cryptography.utils import int_from_bytes May 19 19:49:16.480514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:49:16.480514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: from cryptography.utils import int_from_bytes May 19 19:49:16.529692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 19:49:16.529893 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 19:49:16.530061 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] command line args: [] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 19:49:16.530364 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 19:49:16.530517 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ================================================================================ {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 19:49:16.530770 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] admin_token = **** {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.531013 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] config_dir = [] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.531161 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.531345 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] config_source = [] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.531500 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] debug = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.531729 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.532073 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.532290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] insecure_debug = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.532516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.532784 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.532989 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] list_limit = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.533196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] log_config_append = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.533346 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.533455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:49:16.533455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: from cryptography.utils import int_from_bytes May 19 19:49:16.534136 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] log_dir = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.534234 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:49:16.534234 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: from cryptography.utils import int_from_bytes May 19 19:49:16.534402 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] log_file = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.534402 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.534402 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.534627 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.534703 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.534883 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.535047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.535276 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.535520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.535737 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.535971 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.536200 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.536432 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.536676 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.536919 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] notification_format = cadf {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.537193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.537685 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.84/identity {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.537866 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] publish_errors = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.538103 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.538353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.538535 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.538762 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.538994 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.539228 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] standard_threads = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.539458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] strict_password_check = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.539700 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.539876 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] use_eventlog = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.540052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] use_journal = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.540228 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] use_json = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.540401 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] use_stderr = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.540576 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] use_syslog = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.540804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] watch_log_file = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.541061 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.541298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.541556 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.541794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.542042 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.542359 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.542616 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.542854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.543104 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.543353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.543604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.543838 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.544085 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.544319 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.544566 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.544814 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.545051 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.545305 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.545557 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.545802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.546037 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.546315 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.546580 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.546834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.547084 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.547314 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.547545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.547773 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.548011 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.548259 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.548508 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.548737 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.548966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.549213 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.549550 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.549866 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.550122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.550455 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.550697 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.550936 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.551170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.551401 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.551633 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.551863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.552100 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.552352 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.552584 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.552823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.553053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.553290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.553530 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.553779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.554010 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.554248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.554508 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.554747 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.554984 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.555222 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.555454 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.555707 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.555942 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.556180 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.556413 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.556670 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.556907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.557137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.557373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.557609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.557884 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.558121 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.558392 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.558641 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.558879 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.559115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.559351 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.559599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.enabled = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.559839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.560075 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.560315 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.560554 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.560790 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.561025 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.561262 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.561498 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.561761 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.562000 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.562241 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.562500 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.562740 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.562978 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.563214 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.563448 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.563680 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.563909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.564138 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.564372 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.564601 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.564839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.565130 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.565380 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.565619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.565987 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.566250 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.566513 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.566746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] auth.external = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.566978 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] auth.mapped = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.567274 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.567509 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.567740 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] auth.password = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.567973 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] auth.token = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.568210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.568446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] catalog.caching = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.568691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.568923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.569173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.569419 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.569650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.569886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] credential.caching = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.570133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] credential.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.570404 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.570653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.570891 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.571134 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.571369 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.571636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.571890 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.572127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.572378 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.572663 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.572908 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.573169 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.573410 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.573666 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.573900 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] federation.caching = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.574137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.574414 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] federation.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.574663 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.574895 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.575142 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.575378 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.575630 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.575868 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.576122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.576358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.576606 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.576841 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity.caching = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.577087 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.577334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.577570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.577810 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.578057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.578313 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.578555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.578803 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.579055 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.579287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.579519 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.579749 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.579992 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.580238 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.580484 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.580574 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 19:49:16.580701 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 19:49:16.580788 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.581018 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.581122 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] command line args: [] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 19:49:16.581296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.581390 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 19:49:16.581485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ================================================================================ {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 19:49:16.581570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.581720 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] admin_token = **** {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.581824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.581964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] config_dir = [] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.582072 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.582172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.582274 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.582378 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] config_source = [] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.582463 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.582546 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] debug = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.582661 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.582770 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.582896 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.582982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.583090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] insecure_debug = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.583195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.583340 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.583433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.583580 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.583677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.583805 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] list_limit = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.583904 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.583989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] log_config_append = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.584080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.584176 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.584273 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.584357 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] log_dir = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.584468 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] log_file = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.584564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.584686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.584776 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.584931 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.585023 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.585174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.585272 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.585405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.password = **** {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.585498 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.585597 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.585682 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.585775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.585875 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.585970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.586094 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.586185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.586351 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.586454 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.586590 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.586681 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.586841 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.586943 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.587086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.587186 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.587316 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.587408 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.587546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.587652 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] notification_format = cadf {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.587799 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.587909 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.588044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.588278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.588370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.84/identity {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.588456 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] publish_errors = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.588547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.588676 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.588780 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.588902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.588993 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.589094 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.589180 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.589305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.589417 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.589536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.589633 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.589766 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] standard_threads = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.589882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.589985 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] strict_password_check = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.590128 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.590229 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.590400 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.590502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] use_eventlog = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.590588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] use_journal = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.590669 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.590752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] use_json = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.590842 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.590927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] use_stderr = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.591035 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] use_syslog = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.591119 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.591266 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] watch_log_file = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:49:16.591375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.591509 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.591599 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.591739 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.591842 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.591984 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.592076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.592219 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.592318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.592461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.592562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.592693 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.592784 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.592935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.593033 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.593163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.593268 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.593406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.593503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.593639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.593735 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.593875 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.593981 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.594104 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.594209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.594357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.594448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.594595 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.594701 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.594842 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.595088 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.595196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.595321 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.595424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.595572 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.595671 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] policy.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.595821 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.595919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.596061 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.596158 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.596334 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.596431 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.596583 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.596665 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] receipt.caching = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.596819 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.596897 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.597051 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.597139 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.597295 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.597386 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.597525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.597617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.597750 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.597851 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.597975 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.598412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] resource.caching = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.598543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.598635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.598787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.598879 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] resource.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.599030 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.599121 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.599254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.599350 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.599479 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.599570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.599722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.599813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] revoke.caching = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.599965 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.600069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.600198 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.600290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.600433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] role.cache_time = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.600532 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.600658 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.600758 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] role.caching = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.600890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.600981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] role.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.601118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] role.list_limit = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.601207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.601341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.601446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.601567 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.601659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.601793 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.601892 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.602018 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.602118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.602250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.602395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.602517 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.602610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.602745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.602978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.603164 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.603257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.603362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.603448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.603530 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.603627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.603758 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.603874 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.604011 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.604119 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.604260 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.604360 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.604505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.604597 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.604761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.604915 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.605005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.605152 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.605250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.605402 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.605503 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.605651 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.605745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.605880 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.605971 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.606112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.606202 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.606364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.606471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.606632 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.606726 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.606866 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.606969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.607093 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.607185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.607325 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.607415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.607557 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.607672 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.607815 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.607906 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.608047 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.608148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.608294 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.608394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.608535 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.608626 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.608768 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.608858 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.609000 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.609110 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.609230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] token.cache_time = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.609332 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.609467 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] token.caching = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.609566 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.enabled = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.609700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.609803 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.609932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.610030 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] token.provider = fernet {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.610166 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.610255 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.610428 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.610520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.610662 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.610754 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.610893 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.610994 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.611124 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.611216 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.611353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.611457 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.611600 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] trust.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.611690 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.611834 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.611942 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.612123 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.612222 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.612324 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.612413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.612602 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.612701 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.612836 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.612941 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.613068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.613159 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.613302 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.613414 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.613531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.613675 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.613774 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.613937 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.connection = **** {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.614038 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.614182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.614294 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.614431 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.614544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.614666 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.614771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.614898 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.615003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.615135 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.615240 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.615366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.615457 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.615600 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.615717 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.615824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.615978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.616072 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.616217 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.616321 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.616448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] auth.external = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.616538 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.616677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] auth.mapped = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.616769 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.616971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.617067 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.617201 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.617300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.617431 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] auth.password = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.617523 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.617659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] auth.token = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.617750 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.617895 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.618004 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.618127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.618221 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] catalog.caching = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.618336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=150549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 19:49:16.618423 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.618612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.618859 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.619104 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.619333 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.619576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] credential.caching = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.619822 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] credential.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.620068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.620321 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.620556 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.620797 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.621028 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.621287 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.621537 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.621769 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.622022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.622321 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.622564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.622823 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.623061 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.623313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.623545 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] federation.caching = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.623778 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.624022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] federation.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.624265 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.624491 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.624737 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.624971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.625222 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.625457 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.625708 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.625941 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.626183 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.626442 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity.caching = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.626689 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.626932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.627164 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.627395 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.627748 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.627972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.628235 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.628479 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.628690 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.628917 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.629144 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.629370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.629643 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.629853 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.630114 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.630375 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.630621 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.630889 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.631126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.631361 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.631589 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.631823 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.632050 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.632295 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.632532 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.632769 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.633017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.633246 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.633491 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.633734 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.633976 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.634218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.634484 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.634709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.634944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.635171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.password = **** {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.635406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.635640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.635881 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.636116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.636349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.636593 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.636836 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.637063 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.637287 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.637532 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.637775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.638008 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.638239 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.638495 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.638722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.638958 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.639214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.639442 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.639686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.639928 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.640176 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.640407 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.640631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.640862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.641094 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.641325 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.641550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.641792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.642035 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.642297 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.642542 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.642785 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.643011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.643304 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.643501 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.643733 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.643965 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.644195 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.644437 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.644680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.644912 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.645160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] policy.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.645384 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.645624 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.645855 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.646096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] receipt.caching = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.646354 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.646602 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.646844 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.647072 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.647307 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.647546 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] resource.caching = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.647794 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.648037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] resource.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.648270 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.648518 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.648765 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.648998 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] revoke.caching = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.649233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.649332 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.649468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.649574 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.649698 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.649800 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] role.cache_time = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.649929 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] role.caching = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.650176 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] role.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.650456 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] role.list_limit = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.650675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.650921 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.651166 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.651410 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.651653 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.651898 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.652142 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.652384 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.652612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.652857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.653112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.653358 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.653602 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.654008 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.654240 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.654510 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.654758 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.655004 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.655249 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.655480 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.655747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.655995 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.656230 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.656457 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.656684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.656913 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.657159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.657411 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.657655 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.657902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.658135 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.658386 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] token.cache_time = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.658480 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.658622 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] token.caching = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.658916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.659162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] token.provider = fernet {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.659396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.659648 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.659891 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.660132 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.660373 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.660614 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.660859 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] trust.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.661093 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.661328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.661558 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.661801 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.662043 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.662300 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.662531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.662829 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.663089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.connection = **** {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.663331 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.663575 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.663820 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.664059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.664294 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.664528 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.664761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.664994 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.665238 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.665472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.665711 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.665944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.666189 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.666449 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.666692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.666919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.667157 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.667389 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:49:16.667556 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=150548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 19:49:16.686965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.687199 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.692824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.693057 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.693282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.703244 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.721675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.722041 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.740149 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.759443 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.790507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.790719 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.795978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.796213 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.855587 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.855767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.855846 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.856019 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.856115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.856214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.856301 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.856381 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.856466 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.856610 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.856774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.856963 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.879696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.879987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.880167 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.880260 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.880373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.880451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.880535 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.880622 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.880709 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.880808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.880903 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.880978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.881070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.881235 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.892091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.892401 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.904500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.904711 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.904968 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.905187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.905375 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.905594 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.905830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=150548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.907074 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.907291 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.907531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.907755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.907980 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.908175 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.908375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=150549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:49:16.922872 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.923053 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:49:16.923416 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.923567 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.923681 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:49:16.923773 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:49:16.923862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.924082 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.924200 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:49:16.924409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.924692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.924801 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.924894 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.924983 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.925413 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.936552 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.936828 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.937101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.937374 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.937515 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.937611 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.937894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.938167 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.945691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:16.946453 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:16.949027 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:16.949309 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.949695 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.949832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:16.949960 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.950065 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.950290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.950443 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.950557 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.950646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.950788 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.950915 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.951037 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.951189 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.951332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.951486 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.951628 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.951739 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.951831 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.951989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.952257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.952524 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.979273 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.979452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.980880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:16.981041 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:16.981133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.981265 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.986459 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:16.986629 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:16.986725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.986930 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.987170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.987338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.994645 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:16.994831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.994941 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:16.995140 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.999477 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:16.999757 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:16.999919 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.000011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.000100 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.000306 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.000464 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.000561 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.000665 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.000838 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.000998 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.001112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.001209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.001333 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.001439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.001625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.020216 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.020493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.021291 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.021569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.025022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.026246 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.028660 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.028959 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.029932 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.030227 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.033941 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.035298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.037711 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.038952 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.041460 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.041748 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.042026 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.042303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.042599 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.042877 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.043347 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.043644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.043925 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.044213 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.044493 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.044756 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.061877 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.062254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.062547 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.062828 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.063903 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.064279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.064549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.064816 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.070833 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.072536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.072808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.072966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.074606 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.074880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.075863 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.076135 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.076422 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.076708 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.076985 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.077941 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.078220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.078537 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.078819 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.079102 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.090443 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.090726 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.091014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.091326 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.091610 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.091881 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.092151 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.092739 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.092992 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.093435 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.093761 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.094047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.094368 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.094658 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.111509 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.113874 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.115520 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.117191 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.119676 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.120553 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.121271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.122161 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.122943 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.123223 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.123497 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.123839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.124127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.124404 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.130977 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.131980 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.132624 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.133612 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.135866 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.136159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.136439 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.136709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.136866 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.137162 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.137447 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.137726 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.145976 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.146289 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.146563 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.147290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.147588 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.147866 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.154755 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=150548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.155031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.155302 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.155571 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.155806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=150549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:49:17.156046 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.156146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.156225 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.156311 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.156413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.156619 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.156758 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.156842 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.156942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.157137 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.157302 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.157420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.157499 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.157654 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.157802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.157885 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.157999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.158157 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=150548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.158253 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.158468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.158750 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.159022 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.159299 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=150549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:49:17.197058 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5587b542f9c0 pid: 150548 (default app) May 19 19:49:17.198291 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5587b542f9c0 pid: 150549 (default app) May 19 19:49:17.200553 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a735-4913-48f5-8dc9-28afc699ee52 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:17.200861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a735-4913-48f5-8dc9-28afc699ee52 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:17.201141 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a735-4913-48f5-8dc9-28afc699ee52 None None] PATH_INFO: `/v3` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:17.205356 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 1/1] 10.222.0.84 () {50 vars in 791 bytes} [Thu May 19 19:49:17 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:25.467880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa23ff-87d7-461f-b72c-3735b6d59a49 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:25.468884 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa23ff-87d7-461f-b72c-3735b6d59a49 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:25.468884 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa23ff-87d7-461f-b72c-3735b6d59a49 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:25.476267 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 1/2] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:25 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:25.484947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612542b-a254-4055-8621-7392c060934b None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:25.485697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612542b-a254-4055-8621-7392c060934b None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:25.485851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612542b-a254-4055-8621-7392c060934b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:25.555042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: INFO dbcounter [None req-7612542b-a254-4055-8621-7392c060934b None None] Registered counter for database keystone May 19 19:49:25.580737 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG oslo_db.sqlalchemy.engines [None req-7612542b-a254-4055-8621-7392c060934b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=150548) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 19:49:25.593013 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writer thread running {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 19:49:25.778770 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG passlib.handlers.bcrypt [None req-7612542b-a254-4055-8621-7392c060934b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=150548) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 19:49:25.779111 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG passlib.handlers.bcrypt [None req-7612542b-a254-4055-8621-7392c060934b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=150548) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 19:49:25.837352 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-7612542b-a254-4055-8621-7392c060934b None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:25.839020 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-7612542b-a254-4055-8621-7392c060934b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:25.977284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 2/3] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 19:49:26.001297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa23ff-87d7-461f-b72c-3735b6d59a49 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:26.073938 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: INFO dbcounter [None req-fbfa23ff-87d7-461f-b72c-3735b6d59a49 None None] Registered counter for database keystone May 19 19:49:26.096440 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG oslo_db.sqlalchemy.engines [None req-fbfa23ff-87d7-461f-b72c-3735b6d59a49 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=150549) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 19:49:26.104995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writer thread running {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 19:49:26.228569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa23ff-87d7-461f-b72c-3735b6d59a49 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:26.234148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4d912-3ba1-4ad6-a96d-c252805fe8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '682345a068d445309abcfb72f27e61bb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:26.234846 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4d912-3ba1-4ad6-a96d-c252805fe8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:26.235134 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4d912-3ba1-4ad6-a96d-c252805fe8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:26.235409 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4d912-3ba1-4ad6-a96d-c252805fe8a4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:26.244744 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4d912-3ba1-4ad6-a96d-c252805fe8a4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:26.298413 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0de4d912-3ba1-4ad6-a96d-c252805fe8a4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 19:49:26.298770 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4d912-3ba1-4ad6-a96d-c252805fe8a4 admin admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:26.311774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 2/4] 10.222.0.84 () {58 vars in 1324 bytes} [Thu May 19 19:49:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 19:49:26.319535 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612542b-a254-4055-8621-7392c060934b None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:26.335282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612542b-a254-4055-8621-7392c060934b None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:26.341079 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52af8c-ccc2-4b2e-88d6-ebcba33e12b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '682345a068d445309abcfb72f27e61bb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:26.342247 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52af8c-ccc2-4b2e-88d6-ebcba33e12b1 admin admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:26.342565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52af8c-ccc2-4b2e-88d6-ebcba33e12b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:26.342847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52af8c-ccc2-4b2e-88d6-ebcba33e12b1 admin admin] PATH_INFO: `/v3/projects/682345a068d445309abcfb72f27e61bb` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:26.344541 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52af8c-ccc2-4b2e-88d6-ebcba33e12b1 admin admin] RBAC: Authorizing `identity:get_project(project_id=682345a068d445309abcfb72f27e61bb)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:26.397062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ce52af8c-ccc2-4b2e-88d6-ebcba33e12b1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 19:49:26.397400 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52af8c-ccc2-4b2e-88d6-ebcba33e12b1 admin admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:26.402062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 3/5] 10.222.0.84 () {56 vars in 1207 bytes} [Thu May 19 19:49:26 2022] GET /identity/v3/projects/682345a068d445309abcfb72f27e61bb => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:28.637734 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85574a-3274-4d12-aeb6-9044a2a1d462 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:28.638328 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85574a-3274-4d12-aeb6-9044a2a1d462 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:28.638328 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85574a-3274-4d12-aeb6-9044a2a1d462 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:28.642483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 3/6] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:49:28.649485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7ff39-05ec-4bfd-a5e3-ffc5a0055f56 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:28.650287 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7ff39-05ec-4bfd-a5e3-ffc5a0055f56 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:28.650454 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7ff39-05ec-4bfd-a5e3-ffc5a0055f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:28.673489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-f6a7ff39-05ec-4bfd-a5e3-ffc5a0055f56 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:28.675237 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-f6a7ff39-05ec-4bfd-a5e3-ffc5a0055f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:28.684226 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 4/7] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 19:49:28.707989 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85574a-3274-4d12-aeb6-9044a2a1d462 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:28.722754 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85574a-3274-4d12-aeb6-9044a2a1d462 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:28.728625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11b216-63f5-4dfd-8ae0-d4dd2ce18fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '682345a068d445309abcfb72f27e61bb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:28.729276 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11b216-63f5-4dfd-8ae0-d4dd2ce18fed admin admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:28.729566 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11b216-63f5-4dfd-8ae0-d4dd2ce18fed admin admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:28.729848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11b216-63f5-4dfd-8ae0-d4dd2ce18fed admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:28.740017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11b216-63f5-4dfd-8ae0-d4dd2ce18fed admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:28.741834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11b216-63f5-4dfd-8ae0-d4dd2ce18fed admin admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:28.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 4/8] 10.222.0.84 () {58 vars in 1324 bytes} [Thu May 19 19:49:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 19:49:28.758599 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7ff39-05ec-4bfd-a5e3-ffc5a0055f56 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:28.774001 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7ff39-05ec-4bfd-a5e3-ffc5a0055f56 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:28.781376 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefc7ad-649b-4950-93ee-36575e113c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '682345a068d445309abcfb72f27e61bb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:28.782307 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefc7ad-649b-4950-93ee-36575e113c47 admin admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:28.782585 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefc7ad-649b-4950-93ee-36575e113c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:28.782858 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefc7ad-649b-4950-93ee-36575e113c47 admin admin] PATH_INFO: `/v3/users/15d8f48fa0c344f1aed458217ad0c20b` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:28.784511 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefc7ad-649b-4950-93ee-36575e113c47 admin admin] RBAC: Authorizing `identity:get_user(user_id=15d8f48fa0c344f1aed458217ad0c20b)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:28.786391 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefc7ad-649b-4950-93ee-36575e113c47 admin admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:28.787993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 5/9] 10.222.0.84 () {56 vars in 1198 bytes} [Thu May 19 19:49:28 2022] GET /identity/v3/users/15d8f48fa0c344f1aed458217ad0c20b => generated 247 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:31.044496 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3e3d1-a252-4d02-92f2-89c6a1bf6a98 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:31.045081 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3e3d1-a252-4d02-92f2-89c6a1bf6a98 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:31.045081 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3e3d1-a252-4d02-92f2-89c6a1bf6a98 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:31.046422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 5/10] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:31.051793 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935d571-5a69-4870-9e6d-7b622131dfe5 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:31.052242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935d571-5a69-4870-9e6d-7b622131dfe5 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:31.052344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935d571-5a69-4870-9e6d-7b622131dfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:31.070241 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-a935d571-5a69-4870-9e6d-7b622131dfe5 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:31.071476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-a935d571-5a69-4870-9e6d-7b622131dfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:31.156568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 6/11] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:31.166939 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561a8d3-8003-4b25-987c-781f113b25c3 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:31.167233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561a8d3-8003-4b25-987c-781f113b25c3 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:31.167521 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561a8d3-8003-4b25-987c-781f113b25c3 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:31.168883 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 6/12] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:31.174284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab494e-9242-4a24-9932-7ce946680f19 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:31.174585 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab494e-9242-4a24-9932-7ce946680f19 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:31.175017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab494e-9242-4a24-9932-7ce946680f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:31.180000 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561a8d3-8003-4b25-987c-781f113b25c3 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:31.193106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-e2ab494e-9242-4a24-9932-7ce946680f19 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:31.194362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-e2ab494e-9242-4a24-9932-7ce946680f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:31.212512 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:31.279171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 7/13] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:31.301714 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab494e-9242-4a24-9932-7ce946680f19 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:31.327477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561a8d3-8003-4b25-987c-781f113b25c3 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:31.362524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f3de6-5a6c-44e1-b433-5d1331cdf012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:31.363330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f3de6-5a6c-44e1-b433-5d1331cdf012 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:31.363616 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f3de6-5a6c-44e1-b433-5d1331cdf012 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:31.363890 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f3de6-5a6c-44e1-b433-5d1331cdf012 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:31.375066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f3de6-5a6c-44e1-b433-5d1331cdf012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-5a7f3de6-5a6c-44e1-b433-5d1331cdf012 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:31.386229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:31.387373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:31.388339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:31.388339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:31.388339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:31.388339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:31.388339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:31.388339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:31.388339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:31.388339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:31.388339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:31.388339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:31.388339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:49:31.388339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:49:31.388339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 7/14] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:49:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:49:31.393962 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f3de6-5a6c-44e1-b433-5d1331cdf012 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:31.407433 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab494e-9242-4a24-9932-7ce946680f19 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:31.431232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:31.443204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a03f2e-0a14-4f82-9f46-371b6d3da1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:31.443876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a03f2e-0a14-4f82-9f46-371b6d3da1d4 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:31.444160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a03f2e-0a14-4f82-9f46-371b6d3da1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:31.444438 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a03f2e-0a14-4f82-9f46-371b6d3da1d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:31.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:31.619658 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f3de6-5a6c-44e1-b433-5d1331cdf012 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:31.630667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a03f2e-0a14-4f82-9f46-371b6d3da1d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:31.632172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a03f2e-0a14-4f82-9f46-371b6d3da1d4 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:31.654110 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fa110-a1f6-4810-ae7e-a40d9c146216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:31.654821 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fa110-a1f6-4810-ae7e-a40d9c146216 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:31.655112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fa110-a1f6-4810-ae7e-a40d9c146216 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:31.655405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fa110-a1f6-4810-ae7e-a40d9c146216 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:31.656175 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925fa110-a1f6-4810-ae7e-a40d9c146216 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:31.656523 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925fa110-a1f6-4810-ae7e-a40d9c146216 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:31.657924 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925fa110-a1f6-4810-ae7e-a40d9c146216 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:31.670770 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 8/15] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:49:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:49:31.676490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fa110-a1f6-4810-ae7e-a40d9c146216 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:31.736539 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 8/16] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:49:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:49:31.742090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a03f2e-0a14-4f82-9f46-371b6d3da1d4 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:31.783181 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:31.789622 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fa110-a1f6-4810-ae7e-a40d9c146216 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:31.794491 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:31.824804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd45ee-a744-437f-b58c-10217efdda2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:31.825694 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd45ee-a744-437f-b58c-10217efdda2a None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:31.825976 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd45ee-a744-437f-b58c-10217efdda2a None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:31.826256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd45ee-a744-437f-b58c-10217efdda2a None admin] PATH_INFO: `/v3/users/15d8f48fa0c344f1aed458217ad0c20b` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:31.827952 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd45ee-a744-437f-b58c-10217efdda2a None admin] RBAC: Authorizing `identity:get_user(user_id=15d8f48fa0c344f1aed458217ad0c20b)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:31.829407 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd45ee-a744-437f-b58c-10217efdda2a None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:31.831099 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 9/17] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:31 2022] GET /identity/v3/users/15d8f48fa0c344f1aed458217ad0c20b => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:49:31.836612 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd45ee-a744-437f-b58c-10217efdda2a None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:31.840883 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a03f2e-0a14-4f82-9f46-371b6d3da1d4 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:31.875022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bfddf-9670-4df0-9362-136576927f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:31.875820 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bfddf-9670-4df0-9362-136576927f60 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:31.876099 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bfddf-9670-4df0-9362-136576927f60 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:31.876372 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bfddf-9670-4df0-9362-136576927f60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:31.886691 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8bfddf-9670-4df0-9362-136576927f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:31.888169 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8bfddf-9670-4df0-9362-136576927f60 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:31.908565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-9c8bfddf-9670-4df0-9362-136576927f60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:49:31.908565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:31.908565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:31.908565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:31.908565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:31.908565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:31.908565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:31.908565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:31.908565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:31.908565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:31.908565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:31.908565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:31.908565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:49:31.908565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:49:31.908565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:31.909580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:31.910564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:49:31.910564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:49:31.910564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:49:31.910564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:49:31.910564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 9/18] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:49:31.915634 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bfddf-9670-4df0-9362-136576927f60 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:31.936277 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd45ee-a744-437f-b58c-10217efdda2a None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:31.971816 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8519bc-672e-4d6d-a1bf-fb659f74acc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:31.972628 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8519bc-672e-4d6d-a1bf-fb659f74acc0 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:31.973204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8519bc-672e-4d6d-a1bf-fb659f74acc0 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:31.973499 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8519bc-672e-4d6d-a1bf-fb659f74acc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:31.974836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8519bc-672e-4d6d-a1bf-fb659f74acc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:31.976280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8519bc-672e-4d6d-a1bf-fb659f74acc0 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:31.978172 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 10/19] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:49:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:49:31.983844 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8519bc-672e-4d6d-a1bf-fb659f74acc0 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:32.014539 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:32.018063 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bfddf-9670-4df0-9362-136576927f60 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:32.020407 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:32.054864 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf3486-e883-4ab4-b431-b2597581d644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:32.055540 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf3486-e883-4ab4-b431-b2597581d644 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:32.055905 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf3486-e883-4ab4-b431-b2597581d644 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:32.056244 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf3486-e883-4ab4-b431-b2597581d644 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:32.056982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf3486-e883-4ab4-b431-b2597581d644 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:32.057294 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf3486-e883-4ab4-b431-b2597581d644 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:32.058720 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf3486-e883-4ab4-b431-b2597581d644 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:32.077087 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 10/20] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:49:32.087446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8519bc-672e-4d6d-a1bf-fb659f74acc0 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:32.125178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba6e61c-00aa-4e13-b795-b41f9edc92f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:32.125853 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6e61c-00aa-4e13-b795-b41f9edc92f2 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:32.126133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6e61c-00aa-4e13-b795-b41f9edc92f2 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:32.126453 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6e61c-00aa-4e13-b795-b41f9edc92f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:32.127309 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba6e61c-00aa-4e13-b795-b41f9edc92f2 None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.domain.id=default, user.id=15d8f48fa0c344f1aed458217ad0c20b) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:32.127626 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba6e61c-00aa-4e13-b795-b41f9edc92f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:32.129344 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba6e61c-00aa-4e13-b795-b41f9edc92f2 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:32.139598 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 11/21] 10.222.0.84 () {56 vars in 1322 bytes} [Thu May 19 19:49:31 2022] GET /identity/v3/role_assignments?user.id=15d8f48fa0c344f1aed458217ad0c20b&scope.domain.id=default&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:34.620134 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb694f6-c205-4fc4-a0a1-f00274fff299 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:34.620701 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb694f6-c205-4fc4-a0a1-f00274fff299 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:34.620701 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb694f6-c205-4fc4-a0a1-f00274fff299 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:34.622217 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 11/22] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:34.628466 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab1abc-9ac3-4c79-b971-24e3b1bc5736 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:34.629479 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab1abc-9ac3-4c79-b971-24e3b1bc5736 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:34.629981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab1abc-9ac3-4c79-b971-24e3b1bc5736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:34.655824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG passlib.handlers.bcrypt [None req-a2ab1abc-9ac3-4c79-b971-24e3b1bc5736 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=150549) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 19:49:34.656626 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG passlib.handlers.bcrypt [None req-a2ab1abc-9ac3-4c79-b971-24e3b1bc5736 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=150549) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 19:49:34.722495 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-a2ab1abc-9ac3-4c79-b971-24e3b1bc5736 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:34.724903 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-a2ab1abc-9ac3-4c79-b971-24e3b1bc5736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:34.833146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 12/23] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:49:34.835010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafb727-eb01-4d2f-bc25-f48e08dadb36 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:34.835307 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafb727-eb01-4d2f-bc25-f48e08dadb36 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:34.835595 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafb727-eb01-4d2f-bc25-f48e08dadb36 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:34.836957 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 12/24] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:34.848842 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193fa6b-7277-43dd-a7fa-c142254e8a3b None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:34.849466 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193fa6b-7277-43dd-a7fa-c142254e8a3b None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:34.849945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193fa6b-7277-43dd-a7fa-c142254e8a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:34.860875 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafb727-eb01-4d2f-bc25-f48e08dadb36 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:34.879072 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-6193fa6b-7277-43dd-a7fa-c142254e8a3b None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:34.880203 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-6193fa6b-7277-43dd-a7fa-c142254e8a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:34.914787 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:35.007917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 13/25] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:35.009302 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabd7e9-3b6e-40ac-9d26-63b374e57c82 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:35.009765 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabd7e9-3b6e-40ac-9d26-63b374e57c82 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:35.010191 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabd7e9-3b6e-40ac-9d26-63b374e57c82 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:35.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 14/26] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:35.035148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a0540-7316-4aa7-938c-d3f6d78909ce None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:35.035996 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a0540-7316-4aa7-938c-d3f6d78909ce None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:35.036478 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a0540-7316-4aa7-938c-d3f6d78909ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:35.063345 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-426a0540-7316-4aa7-938c-d3f6d78909ce None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:35.064759 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-426a0540-7316-4aa7-938c-d3f6d78909ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:35.068514 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafb727-eb01-4d2f-bc25-f48e08dadb36 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:35.131351 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7c75a-3388-4777-bea3-084e291f509a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:35.132077 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7c75a-3388-4777-bea3-084e291f509a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:35.132367 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7c75a-3388-4777-bea3-084e291f509a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:35.132645 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7c75a-3388-4777-bea3-084e291f509a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:35.149515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b7c75a-3388-4777-bea3-084e291f509a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:35.151198 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b7c75a-3388-4777-bea3-084e291f509a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:35.175502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-31b7c75a-3388-4777-bea3-084e291f509a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:49:35.175502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:35.175502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:35.175502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:35.175502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:35.175502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:35.175502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:35.175502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:35.175502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:35.175502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:35.175502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:35.175502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:35.175502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:49:35.175502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:49:35.175502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:35.176550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:35.177500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:49:35.177500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:49:35.177500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:49:35.177500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:49:35.177500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 13/27] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:49:35.177811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7c75a-3388-4777-bea3-084e291f509a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:35.214671 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 15/28] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:35.214892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:35.215346 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a0540-7316-4aa7-938c-d3f6d78909ce None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:35.342214 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:35.343241 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7c75a-3388-4777-bea3-084e291f509a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:35.359835 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a0540-7316-4aa7-938c-d3f6d78909ce None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:35.381807 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c20a5c-4833-4abf-96df-96ea7cd7fba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:35.382471 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c20a5c-4833-4abf-96df-96ea7cd7fba9 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:35.382763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c20a5c-4833-4abf-96df-96ea7cd7fba9 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:35.383042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c20a5c-4833-4abf-96df-96ea7cd7fba9 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:35.383810 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c20a5c-4833-4abf-96df-96ea7cd7fba9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:35.385341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c20a5c-4833-4abf-96df-96ea7cd7fba9 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:35.408313 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5bef5-a62b-4c1e-9529-cfcde0537478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:35.409591 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5bef5-a62b-4c1e-9529-cfcde0537478 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:35.410023 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5bef5-a62b-4c1e-9529-cfcde0537478 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:35.410426 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5bef5-a62b-4c1e-9529-cfcde0537478 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:35.411274 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5bef5-a62b-4c1e-9529-cfcde0537478 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:35.411663 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5bef5-a62b-4c1e-9529-cfcde0537478 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:35.413303 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5bef5-a62b-4c1e-9529-cfcde0537478 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:35.441507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 16/29] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:35.442503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5bef5-a62b-4c1e-9529-cfcde0537478 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:35.539126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 14/30] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 19:49:35 2022] POST /identity/v3/roles => generated 219 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:49:35.547726 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd1907-a13d-4610-863a-331e0edb8dca None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:35.548039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd1907-a13d-4610-863a-331e0edb8dca None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:35.548296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd1907-a13d-4610-863a-331e0edb8dca None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:35.549860 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 15/31] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:35 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:35.550676 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd1907-a13d-4610-863a-331e0edb8dca None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:35.563820 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5bef5-a62b-4c1e-9529-cfcde0537478 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:35.630886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab4bf4-1ab4-45cd-806e-797709f8abbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:35.631707 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab4bf4-1ab4-45cd-806e-797709f8abbe None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:35.632052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab4bf4-1ab4-45cd-806e-797709f8abbe None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:35.632277 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab4bf4-1ab4-45cd-806e-797709f8abbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:35.652583 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab4bf4-1ab4-45cd-806e-797709f8abbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-2dab4bf4-1ab4-45cd-806e-797709f8abbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:35.665300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:35.667639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:35.669686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:35.669686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:35.669686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:35.669686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:35.669686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:35.669686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:35.669686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:35.669686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:35.669686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:35.669686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:35.669686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:49:35.669686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:49:35.669686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 17/32] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:49:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:49:35.669686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716729d7-bad8-4daf-a03f-c732985e6f21 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:35.671512 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716729d7-bad8-4daf-a03f-c732985e6f21 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:35.671512 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716729d7-bad8-4daf-a03f-c732985e6f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:35.694773 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:35.717290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-716729d7-bad8-4daf-a03f-c732985e6f21 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:35.717290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-716729d7-bad8-4daf-a03f-c732985e6f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:35.742365 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:35.787673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd1907-a13d-4610-863a-331e0edb8dca None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:35.831930 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 18/33] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:35.835334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716729d7-bad8-4daf-a03f-c732985e6f21 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:35.841837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81fdb10-ba1a-4436-9bbb-23feb318f30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:35.844091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fdb10-ba1a-4436-9bbb-23feb318f30c None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:35.844376 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fdb10-ba1a-4436-9bbb-23feb318f30c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:35.846944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fdb10-ba1a-4436-9bbb-23feb318f30c None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:35.847766 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81fdb10-ba1a-4436-9bbb-23feb318f30c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:35.852322 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81fdb10-ba1a-4436-9bbb-23feb318f30c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:35.893866 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 16/34] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:49:35 2022] POST /identity/v3/projects => generated 301 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:49:35.898576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81fdb10-ba1a-4436-9bbb-23feb318f30c None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:36.027010 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716729d7-bad8-4daf-a03f-c732985e6f21 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:36.042704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81fdb10-ba1a-4436-9bbb-23feb318f30c None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:36.075744 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:36.086757 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5abffb-9de4-49f0-b79c-105e318db4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:36.087851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5abffb-9de4-49f0-b79c-105e318db4f7 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:36.088348 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5abffb-9de4-49f0-b79c-105e318db4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:36.088773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5abffb-9de4-49f0-b79c-105e318db4f7 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:36.089701 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5abffb-9de4-49f0-b79c-105e318db4f7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:36.091311 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5abffb-9de4-49f0-b79c-105e318db4f7 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:36.109624 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 17/35] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 19:49:35 2022] POST /identity/v3/roles => generated 225 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:49:36.155316 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6bd21-c724-4e9c-b5f2-e70211f4338f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:36.155947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6bd21-c724-4e9c-b5f2-e70211f4338f None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:36.156241 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6bd21-c724-4e9c-b5f2-e70211f4338f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:36.156521 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6bd21-c724-4e9c-b5f2-e70211f4338f None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:36.157311 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6bd21-c724-4e9c-b5f2-e70211f4338f None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:36.157626 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6bd21-c724-4e9c-b5f2-e70211f4338f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:36.159176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6bd21-c724-4e9c-b5f2-e70211f4338f None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:36.166680 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:36.177277 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 19/36] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:49:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:36.182073 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5abffb-9de4-49f0-b79c-105e318db4f7 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:36.308841 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5abffb-9de4-49f0-b79c-105e318db4f7 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:36.353796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f3372-5af9-4a8f-b0fc-44fcdbee0020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:36.355050 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f3372-5af9-4a8f-b0fc-44fcdbee0020 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:36.355504 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f3372-5af9-4a8f-b0fc-44fcdbee0020 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:36.355940 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f3372-5af9-4a8f-b0fc-44fcdbee0020 None admin] PATH_INFO: `/v3/users/15d8f48fa0c344f1aed458217ad0c20b` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:36.357446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f3372-5af9-4a8f-b0fc-44fcdbee0020 None admin] RBAC: Authorizing `identity:get_user(user_id=15d8f48fa0c344f1aed458217ad0c20b)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:36.359172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f3372-5af9-4a8f-b0fc-44fcdbee0020 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:36.361006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 18/37] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:36 2022] GET /identity/v3/users/15d8f48fa0c344f1aed458217ad0c20b => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:36.366740 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6bd21-c724-4e9c-b5f2-e70211f4338f None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:36.412677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb90d3-d547-4bcf-b9e0-cdea7532b93b None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:36.412677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb90d3-d547-4bcf-b9e0-cdea7532b93b None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:36.413205 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb90d3-d547-4bcf-b9e0-cdea7532b93b None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:36.414756 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 19/38] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:49:36.420389 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff45a3a-9b1f-4958-90bc-02d667729238 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:36.420823 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff45a3a-9b1f-4958-90bc-02d667729238 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:36.421969 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff45a3a-9b1f-4958-90bc-02d667729238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:36.439006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-aff45a3a-9b1f-4958-90bc-02d667729238 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:36.441212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-aff45a3a-9b1f-4958-90bc-02d667729238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:36.497946 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:36.499620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6bd21-c724-4e9c-b5f2-e70211f4338f None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:36.517843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 20/39] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:49:36.525572 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaa3ce-85e6-4622-a91f-48dd297b6c38 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:36.525876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaa3ce-85e6-4622-a91f-48dd297b6c38 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:36.526164 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaa3ce-85e6-4622-a91f-48dd297b6c38 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:36.527949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 21/40] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:49:36.532715 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59522c1b-3d57-4cb6-95e4-55cb8de40b6e None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:36.533012 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59522c1b-3d57-4cb6-95e4-55cb8de40b6e None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:36.533358 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59522c1b-3d57-4cb6-95e4-55cb8de40b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:36.536973 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91b045-5cfc-4f2f-bf4b-4f810467f1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:36.537770 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91b045-5cfc-4f2f-bf4b-4f810467f1a0 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:36.538075 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91b045-5cfc-4f2f-bf4b-4f810467f1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:36.538355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91b045-5cfc-4f2f-bf4b-4f810467f1a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:36.540523 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91b045-5cfc-4f2f-bf4b-4f810467f1a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:36.541258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91b045-5cfc-4f2f-bf4b-4f810467f1a0 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:36.543023 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 20/41] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:49:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:49:36.543765 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91b045-5cfc-4f2f-bf4b-4f810467f1a0 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:36.554201 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-59522c1b-3d57-4cb6-95e4-55cb8de40b6e None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:36.555451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-59522c1b-3d57-4cb6-95e4-55cb8de40b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:36.609492 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:36.624112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 22/42] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:49:36.624940 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59522c1b-3d57-4cb6-95e4-55cb8de40b6e None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:36.649221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91b045-5cfc-4f2f-bf4b-4f810467f1a0 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:36.683400 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bb738-f6f8-40f5-839f-77e137f7be56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:36.683994 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bb738-f6f8-40f5-839f-77e137f7be56 None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:36.684278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bb738-f6f8-40f5-839f-77e137f7be56 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:36.684554 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bb738-f6f8-40f5-839f-77e137f7be56 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:36.685305 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387bb738-f6f8-40f5-839f-77e137f7be56 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:36.686736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387bb738-f6f8-40f5-839f-77e137f7be56 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:36.727737 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59522c1b-3d57-4cb6-95e4-55cb8de40b6e None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:36.744954 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:36.762957 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd894719-7c8f-4cde-95e9-d16655a5be6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:36.763849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd894719-7c8f-4cde-95e9-d16655a5be6d None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:36.764133 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd894719-7c8f-4cde-95e9-d16655a5be6d None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:36.764418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd894719-7c8f-4cde-95e9-d16655a5be6d None admin] PATH_INFO: `/v3/domains/default/users/15d8f48fa0c344f1aed458217ad0c20b/roles/2cd6d3dd613c4824a26cacf53fb6c07f` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:36.766452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd894719-7c8f-4cde-95e9-d16655a5be6d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=15d8f48fa0c344f1aed458217ad0c20b, role_id=2cd6d3dd613c4824a26cacf53fb6c07f)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:36.768026 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd894719-7c8f-4cde-95e9-d16655a5be6d None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:36.782249 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 23/43] 10.222.0.84 () {58 vars in 1361 bytes} [Thu May 19 19:49:36 2022] PUT /identity/v3/domains/default/users/15d8f48fa0c344f1aed458217ad0c20b/roles/2cd6d3dd613c4824a26cacf53fb6c07f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:49:36.783100 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd894719-7c8f-4cde-95e9-d16655a5be6d None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:36.807679 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 21/44] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 19:49:36 2022] POST /identity/v3/roles => generated 223 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:49:36.919373 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd894719-7c8f-4cde-95e9-d16655a5be6d None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:36.960213 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd80dc1-7255-48b1-a003-0bdd48735d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:36.960971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd80dc1-7255-48b1-a003-0bdd48735d7c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:36.961261 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd80dc1-7255-48b1-a003-0bdd48735d7c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:36.961549 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd80dc1-7255-48b1-a003-0bdd48735d7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:36.962921 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd80dc1-7255-48b1-a003-0bdd48735d7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:36.964466 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd80dc1-7255-48b1-a003-0bdd48735d7c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:36.966431 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 24/45] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:49:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:36.972295 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bb738-f6f8-40f5-839f-77e137f7be56 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:37.083741 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bb738-f6f8-40f5-839f-77e137f7be56 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:37.112788 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:37.122346 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171ef18-fb39-4ff3-bdbf-5a4b2beb6da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:37.123030 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171ef18-fb39-4ff3-bdbf-5a4b2beb6da6 None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:37.123316 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171ef18-fb39-4ff3-bdbf-5a4b2beb6da6 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:37.123596 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171ef18-fb39-4ff3-bdbf-5a4b2beb6da6 None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:37.124318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171ef18-fb39-4ff3-bdbf-5a4b2beb6da6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:37.125911 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171ef18-fb39-4ff3-bdbf-5a4b2beb6da6 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:37.159828 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 22/46] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:49:36 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:49:39.014384 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff4c99-de46-4c53-a342-a7c4bd9f4210 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:39.014920 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff4c99-de46-4c53-a342-a7c4bd9f4210 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:39.014920 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff4c99-de46-4c53-a342-a7c4bd9f4210 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:39.016251 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 25/47] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:39.021915 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af0832-3bd5-4636-9e1e-28cdb91dff0e None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:39.022426 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af0832-3bd5-4636-9e1e-28cdb91dff0e None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:39.022522 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af0832-3bd5-4636-9e1e-28cdb91dff0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:39.040665 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-e0af0832-3bd5-4636-9e1e-28cdb91dff0e None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:39.042434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-e0af0832-3bd5-4636-9e1e-28cdb91dff0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:39.112838 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 23/48] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:39.136080 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff4c99-de46-4c53-a342-a7c4bd9f4210 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:39.185811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:39.247772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff4c99-de46-4c53-a342-a7c4bd9f4210 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:39.282579 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903275e-28aa-4237-bd3e-98af76370d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:39.283433 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903275e-28aa-4237-bd3e-98af76370d35 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:39.283913 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903275e-28aa-4237-bd3e-98af76370d35 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:39.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903275e-28aa-4237-bd3e-98af76370d35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:39.298145 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903275e-28aa-4237-bd3e-98af76370d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-c903275e-28aa-4237-bd3e-98af76370d35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:39.306972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:39.307993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:39.308923 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:39.308923 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:39.308923 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:39.308923 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:39.308923 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:39.308923 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:39.308923 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:39.308923 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:39.308923 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:39.308923 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:39.308923 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:49:39.308923 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:49:39.308923 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 26/49] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:49:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:49:39.314540 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af0832-3bd5-4636-9e1e-28cdb91dff0e None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:39.371859 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0528f-a946-4c7c-9bd8-d88f800fa655 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:39.372185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0528f-a946-4c7c-9bd8-d88f800fa655 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:39.372574 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0528f-a946-4c7c-9bd8-d88f800fa655 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:39.373946 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 27/50] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:39.379726 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f2e3d-2497-401b-a2bd-3d9c861cfb25 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:39.380021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f2e3d-2497-401b-a2bd-3d9c861cfb25 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:39.380306 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f2e3d-2497-401b-a2bd-3d9c861cfb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:39.398659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-ba1f2e3d-2497-401b-a2bd-3d9c861cfb25 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:39.399880 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-ba1f2e3d-2497-401b-a2bd-3d9c861cfb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:39.417067 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af0832-3bd5-4636-9e1e-28cdb91dff0e None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:39.427530 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:39.453479 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e75ac-35f7-43fe-aba1-c50013a77ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:39.454165 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e75ac-35f7-43fe-aba1-c50013a77ac6 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:39.454509 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e75ac-35f7-43fe-aba1-c50013a77ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:39.454902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e75ac-35f7-43fe-aba1-c50013a77ac6 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:39.456137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e75ac-35f7-43fe-aba1-c50013a77ac6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:39.456457 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e75ac-35f7-43fe-aba1-c50013a77ac6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:39.457875 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e75ac-35f7-43fe-aba1-c50013a77ac6 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:39.466441 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 24/51] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:49:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:39.470717 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 28/52] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:39.473241 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e75ac-35f7-43fe-aba1-c50013a77ac6 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:39.494034 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f2e3d-2497-401b-a2bd-3d9c861cfb25 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:39.532855 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:39.582937 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e75ac-35f7-43fe-aba1-c50013a77ac6 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:39.610156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f2e3d-2497-401b-a2bd-3d9c861cfb25 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:39.617215 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4df09-60ea-4c47-b594-04ba697ee66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:39.618105 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4df09-60ea-4c47-b594-04ba697ee66b None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:39.618425 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4df09-60ea-4c47-b594-04ba697ee66b None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:39.618709 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4df09-60ea-4c47-b594-04ba697ee66b None admin] PATH_INFO: `/v3/users/15d8f48fa0c344f1aed458217ad0c20b` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:39.620236 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4df09-60ea-4c47-b594-04ba697ee66b None admin] RBAC: Authorizing `identity:get_user(user_id=15d8f48fa0c344f1aed458217ad0c20b)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:39.621701 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4df09-60ea-4c47-b594-04ba697ee66b None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:39.623438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 25/53] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:39 2022] GET /identity/v3/users/15d8f48fa0c344f1aed458217ad0c20b => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:49:39.629323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4df09-60ea-4c47-b594-04ba697ee66b None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:39.647849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301decc-e785-43ca-864f-a10a0479c4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:39.648656 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301decc-e785-43ca-864f-a10a0479c4da None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:39.648945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301decc-e785-43ca-864f-a10a0479c4da None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:39.649229 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301decc-e785-43ca-864f-a10a0479c4da None admin] PATH_INFO: `/v3/domains/default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:39.650483 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301decc-e785-43ca-864f-a10a0479c4da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:39.651953 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301decc-e785-43ca-864f-a10a0479c4da None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:39.653609 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 29/54] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:49:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:39.659540 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301decc-e785-43ca-864f-a10a0479c4da None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:39.672169 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:39.734973 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4df09-60ea-4c47-b594-04ba697ee66b None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:39.765089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301decc-e785-43ca-864f-a10a0479c4da None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:39.766689 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:39.769159 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b1876-19e7-4f75-9184-e5f3c2d50db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:39.769952 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b1876-19e7-4f75-9184-e5f3c2d50db6 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:39.770221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b1876-19e7-4f75-9184-e5f3c2d50db6 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:39.770526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b1876-19e7-4f75-9184-e5f3c2d50db6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:39.771737 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b1876-19e7-4f75-9184-e5f3c2d50db6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:39.773204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b1876-19e7-4f75-9184-e5f3c2d50db6 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:39.774879 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 26/55] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:49:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:39.781054 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b1876-19e7-4f75-9184-e5f3c2d50db6 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:39.801327 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d9308-0f2d-4038-b469-c8ba24afcd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:39.801665 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d9308-0f2d-4038-b469-c8ba24afcd1d None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:39.801950 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d9308-0f2d-4038-b469-c8ba24afcd1d None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:39.802230 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d9308-0f2d-4038-b469-c8ba24afcd1d None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:39.802987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d9308-0f2d-4038-b469-c8ba24afcd1d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:39.804452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d9308-0f2d-4038-b469-c8ba24afcd1d None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:39.831505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 30/56] 10.222.0.84 () {60 vars in 1140 bytes} [Thu May 19 19:49:39 2022] POST /identity/v3/projects => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:49:39.895742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b1876-19e7-4f75-9184-e5f3c2d50db6 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:39.903902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:39.933778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c8ccd-c96a-41f6-a101-e02918409c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:39.934517 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c8ccd-c96a-41f6-a101-e02918409c54 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:39.934803 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c8ccd-c96a-41f6-a101-e02918409c54 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:39.935080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c8ccd-c96a-41f6-a101-e02918409c54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:39.935951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c8ccd-c96a-41f6-a101-e02918409c54 None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.domain.id=default, user.id=15d8f48fa0c344f1aed458217ad0c20b) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:39.936261 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c8ccd-c96a-41f6-a101-e02918409c54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:39.937771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c8ccd-c96a-41f6-a101-e02918409c54 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:39.944719 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 27/57] 10.222.0.84 () {56 vars in 1322 bytes} [Thu May 19 19:49:39 2022] GET /identity/v3/role_assignments?user.id=15d8f48fa0c344f1aed458217ad0c20b&scope.domain.id=default&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 530 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:42.055317 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b04b5f-9a8d-46e5-8185-e6bc5bb21693 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:42.055894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b04b5f-9a8d-46e5-8185-e6bc5bb21693 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:42.055894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b04b5f-9a8d-46e5-8185-e6bc5bb21693 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:42.057301 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 31/58] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:42.063885 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c34ea8-4ed8-4fc6-a173-8bfde0a5dc70 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:42.064669 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c34ea8-4ed8-4fc6-a173-8bfde0a5dc70 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:42.064931 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c34ea8-4ed8-4fc6-a173-8bfde0a5dc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:42.093922 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-b1c34ea8-4ed8-4fc6-a173-8bfde0a5dc70 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:42.095854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-b1c34ea8-4ed8-4fc6-a173-8bfde0a5dc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:42.165350 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 28/59] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:42.190437 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b04b5f-9a8d-46e5-8185-e6bc5bb21693 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:42.302718 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b04b5f-9a8d-46e5-8185-e6bc5bb21693 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:42.337213 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8071a-e5b4-4135-9a63-7c2537fee116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:42.337944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8071a-e5b4-4135-9a63-7c2537fee116 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:42.338226 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8071a-e5b4-4135-9a63-7c2537fee116 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:42.338537 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8071a-e5b4-4135-9a63-7c2537fee116 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:42.339784 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8071a-e5b4-4135-9a63-7c2537fee116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:42.341252 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8071a-e5b4-4135-9a63-7c2537fee116 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:42.342895 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 32/60] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:49:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:42.349889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c34ea8-4ed8-4fc6-a173-8bfde0a5dc70 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:42.457737 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:42.461607 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c34ea8-4ed8-4fc6-a173-8bfde0a5dc70 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:42.501524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e0f7c-7e0e-4c24-b327-7d49ec74975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:42.502199 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e0f7c-7e0e-4c24-b327-7d49ec74975c None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:42.502516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e0f7c-7e0e-4c24-b327-7d49ec74975c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:42.502799 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e0f7c-7e0e-4c24-b327-7d49ec74975c None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:42.503517 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e0f7c-7e0e-4c24-b327-7d49ec74975c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:42.504993 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e0f7c-7e0e-4c24-b327-7d49ec74975c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:42.539161 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 29/61] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:49:42 2022] POST /identity/v3/users => generated 275 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:49:45.991663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a23ed-c980-4d38-bfb7-f5b4e3aa9efb None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:45.992705 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a23ed-c980-4d38-bfb7-f5b4e3aa9efb None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:45.993205 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a23ed-c980-4d38-bfb7-f5b4e3aa9efb None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:45.994826 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 33/62] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:46.000790 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baf15c-c0e4-493f-ad86-ac733eea3311 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:46.001810 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baf15c-c0e4-493f-ad86-ac733eea3311 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:46.002330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baf15c-c0e4-493f-ad86-ac733eea3311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:46.019658 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-e0baf15c-c0e4-493f-ad86-ac733eea3311 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:46.021196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-e0baf15c-c0e4-493f-ad86-ac733eea3311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:46.094006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0969c-1364-40c2-aa10-b21df6f61653 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:46.094693 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0969c-1364-40c2-aa10-b21df6f61653 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:46.095201 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0969c-1364-40c2-aa10-b21df6f61653 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:46.097311 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 34/63] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:46.102919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b0b76-dc72-433a-82fb-cdbda890e506 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:46.103508 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b0b76-dc72-433a-82fb-cdbda890e506 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:46.103985 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b0b76-dc72-433a-82fb-cdbda890e506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:46.128618 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-8c8b0b76-dc72-433a-82fb-cdbda890e506 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:46.130729 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-8c8b0b76-dc72-433a-82fb-cdbda890e506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:46.136638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 30/64] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:46.161261 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0baf15c-c0e4-493f-ad86-ac733eea3311 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:46.164538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:46.247322 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 35/65] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:49:46.292421 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b0b76-dc72-433a-82fb-cdbda890e506 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:46.316477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:46.345449 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0baf15c-c0e4-493f-ad86-ac733eea3311 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:46.400515 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a187d1d-bc78-4564-870f-bebba6f86c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:46.401763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a187d1d-bc78-4564-870f-bebba6f86c14 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:46.402061 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a187d1d-bc78-4564-870f-bebba6f86c14 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:46.402389 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a187d1d-bc78-4564-870f-bebba6f86c14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:46.413834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a187d1d-bc78-4564-870f-bebba6f86c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:46.414592 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b0b76-dc72-433a-82fb-cdbda890e506 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-2a187d1d-bc78-4564-870f-bebba6f86c14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:46.421599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:46.422597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:46.423549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:46.423549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:46.423549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:46.423549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:46.423549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:46.423549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:46.423549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:46.423549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:46.423549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:46.423549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:46.423549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:49:46.423549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:49:46.423549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 31/66] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:49:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:49:46.425041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7d096-7d45-4454-a79c-beb7a5a34453 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:46.425041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7d096-7d45-4454-a79c-beb7a5a34453 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:46.425041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7d096-7d45-4454-a79c-beb7a5a34453 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:46.426314 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 32/67] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:46.427290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0b2dd-51a6-41e6-a4fb-07cedd22db64 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:46.427577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0b2dd-51a6-41e6-a4fb-07cedd22db64 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:46.427857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0b2dd-51a6-41e6-a4fb-07cedd22db64 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:46.429199 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 33/68] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:46.433991 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0b2dd-51a6-41e6-a4fb-07cedd22db64 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:46.458938 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4633dd-21f5-4976-bdc3-def18c311b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:46.460045 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4633dd-21f5-4976-bdc3-def18c311b71 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:46.460336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4633dd-21f5-4976-bdc3-def18c311b71 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:46.460612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4633dd-21f5-4976-bdc3-def18c311b71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:46.471212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4633dd-21f5-4976-bdc3-def18c311b71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-af4633dd-21f5-4976-bdc3-def18c311b71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:46.479171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:46.480200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:46.481112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:46.481112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:46.481112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:46.481112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:46.481112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:46.481112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:46.481112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:46.481112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:46.481112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:46.481112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:46.481112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:49:46.481112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:49:46.481112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 36/69] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:49:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:49:46.481863 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd1bae-95f6-48e3-987b-9167804c7d98 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:46.481863 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd1bae-95f6-48e3-987b-9167804c7d98 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:46.482130 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd1bae-95f6-48e3-987b-9167804c7d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:46.502299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-73cd1bae-95f6-48e3-987b-9167804c7d98 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:46.503732 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-73cd1bae-95f6-48e3-987b-9167804c7d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:46.511377 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:46.548572 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0b2dd-51a6-41e6-a4fb-07cedd22db64 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:46.570951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 37/70] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:46.575615 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd47484-aa42-422d-8eab-1e589b8d08bb None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:46.575912 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd47484-aa42-422d-8eab-1e589b8d08bb None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:46.576234 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd47484-aa42-422d-8eab-1e589b8d08bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:46.581915 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3da93-6ae5-4163-acb2-8d668bb45e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:46.582878 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3da93-6ae5-4163-acb2-8d668bb45e7b None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:46.583372 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3da93-6ae5-4163-acb2-8d668bb45e7b None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:46.583845 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3da93-6ae5-4163-acb2-8d668bb45e7b None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:46.584800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3da93-6ae5-4163-acb2-8d668bb45e7b None admin] RBAC: Adding query filter params (name=member) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:46.585296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3da93-6ae5-4163-acb2-8d668bb45e7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:46.586929 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3da93-6ae5-4163-acb2-8d668bb45e7b None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:46.592699 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-fbd47484-aa42-422d-8eab-1e589b8d08bb None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:46.594203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-fbd47484-aa42-422d-8eab-1e589b8d08bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:46.596114 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 34/71] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:49:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:46.596940 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3da93-6ae5-4163-acb2-8d668bb45e7b None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:46.609117 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:46.664403 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 38/72] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:46.665974 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b22d6-f7bc-44d8-8358-0f3525e097fd None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:46.666280 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b22d6-f7bc-44d8-8358-0f3525e097fd None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:46.666568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b22d6-f7bc-44d8-8358-0f3525e097fd None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:46.667835 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 39/73] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:46.668525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b22d6-f7bc-44d8-8358-0f3525e097fd None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:46.709644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3da93-6ae5-4163-acb2-8d668bb45e7b None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:46.743722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d86ce-15f4-409c-b368-42e0cb40d616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:46.744380 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d86ce-15f4-409c-b368-42e0cb40d616 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:46.744671 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d86ce-15f4-409c-b368-42e0cb40d616 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:46.744802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:46.744947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d86ce-15f4-409c-b368-42e0cb40d616 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:46.745906 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d86ce-15f4-409c-b368-42e0cb40d616 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:46.746353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d86ce-15f4-409c-b368-42e0cb40d616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:46.747758 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d86ce-15f4-409c-b368-42e0cb40d616 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:46.756519 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 35/74] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:49:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:46.757259 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d86ce-15f4-409c-b368-42e0cb40d616 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:46.776411 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b22d6-f7bc-44d8-8358-0f3525e097fd None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:46.813241 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45353e-3841-43c1-9115-19fb66c3a2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:46.814170 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45353e-3841-43c1-9115-19fb66c3a2b5 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:46.814558 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45353e-3841-43c1-9115-19fb66c3a2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:46.814852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45353e-3841-43c1-9115-19fb66c3a2b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:46.825175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45353e-3841-43c1-9115-19fb66c3a2b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:46.828176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-3a45353e-3841-43c1-9115-19fb66c3a2b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:46.834208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:46.835426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:46.836344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:46.836344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:46.836344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:46.836344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:46.836344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:46.836344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:46.836344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:46.836344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:46.836344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:46.836344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:46.836344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:49:46.836344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:49:46.836344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 40/75] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:49:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:49:46.837094 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca120bd-dff6-4862-a74a-68fdb9699c0b None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:46.837094 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca120bd-dff6-4862-a74a-68fdb9699c0b None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:46.837237 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca120bd-dff6-4862-a74a-68fdb9699c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:46.854867 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-9ca120bd-dff6-4862-a74a-68fdb9699c0b None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:46.856074 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-9ca120bd-dff6-4862-a74a-68fdb9699c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:46.865598 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d86ce-15f4-409c-b368-42e0cb40d616 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:46.900397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86427199-076a-4f83-809c-857e774bb591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:46.901301 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86427199-076a-4f83-809c-857e774bb591 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:46.901586 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86427199-076a-4f83-809c-857e774bb591 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:46.901885 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86427199-076a-4f83-809c-857e774bb591 None admin] PATH_INFO: `/v3/users/85b48f8334a3449caeae652db3f7ae7f` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:46.917267 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86427199-076a-4f83-809c-857e774bb591 None admin] RBAC: Authorizing `identity:get_user(user_id=85b48f8334a3449caeae652db3f7ae7f)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:46.918800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86427199-076a-4f83-809c-857e774bb591 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:46.920507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 36/76] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:46 2022] GET /identity/v3/users/85b48f8334a3449caeae652db3f7ae7f => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:46.923868 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 41/77] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:46.924306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86427199-076a-4f83-809c-857e774bb591 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:46.924816 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca120bd-dff6-4862-a74a-68fdb9699c0b None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:46.988400 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:47.028591 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86427199-076a-4f83-809c-857e774bb591 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:47.029143 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca120bd-dff6-4862-a74a-68fdb9699c0b None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:47.055323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:47.063094 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9322b75-9fdc-4a68-bd5e-c5db32978007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:47.063527 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2137841-ef49-4a91-9f60-c789cfcac87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:47.063987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9322b75-9fdc-4a68-bd5e-c5db32978007 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:47.064290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9322b75-9fdc-4a68-bd5e-c5db32978007 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:47.064520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2137841-ef49-4a91-9f60-c789cfcac87b None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:47.064616 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9322b75-9fdc-4a68-bd5e-c5db32978007 None admin] PATH_INFO: `/v3/users/15d8f48fa0c344f1aed458217ad0c20b` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:47.065014 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2137841-ef49-4a91-9f60-c789cfcac87b None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:47.065308 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2137841-ef49-4a91-9f60-c789cfcac87b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:47.065756 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9322b75-9fdc-4a68-bd5e-c5db32978007 None admin] RBAC: Authorizing `identity:get_user(user_id=15d8f48fa0c344f1aed458217ad0c20b)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:47.067255 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9322b75-9fdc-4a68-bd5e-c5db32978007 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:47.068906 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 42/78] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:46 2022] GET /identity/v3/users/15d8f48fa0c344f1aed458217ad0c20b => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:47.069660 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9322b75-9fdc-4a68-bd5e-c5db32978007 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:47.079811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2137841-ef49-4a91-9f60-c789cfcac87b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-e2137841-ef49-4a91-9f60-c789cfcac87b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:47.088899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:47.090122 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:47.095966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:47.095966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:47.095966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:47.095966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:47.095966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:47.095966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:47.095966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:47.095966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:47.095966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:47.095966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:47.095966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:49:47.095966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:49:47.095966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 37/79] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:49:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:49:47.095966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2137841-ef49-4a91-9f60-c789cfcac87b None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:47.174800 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9322b75-9fdc-4a68-bd5e-c5db32978007 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:47.190830 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2137841-ef49-4a91-9f60-c789cfcac87b None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:47.202878 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:47.209949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62ad29-e013-4bba-a0d4-d591ab1ade88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:47.210664 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62ad29-e013-4bba-a0d4-d591ab1ade88 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:47.210953 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62ad29-e013-4bba-a0d4-d591ab1ade88 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:47.211231 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62ad29-e013-4bba-a0d4-d591ab1ade88 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:47.212007 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62ad29-e013-4bba-a0d4-d591ab1ade88 None admin] RBAC: Adding query filter params (name=member) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:47.212319 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62ad29-e013-4bba-a0d4-d591ab1ade88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:47.213709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62ad29-e013-4bba-a0d4-d591ab1ade88 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:47.225909 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 43/80] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:49:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:47.226732 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62ad29-e013-4bba-a0d4-d591ab1ade88 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:47.231761 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701173d-470d-4441-8426-564aa3c443c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:47.232424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701173d-470d-4441-8426-564aa3c443c3 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:47.232715 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701173d-470d-4441-8426-564aa3c443c3 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:47.232993 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701173d-470d-4441-8426-564aa3c443c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:47.289739 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:47.302945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701173d-470d-4441-8426-564aa3c443c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:47.304467 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701173d-470d-4441-8426-564aa3c443c3 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:47.325472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62ad29-e013-4bba-a0d4-d591ab1ade88 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:47.359866 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2630b-ec5f-4e99-aeb5-6ef7abd72059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:47.360746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2630b-ec5f-4e99-aeb5-6ef7abd72059 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:47.361144 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2630b-ec5f-4e99-aeb5-6ef7abd72059 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:47.361496 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2630b-ec5f-4e99-aeb5-6ef7abd72059 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:47.362700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2630b-ec5f-4e99-aeb5-6ef7abd72059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:47.364171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2630b-ec5f-4e99-aeb5-6ef7abd72059 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:47.365854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 44/81] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:49:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:47.366623 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2630b-ec5f-4e99-aeb5-6ef7abd72059 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:47.403980 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 38/82] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:49:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:49:47.403980 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701173d-470d-4441-8426-564aa3c443c3 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:47.432987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:47.475435 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2630b-ec5f-4e99-aeb5-6ef7abd72059 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:47.495024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:47.508090 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701173d-470d-4441-8426-564aa3c443c3 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:47.508870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e5b6d-bf49-46b2-9406-d8c35e53113e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:47.509522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e5b6d-bf49-46b2-9406-d8c35e53113e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:47.509813 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e5b6d-bf49-46b2-9406-d8c35e53113e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:47.510096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e5b6d-bf49-46b2-9406-d8c35e53113e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:47.541961 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d740243-db30-460b-8aff-36d7322b39b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:47.542583 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d740243-db30-460b-8aff-36d7322b39b5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:47.542869 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d740243-db30-460b-8aff-36d7322b39b5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:47.543146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d740243-db30-460b-8aff-36d7322b39b5 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:47.543899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d740243-db30-460b-8aff-36d7322b39b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:47.544213 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d740243-db30-460b-8aff-36d7322b39b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:47.545598 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d740243-db30-460b-8aff-36d7322b39b5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:47.553400 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 39/83] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:49:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:47.554183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d740243-db30-460b-8aff-36d7322b39b5 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:47.578544 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e5b6d-bf49-46b2-9406-d8c35e53113e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:47.580317 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e5b6d-bf49-46b2-9406-d8c35e53113e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:47.651220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d740243-db30-460b-8aff-36d7322b39b5 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:47.652671 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:47.686710 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98ce21-60c3-4465-9f79-b5541798918a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:47.687585 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98ce21-60c3-4465-9f79-b5541798918a None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:47.687869 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98ce21-60c3-4465-9f79-b5541798918a None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:47.688147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98ce21-60c3-4465-9f79-b5541798918a None admin] PATH_INFO: `/v3/users/85b48f8334a3449caeae652db3f7ae7f` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:47.689395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98ce21-60c3-4465-9f79-b5541798918a None admin] RBAC: Authorizing `identity:get_user(user_id=85b48f8334a3449caeae652db3f7ae7f)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:47.690887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98ce21-60c3-4465-9f79-b5541798918a None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:47.692531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 40/84] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:47 2022] GET /identity/v3/users/85b48f8334a3449caeae652db3f7ae7f => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:47.693271 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98ce21-60c3-4465-9f79-b5541798918a None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:47.696187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 45/85] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:49:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:49:47.697157 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e5b6d-bf49-46b2-9406-d8c35e53113e None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:47.712983 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:47.794232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98ce21-60c3-4465-9f79-b5541798918a None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:47.794644 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e5b6d-bf49-46b2-9406-d8c35e53113e None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:47.832083 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28755e-9311-40d2-aff4-59570710773f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:47.832591 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65070c6c-8588-4a9e-af9c-460615efe4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:47.832722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28755e-9311-40d2-aff4-59570710773f None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:47.833024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28755e-9311-40d2-aff4-59570710773f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:47.833298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28755e-9311-40d2-aff4-59570710773f None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:47.833461 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65070c6c-8588-4a9e-af9c-460615efe4ec None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:47.833747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65070c6c-8588-4a9e-af9c-460615efe4ec None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:47.834020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec28755e-9311-40d2-aff4-59570710773f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:47.834120 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65070c6c-8588-4a9e-af9c-460615efe4ec None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:47.835258 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65070c6c-8588-4a9e-af9c-460615efe4ec None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:47.835529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec28755e-9311-40d2-aff4-59570710773f None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:47.836706 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65070c6c-8588-4a9e-af9c-460615efe4ec None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:47.838458 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 46/86] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:49:47 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:47.839303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65070c6c-8588-4a9e-af9c-460615efe4ec None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:47.860215 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 41/87] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:49:47 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:49:47.861179 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28755e-9311-40d2-aff4-59570710773f None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:47.872485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:47.954663 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:47.973142 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65070c6c-8588-4a9e-af9c-460615efe4ec None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:47.987610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28755e-9311-40d2-aff4-59570710773f None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:48.019623 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56695554-8c9e-4736-bdd6-99bd9754690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:48.020942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56695554-8c9e-4736-bdd6-99bd9754690e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:48.021330 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56695554-8c9e-4736-bdd6-99bd9754690e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:48.021718 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56695554-8c9e-4736-bdd6-99bd9754690e None admin] PATH_INFO: `/v3/users/85b48f8334a3449caeae652db3f7ae7f` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:48.023109 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56695554-8c9e-4736-bdd6-99bd9754690e None admin] RBAC: Authorizing `identity:get_user(user_id=85b48f8334a3449caeae652db3f7ae7f)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:48.023223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18b397-88ad-43d1-ba4f-7e8f3e384894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:48.024022 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18b397-88ad-43d1-ba4f-7e8f3e384894 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:48.024305 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18b397-88ad-43d1-ba4f-7e8f3e384894 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:48.024581 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18b397-88ad-43d1-ba4f-7e8f3e384894 None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:48.024680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56695554-8c9e-4736-bdd6-99bd9754690e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:48.025880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f18b397-88ad-43d1-ba4f-7e8f3e384894 None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:48.026624 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 47/88] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:47 2022] GET /identity/v3/users/85b48f8334a3449caeae652db3f7ae7f => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:48.027208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56695554-8c9e-4736-bdd6-99bd9754690e None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:48.027578 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f18b397-88ad-43d1-ba4f-7e8f3e384894 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:48.034092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 42/89] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:49:47 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:48.034092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18b397-88ad-43d1-ba4f-7e8f3e384894 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:48.148640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:48.156787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56695554-8c9e-4736-bdd6-99bd9754690e None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:48.157859 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18b397-88ad-43d1-ba4f-7e8f3e384894 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:48.205024 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42be1a3-394c-48f2-9aef-24ce83ae5db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:48.205675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42be1a3-394c-48f2-9aef-24ce83ae5db9 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:48.205948 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42be1a3-394c-48f2-9aef-24ce83ae5db9 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:48.206227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42be1a3-394c-48f2-9aef-24ce83ae5db9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:48.207980 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77aa6c1-5a1a-44ab-8d3c-7555872b881d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:48.209625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77aa6c1-5a1a-44ab-8d3c-7555872b881d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:48.209625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77aa6c1-5a1a-44ab-8d3c-7555872b881d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:48.209625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77aa6c1-5a1a-44ab-8d3c-7555872b881d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:48.210051 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77aa6c1-5a1a-44ab-8d3c-7555872b881d None admin] RBAC: Adding query filter params (role.id=2fd6a3e7df18446993c578dab05dfdce, scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18, user.id=85b48f8334a3449caeae652db3f7ae7f) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:48.214355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77aa6c1-5a1a-44ab-8d3c-7555872b881d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:48.214355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77aa6c1-5a1a-44ab-8d3c-7555872b881d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:48.218977 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 43/90] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:49:48 2022] GET /identity/v3/role_assignments?user.id=85b48f8334a3449caeae652db3f7ae7f&scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18&role.id=2fd6a3e7df18446993c578dab05dfdce => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:48.220950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77aa6c1-5a1a-44ab-8d3c-7555872b881d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:48.233088 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:48.312671 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42be1a3-394c-48f2-9aef-24ce83ae5db9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:48.314287 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42be1a3-394c-48f2-9aef-24ce83ae5db9 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:48.401991 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77aa6c1-5a1a-44ab-8d3c-7555872b881d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:48.445182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c03ab5-7c35-408a-9725-6f4a168273f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:48.446112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c03ab5-7c35-408a-9725-6f4a168273f4 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:48.446440 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c03ab5-7c35-408a-9725-6f4a168273f4 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:48.446721 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c03ab5-7c35-408a-9725-6f4a168273f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:48.509939 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:48.520544 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 48/91] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:49:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:49:48.521455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42be1a3-394c-48f2-9aef-24ce83ae5db9 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:48.563295 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c03ab5-7c35-408a-9725-6f4a168273f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:48.565013 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c03ab5-7c35-408a-9725-6f4a168273f4 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:48.578222 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:48.709879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42be1a3-394c-48f2-9aef-24ce83ae5db9 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:48.753384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 44/92] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:49:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:49:48.754287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c03ab5-7c35-408a-9725-6f4a168273f4 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:48.777450 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af87430-f932-4ed9-bc49-c5011cfb83e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:48.778126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af87430-f932-4ed9-bc49-c5011cfb83e3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:48.778440 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af87430-f932-4ed9-bc49-c5011cfb83e3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:48.778721 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af87430-f932-4ed9-bc49-c5011cfb83e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:48.779579 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af87430-f932-4ed9-bc49-c5011cfb83e3 None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18, user.id=15d8f48fa0c344f1aed458217ad0c20b) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:48.779891 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af87430-f932-4ed9-bc49-c5011cfb83e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:48.781631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af87430-f932-4ed9-bc49-c5011cfb83e3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:48.796368 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 49/93] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:49:48 2022] GET /identity/v3/role_assignments?user.id=15d8f48fa0c344f1aed458217ad0c20b&scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:48.797229 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af87430-f932-4ed9-bc49-c5011cfb83e3 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:48.924644 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:48.943863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c03ab5-7c35-408a-9725-6f4a168273f4 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:48.951718 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:48.981624 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35a7df-ce6f-41ca-827a-d92ce2da43c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:48.982363 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35a7df-ce6f-41ca-827a-d92ce2da43c3 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:48.982654 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35a7df-ce6f-41ca-827a-d92ce2da43c3 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:48.982936 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35a7df-ce6f-41ca-827a-d92ce2da43c3 None admin] PATH_INFO: `/v3/projects/e6e5887740a34024a565b8aae8b2a3e5` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:48.984284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35a7df-ce6f-41ca-827a-d92ce2da43c3 None admin] RBAC: Authorizing `identity:get_project(project_id=e6e5887740a34024a565b8aae8b2a3e5)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:48.985699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35a7df-ce6f-41ca-827a-d92ce2da43c3 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:48.987648 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 45/94] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:49:48 2022] GET /identity/v3/projects/e6e5887740a34024a565b8aae8b2a3e5 => generated 312 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:48.991737 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af87430-f932-4ed9-bc49-c5011cfb83e3 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:48.995333 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35a7df-ce6f-41ca-827a-d92ce2da43c3 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:49.056534 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b874066-49ca-4f39-839f-6508821efb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:49.057935 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b874066-49ca-4f39-839f-6508821efb25 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:49.060391 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b874066-49ca-4f39-839f-6508821efb25 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:49.060872 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b874066-49ca-4f39-839f-6508821efb25 None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:49.062662 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b874066-49ca-4f39-839f-6508821efb25 None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:49.065596 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b874066-49ca-4f39-839f-6508821efb25 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:49.072465 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 50/95] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:49:48 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:49.078567 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b874066-49ca-4f39-839f-6508821efb25 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:49.197035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35a7df-ce6f-41ca-827a-d92ce2da43c3 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:49.247604 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b874066-49ca-4f39-839f-6508821efb25 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:49.257859 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56973b-03ee-48c1-9474-db61612b75b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:49.262299 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56973b-03ee-48c1-9474-db61612b75b7 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:49.262815 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56973b-03ee-48c1-9474-db61612b75b7 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:49.263292 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56973b-03ee-48c1-9474-db61612b75b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:49.264325 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56973b-03ee-48c1-9474-db61612b75b7 None admin] RBAC: Adding query filter params (role.id=2fd6a3e7df18446993c578dab05dfdce, scope.project.id=e6e5887740a34024a565b8aae8b2a3e5, user.id=85b48f8334a3449caeae652db3f7ae7f) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:49.264823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56973b-03ee-48c1-9474-db61612b75b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:49.266535 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56973b-03ee-48c1-9474-db61612b75b7 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:49.279739 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 46/96] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:49:48 2022] GET /identity/v3/role_assignments?user.id=85b48f8334a3449caeae652db3f7ae7f&scope.project.id=e6e5887740a34024a565b8aae8b2a3e5&role.id=2fd6a3e7df18446993c578dab05dfdce => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:49.295510 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:49.305379 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f35b7b-7ecf-410c-a958-8ebb44f263b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:49.306456 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f35b7b-7ecf-410c-a958-8ebb44f263b2 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:49.306956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f35b7b-7ecf-410c-a958-8ebb44f263b2 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:49.307430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f35b7b-7ecf-410c-a958-8ebb44f263b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:49.308550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f35b7b-7ecf-410c-a958-8ebb44f263b2 None admin] RBAC: Adding query filter params (role.id=964e02e925f147e69a9e02e02a69612b, scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18, user.id=85b48f8334a3449caeae652db3f7ae7f) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:49.309058 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f35b7b-7ecf-410c-a958-8ebb44f263b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:49.310813 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f35b7b-7ecf-410c-a958-8ebb44f263b2 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:49.317858 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 51/97] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:49:49 2022] GET /identity/v3/role_assignments?user.id=85b48f8334a3449caeae652db3f7ae7f&scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18&role.id=964e02e925f147e69a9e02e02a69612b => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:49:50.974952 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b00e3-9147-4523-9c3d-d9ad7e99b16e None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:50.976326 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b00e3-9147-4523-9c3d-d9ad7e99b16e None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:50.976906 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b00e3-9147-4523-9c3d-d9ad7e99b16e None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:50.978928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 47/98] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:50.984476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f9edc-0802-464b-8f55-84eb99583a01 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:50.985129 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f9edc-0802-464b-8f55-84eb99583a01 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:50.985129 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f9edc-0802-464b-8f55-84eb99583a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:51.003469 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-6e7f9edc-0802-464b-8f55-84eb99583a01 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:51.004712 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-6e7f9edc-0802-464b-8f55-84eb99583a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:51.103836 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 52/99] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:51.141958 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b00e3-9147-4523-9c3d-d9ad7e99b16e None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:51.191505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:51.273428 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b00e3-9147-4523-9c3d-d9ad7e99b16e None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:51.276379 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963927d0-f919-411a-bb6a-01d41b5da1f4 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:51.276925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963927d0-f919-411a-bb6a-01d41b5da1f4 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:51.277421 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963927d0-f919-411a-bb6a-01d41b5da1f4 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:51.279067 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 53/100] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:49:51.284706 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bda41d-c285-4e2a-9331-958ce365ced5 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:51.285286 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bda41d-c285-4e2a-9331-958ce365ced5 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:51.285797 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bda41d-c285-4e2a-9331-958ce365ced5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:51.305398 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-26bda41d-c285-4e2a-9331-958ce365ced5 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:51.307249 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-26bda41d-c285-4e2a-9331-958ce365ced5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:51.335019 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2c8b0-66ed-4b43-ba2d-b9d547457705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:51.335848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2c8b0-66ed-4b43-ba2d-b9d547457705 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:51.336143 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2c8b0-66ed-4b43-ba2d-b9d547457705 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:51.336425 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2c8b0-66ed-4b43-ba2d-b9d547457705 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:51.337677 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec2c8b0-66ed-4b43-ba2d-b9d547457705 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:51.339188 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec2c8b0-66ed-4b43-ba2d-b9d547457705 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:51.340964 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 48/101] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:49:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:51.342243 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe3fb9-2326-4eb2-8161-594ca849eaea None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:51.342559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe3fb9-2326-4eb2-8161-594ca849eaea None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:51.342839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe3fb9-2326-4eb2-8161-594ca849eaea None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:51.344209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 49/102] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:51.349963 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe3fb9-2326-4eb2-8161-594ca849eaea None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:51.435977 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 54/103] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:51.437190 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7eef2-4f30-4657-b963-8106f832037d None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:51.437489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7eef2-4f30-4657-b963-8106f832037d None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:51.437774 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7eef2-4f30-4657-b963-8106f832037d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:51.473827 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-6cf7eef2-4f30-4657-b963-8106f832037d None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:51.475362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-6cf7eef2-4f30-4657-b963-8106f832037d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:51.515536 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe3fb9-2326-4eb2-8161-594ca849eaea None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:51.529596 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:51.569942 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d5486-b486-4d38-ad69-d31765f2c531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:51.570644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d5486-b486-4d38-ad69-d31765f2c531 None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:51.570931 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d5486-b486-4d38-ad69-d31765f2c531 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:51.571207 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d5486-b486-4d38-ad69-d31765f2c531 None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:51.571934 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d5486-b486-4d38-ad69-d31765f2c531 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:51.573447 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d5486-b486-4d38-ad69-d31765f2c531 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:51.576771 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:51.608297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 50/104] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:49:51 2022] POST /identity/v3/projects => generated 302 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:49:51.610442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d5486-b486-4d38-ad69-d31765f2c531 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:51.620768 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 55/105] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:51.656514 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7eef2-4f30-4657-b963-8106f832037d None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:51.784837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7eef2-4f30-4657-b963-8106f832037d None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:51.828628 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d5486-b486-4d38-ad69-d31765f2c531 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:51.840980 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e0be5-fef2-4429-a9ad-b009a1e4bd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:51.842075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e0be5-fef2-4429-a9ad-b009a1e4bd7e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:51.842578 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e0be5-fef2-4429-a9ad-b009a1e4bd7e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:51.843532 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e0be5-fef2-4429-a9ad-b009a1e4bd7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:51.868625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e0be5-fef2-4429-a9ad-b009a1e4bd7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-012e0be5-fef2-4429-a9ad-b009a1e4bd7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:51.882533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:51.883991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:51.885152 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:51.885152 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:51.885152 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:51.885152 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:51.885152 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:51.885152 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:51.885152 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:51.885152 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:51.885152 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:51.885152 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:51.885152 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:49:51.885152 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:49:51.888295 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 56/106] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:49:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:49:51.894848 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e0be5-fef2-4429-a9ad-b009a1e4bd7e None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:51.947236 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9aa726-f415-4f1a-b194-5e296607063d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:51.948091 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9aa726-f415-4f1a-b194-5e296607063d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:51.948380 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9aa726-f415-4f1a-b194-5e296607063d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:51.948665 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9aa726-f415-4f1a-b194-5e296607063d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:51.975495 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9aa726-f415-4f1a-b194-5e296607063d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:51.981537 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-1e9aa726-f415-4f1a-b194-5e296607063d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:51.984156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:51.985193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:51.986232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:51.986232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:51.986232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:51.986232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:51.986232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:51.986232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:51.986232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:51.986232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:51.986232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:51.986232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:51.986232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:49:51.986232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:49:51.986232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 51/107] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:49:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:49:51.994171 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9aa726-f415-4f1a-b194-5e296607063d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:52.036833 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:52.048758 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e0be5-fef2-4429-a9ad-b009a1e4bd7e None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:52.085584 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09eb5dd-83a5-4158-ac5d-7d57bf50b78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:52.086365 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09eb5dd-83a5-4158-ac5d-7d57bf50b78a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:52.086656 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09eb5dd-83a5-4158-ac5d-7d57bf50b78a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:52.086935 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09eb5dd-83a5-4158-ac5d-7d57bf50b78a None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:52.087755 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09eb5dd-83a5-4158-ac5d-7d57bf50b78a None admin] RBAC: Adding query filter params (name=member) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:52.088072 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09eb5dd-83a5-4158-ac5d-7d57bf50b78a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:52.089481 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09eb5dd-83a5-4158-ac5d-7d57bf50b78a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:52.099851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 57/108] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:49:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:52.109080 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09eb5dd-83a5-4158-ac5d-7d57bf50b78a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:52.207768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9aa726-f415-4f1a-b194-5e296607063d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:52.265283 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a7e73-453c-4f4c-ad79-c8ea773b07fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:52.266045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a7e73-453c-4f4c-ad79-c8ea773b07fb None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:52.266366 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a7e73-453c-4f4c-ad79-c8ea773b07fb None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:52.266651 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a7e73-453c-4f4c-ad79-c8ea773b07fb None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:52.267554 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a7e73-453c-4f4c-ad79-c8ea773b07fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:52.267872 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a7e73-453c-4f4c-ad79-c8ea773b07fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:52.269899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a7e73-453c-4f4c-ad79-c8ea773b07fb None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:52.282500 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 52/109] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:49:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:52.284388 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa4ffd-e2c0-4db4-b13b-edb4563c208c None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:52.284688 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa4ffd-e2c0-4db4-b13b-edb4563c208c None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:52.284974 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa4ffd-e2c0-4db4-b13b-edb4563c208c None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:52.285746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09eb5dd-83a5-4158-ac5d-7d57bf50b78a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:52.286604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 53/110] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:52.289766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa4ffd-e2c0-4db4-b13b-edb4563c208c None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:52.291659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:52.360914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e4efb-26f9-4f89-96d6-847dba3c54fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:52.362528 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e4efb-26f9-4f89-96d6-847dba3c54fd None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:52.362820 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e4efb-26f9-4f89-96d6-847dba3c54fd None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:52.363396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e4efb-26f9-4f89-96d6-847dba3c54fd None admin] PATH_INFO: `/v3/users/85b48f8334a3449caeae652db3f7ae7f` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:52.366233 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e4efb-26f9-4f89-96d6-847dba3c54fd None admin] RBAC: Authorizing `identity:get_user(user_id=85b48f8334a3449caeae652db3f7ae7f)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:52.368921 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e4efb-26f9-4f89-96d6-847dba3c54fd None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:52.372167 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 58/111] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:52 2022] GET /identity/v3/users/85b48f8334a3449caeae652db3f7ae7f => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:52.374317 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc05b47-4320-459c-9c8d-fae5f47daf70 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:52.386293 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc05b47-4320-459c-9c8d-fae5f47daf70 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:52.386293 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc05b47-4320-459c-9c8d-fae5f47daf70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:52.407430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-4fc05b47-4320-459c-9c8d-fae5f47daf70 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:52.409085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-4fc05b47-4320-459c-9c8d-fae5f47daf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:52.489224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:52.494047 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 59/112] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:52.494955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc05b47-4320-459c-9c8d-fae5f47daf70 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:52.547109 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa4ffd-e2c0-4db4-b13b-edb4563c208c None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:52.609423 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1694ae-8b8c-4159-bae7-22fab7eee21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:52.610156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc05b47-4320-459c-9c8d-fae5f47daf70 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:52.610361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1694ae-8b8c-4159-bae7-22fab7eee21d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:52.610675 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1694ae-8b8c-4159-bae7-22fab7eee21d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:52.610951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1694ae-8b8c-4159-bae7-22fab7eee21d None admin] PATH_INFO: `/v3/users/d4d97de3ef7545e8b15afe6452141ad3` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:52.612668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:52.626678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1694ae-8b8c-4159-bae7-22fab7eee21d None admin] RBAC: Authorizing `identity:get_user(user_id=d4d97de3ef7545e8b15afe6452141ad3)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:52.628303 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1694ae-8b8c-4159-bae7-22fab7eee21d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:52.630230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 54/113] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:52 2022] GET /identity/v3/users/d4d97de3ef7545e8b15afe6452141ad3 => generated 289 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:49:52.635493 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1694ae-8b8c-4159-bae7-22fab7eee21d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:52.648712 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ad175-7578-45ab-8f02-761685adcd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:52.649604 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ad175-7578-45ab-8f02-761685adcd59 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:52.649898 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ad175-7578-45ab-8f02-761685adcd59 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:52.650185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ad175-7578-45ab-8f02-761685adcd59 None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:52.651423 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ad175-7578-45ab-8f02-761685adcd59 None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:52.652905 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ad175-7578-45ab-8f02-761685adcd59 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:52.655696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 60/114] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:49:52 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:52.656946 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07e1f9-ae2a-49e5-8aad-ef74f27030f7 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:52.657249 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07e1f9-ae2a-49e5-8aad-ef74f27030f7 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:52.657592 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07e1f9-ae2a-49e5-8aad-ef74f27030f7 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:52.664131 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 61/115] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:52.664811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07e1f9-ae2a-49e5-8aad-ef74f27030f7 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:52.759482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1694ae-8b8c-4159-bae7-22fab7eee21d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:52.795300 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07e1f9-ae2a-49e5-8aad-ef74f27030f7 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:52.795904 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f120e75-bda2-4c16-bda1-63c319a4eb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:52.796774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f120e75-bda2-4c16-bda1-63c319a4eb3a None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:52.796998 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f120e75-bda2-4c16-bda1-63c319a4eb3a None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:52.797330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f120e75-bda2-4c16-bda1-63c319a4eb3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:52.805846 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:52.809951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f120e75-bda2-4c16-bda1-63c319a4eb3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-0f120e75-bda2-4c16-bda1-63c319a4eb3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:52.818848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:52.820045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:52.825742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:52.825742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:52.825742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:52.825742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:52.825742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:52.825742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:52.825742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:52.825742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:52.825742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:52.825742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:52.825742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:49:52.825742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:49:52.825742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 55/116] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:49:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:49:52.825742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f120e75-bda2-4c16-bda1-63c319a4eb3a None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:52.840754 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c22be2a-66f0-4762-976f-030d616946e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:52.841532 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22be2a-66f0-4762-976f-030d616946e5 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:52.841822 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22be2a-66f0-4762-976f-030d616946e5 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:52.842100 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22be2a-66f0-4762-976f-030d616946e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:52.884234 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:52.915974 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c22be2a-66f0-4762-976f-030d616946e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:52.917454 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c22be2a-66f0-4762-976f-030d616946e5 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:52.925601 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f120e75-bda2-4c16-bda1-63c319a4eb3a None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:52.962545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8b09d-c852-4728-9b2a-5e11086c53c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:52.963777 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8b09d-c852-4728-9b2a-5e11086c53c0 None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:52.964067 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8b09d-c852-4728-9b2a-5e11086c53c0 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:52.964346 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8b09d-c852-4728-9b2a-5e11086c53c0 None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18/users/85b48f8334a3449caeae652db3f7ae7f/roles/2fd6a3e7df18446993c578dab05dfdce` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:52.966422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8b09d-c852-4728-9b2a-5e11086c53c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1c47666fb654b3e9538d26cdb9bdd18, user_id=85b48f8334a3449caeae652db3f7ae7f, role_id=2fd6a3e7df18446993c578dab05dfdce)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:52.967913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8b09d-c852-4728-9b2a-5e11086c53c0 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:52.994295 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 56/117] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:49:52 2022] PUT /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18/users/85b48f8334a3449caeae652db3f7ae7f/roles/2fd6a3e7df18446993c578dab05dfdce => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:49:52.994295 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd93be-0667-49a8-8ae4-e08d457f247d None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:52.994295 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd93be-0667-49a8-8ae4-e08d457f247d None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:52.994295 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd93be-0667-49a8-8ae4-e08d457f247d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:53.015804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-a2fd93be-0667-49a8-8ae4-e08d457f247d None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:53.017259 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-a2fd93be-0667-49a8-8ae4-e08d457f247d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:53.038246 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 62/118] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:49:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:49:53.039764 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34091af8-53b8-40b9-a893-56193eea0a5d None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:53.040128 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34091af8-53b8-40b9-a893-56193eea0a5d None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:53.040540 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34091af8-53b8-40b9-a893-56193eea0a5d None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:53.042101 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 63/119] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:53.042806 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34091af8-53b8-40b9-a893-56193eea0a5d None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:53.090559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:53.115565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 57/120] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:53.123232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd93be-0667-49a8-8ae4-e08d457f247d None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:53.139065 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:53.183353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34091af8-53b8-40b9-a893-56193eea0a5d None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:53.233546 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d8a49-3ebc-42f7-a745-31e422ce48a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:53.233546 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d8a49-3ebc-42f7-a745-31e422ce48a4 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:53.233546 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d8a49-3ebc-42f7-a745-31e422ce48a4 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:53.233546 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d8a49-3ebc-42f7-a745-31e422ce48a4 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:53.234002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d8a49-3ebc-42f7-a745-31e422ce48a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:53.234406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d8a49-3ebc-42f7-a745-31e422ce48a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:53.238433 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d8a49-3ebc-42f7-a745-31e422ce48a4 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:53.245801 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 64/121] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:49:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:53.247424 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03714666-933c-4fd8-8180-0076b57d35d8 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:53.247726 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03714666-933c-4fd8-8180-0076b57d35d8 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:53.247989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03714666-933c-4fd8-8180-0076b57d35d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:53.265097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd93be-0667-49a8-8ae4-e08d457f247d None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:53.268671 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-03714666-933c-4fd8-8180-0076b57d35d8 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:53.269898 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-03714666-933c-4fd8-8180-0076b57d35d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:53.307952 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75243fb-b6e2-4b69-b632-e9248f43cca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:53.308869 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75243fb-b6e2-4b69-b632-e9248f43cca2 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:53.309158 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75243fb-b6e2-4b69-b632-e9248f43cca2 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:53.309437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75243fb-b6e2-4b69-b632-e9248f43cca2 None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:53.310772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75243fb-b6e2-4b69-b632-e9248f43cca2 None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:53.312405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75243fb-b6e2-4b69-b632-e9248f43cca2 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:53.314722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 58/122] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:49:53 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:53.315543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75243fb-b6e2-4b69-b632-e9248f43cca2 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:53.345264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 65/123] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:53.346499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03714666-933c-4fd8-8180-0076b57d35d8 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:53.379600 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:53.420349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:53.439776 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75243fb-b6e2-4b69-b632-e9248f43cca2 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:53.470619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf77226-3b2c-4c1b-936d-e551145167de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:53.471434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf77226-3b2c-4c1b-936d-e551145167de None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:53.471722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf77226-3b2c-4c1b-936d-e551145167de None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:53.472004 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf77226-3b2c-4c1b-936d-e551145167de None admin] PATH_INFO: `/v3/roles/member` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:53.473412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03714666-933c-4fd8-8180-0076b57d35d8 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:53.483428 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf77226-3b2c-4c1b-936d-e551145167de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-5cf77226-3b2c-4c1b-936d-e551145167de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:53.492039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:53.494367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:53.496481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:53.496481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:53.496481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:53.496481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:53.496481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:53.496481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:53.496481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:53.496481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:53.496481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:53.496481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:53.496481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:49:53.496481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:49:53.496481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 59/124] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:49:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:49:53.496481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf77226-3b2c-4c1b-936d-e551145167de None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:53.524455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a4ba6-c5c0-4162-b04a-b452061684c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:53.525346 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a4ba6-c5c0-4162-b04a-b452061684c3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:53.525630 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a4ba6-c5c0-4162-b04a-b452061684c3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:53.525906 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a4ba6-c5c0-4162-b04a-b452061684c3 None admin] PATH_INFO: `/v3/users/15d8f48fa0c344f1aed458217ad0c20b` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:53.527150 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a4ba6-c5c0-4162-b04a-b452061684c3 None admin] RBAC: Authorizing `identity:get_user(user_id=15d8f48fa0c344f1aed458217ad0c20b)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:53.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a4ba6-c5c0-4162-b04a-b452061684c3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:53.530553 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 66/125] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:53 2022] GET /identity/v3/users/15d8f48fa0c344f1aed458217ad0c20b => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:53.536957 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a4ba6-c5c0-4162-b04a-b452061684c3 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:53.618225 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf77226-3b2c-4c1b-936d-e551145167de None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:53.634814 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:53.651334 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a4ba6-c5c0-4162-b04a-b452061684c3 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:53.654982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca94f60-292e-419d-9384-6d01de38e090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:53.655890 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca94f60-292e-419d-9384-6d01de38e090 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:53.656411 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca94f60-292e-419d-9384-6d01de38e090 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:53.656925 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca94f60-292e-419d-9384-6d01de38e090 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:53.658098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca94f60-292e-419d-9384-6d01de38e090 None admin] RBAC: Adding query filter params (role.id=1e48f1df36584bb08d55fde909d54738, scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18, user.id=d4d97de3ef7545e8b15afe6452141ad3) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:53.658794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca94f60-292e-419d-9384-6d01de38e090 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:53.660515 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca94f60-292e-419d-9384-6d01de38e090 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:53.666948 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 60/126] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:49:53 2022] GET /identity/v3/role_assignments?user.id=d4d97de3ef7545e8b15afe6452141ad3&scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18&role.id=1e48f1df36584bb08d55fde909d54738 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:53.668048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca94f60-292e-419d-9384-6d01de38e090 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:53.708176 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:53.715373 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed372d3-e422-4764-a9a9-ef1b9e65cd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:53.717407 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed372d3-e422-4764-a9a9-ef1b9e65cd64 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:53.717651 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed372d3-e422-4764-a9a9-ef1b9e65cd64 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:53.722177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed372d3-e422-4764-a9a9-ef1b9e65cd64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:53.742377 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed372d3-e422-4764-a9a9-ef1b9e65cd64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-3ed372d3-e422-4764-a9a9-ef1b9e65cd64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:53.760747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:53.761871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:53.762812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:53.762812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:53.762812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:53.762812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:53.762812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:53.762812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:53.762812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:53.762812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:53.762812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:53.762812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:53.762812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:49:53.762812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:49:53.762812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 67/127] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:49:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:49:53.766770 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed372d3-e422-4764-a9a9-ef1b9e65cd64 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:53.886008 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca94f60-292e-419d-9384-6d01de38e090 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:53.945037 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44525991-7351-4d28-a038-6cbda7cdd8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:53.945732 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44525991-7351-4d28-a038-6cbda7cdd8ad None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:53.946017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44525991-7351-4d28-a038-6cbda7cdd8ad None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:53.946317 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44525991-7351-4d28-a038-6cbda7cdd8ad None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:53.947094 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44525991-7351-4d28-a038-6cbda7cdd8ad None admin] RBAC: Adding query filter params (name=member) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:53.947407 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44525991-7351-4d28-a038-6cbda7cdd8ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:53.948826 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44525991-7351-4d28-a038-6cbda7cdd8ad None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:53.958902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 61/128] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:49:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:53.959597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44525991-7351-4d28-a038-6cbda7cdd8ad None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:53.994033 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed372d3-e422-4764-a9a9-ef1b9e65cd64 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:54.014141 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:54.080966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca98ac2-2e8d-4fa9-ab37-837b623adc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:54.082317 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca98ac2-2e8d-4fa9-ab37-837b623adc63 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:54.082865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca98ac2-2e8d-4fa9-ab37-837b623adc63 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:54.083232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44525991-7351-4d28-a038-6cbda7cdd8ad None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:54.083585 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca98ac2-2e8d-4fa9-ab37-837b623adc63 None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:54.087403 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca98ac2-2e8d-4fa9-ab37-837b623adc63 None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:54.089164 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca98ac2-2e8d-4fa9-ab37-837b623adc63 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:54.091230 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 68/129] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:49:53 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:54.092315 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca98ac2-2e8d-4fa9-ab37-837b623adc63 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:54.123243 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f6160-4d4d-4f5e-85bd-f099a4cab43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:54.123929 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f6160-4d4d-4f5e-85bd-f099a4cab43c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:54.124216 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f6160-4d4d-4f5e-85bd-f099a4cab43c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:54.124494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f6160-4d4d-4f5e-85bd-f099a4cab43c None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:54.125257 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f6160-4d4d-4f5e-85bd-f099a4cab43c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:54.125567 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f6160-4d4d-4f5e-85bd-f099a4cab43c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:54.127346 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f6160-4d4d-4f5e-85bd-f099a4cab43c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:54.137110 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 62/130] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:49:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:54.137836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f6160-4d4d-4f5e-85bd-f099a4cab43c None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:54.256972 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f6160-4d4d-4f5e-85bd-f099a4cab43c None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:54.259736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:54.302015 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb43c09-0370-4a28-aac8-27205743be85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:54.303180 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb43c09-0370-4a28-aac8-27205743be85 None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:54.303473 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb43c09-0370-4a28-aac8-27205743be85 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:54.303757 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb43c09-0370-4a28-aac8-27205743be85 None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18/users/15d8f48fa0c344f1aed458217ad0c20b/roles/2cd6d3dd613c4824a26cacf53fb6c07f` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:54.304247 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:54.305940 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb43c09-0370-4a28-aac8-27205743be85 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1c47666fb654b3e9538d26cdb9bdd18, user_id=15d8f48fa0c344f1aed458217ad0c20b, role_id=2cd6d3dd613c4824a26cacf53fb6c07f)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:54.307435 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb43c09-0370-4a28-aac8-27205743be85 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:54.328821 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 63/131] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:49:54 2022] PUT /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18/users/15d8f48fa0c344f1aed458217ad0c20b/roles/2cd6d3dd613c4824a26cacf53fb6c07f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:49:54.329682 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb43c09-0370-4a28-aac8-27205743be85 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:54.406387 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca98ac2-2e8d-4fa9-ab37-837b623adc63 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:54.484370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd3902-c5d9-4f0d-aa24-07b227a83f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:54.485304 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd3902-c5d9-4f0d-aa24-07b227a83f6e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:54.485635 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd3902-c5d9-4f0d-aa24-07b227a83f6e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:54.485953 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd3902-c5d9-4f0d-aa24-07b227a83f6e None admin] PATH_INFO: `/v3/users/85b48f8334a3449caeae652db3f7ae7f` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:54.492204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd3902-c5d9-4f0d-aa24-07b227a83f6e None admin] RBAC: Authorizing `identity:get_user(user_id=85b48f8334a3449caeae652db3f7ae7f)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:54.493749 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd3902-c5d9-4f0d-aa24-07b227a83f6e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:54.496763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 69/132] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:54 2022] GET /identity/v3/users/85b48f8334a3449caeae652db3f7ae7f => generated 275 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:54.503480 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf78f0e-0a8a-453a-92e9-bbf95d8d35be None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:54.503929 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf78f0e-0a8a-453a-92e9-bbf95d8d35be None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:54.506570 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf78f0e-0a8a-453a-92e9-bbf95d8d35be None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:54.511517 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 70/133] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:54 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:54.518705 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf78f0e-0a8a-453a-92e9-bbf95d8d35be None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:54.545989 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb43c09-0370-4a28-aac8-27205743be85 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:54.601722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b19ae-0232-408b-a6d6-11fbf7f97824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:54.602705 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b19ae-0232-408b-a6d6-11fbf7f97824 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:54.603252 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b19ae-0232-408b-a6d6-11fbf7f97824 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:54.603252 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b19ae-0232-408b-a6d6-11fbf7f97824 None admin] PATH_INFO: `/v3/users/85b48f8334a3449caeae652db3f7ae7f` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:54.605000 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b19ae-0232-408b-a6d6-11fbf7f97824 None admin] RBAC: Authorizing `identity:get_user(user_id=85b48f8334a3449caeae652db3f7ae7f)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:54.606539 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b19ae-0232-408b-a6d6-11fbf7f97824 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:54.608395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 64/134] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:54 2022] GET /identity/v3/users/85b48f8334a3449caeae652db3f7ae7f => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:54.609405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a3e44-20bb-466d-86a0-69e07e0bab91 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:54.609699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a3e44-20bb-466d-86a0-69e07e0bab91 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:54.609981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a3e44-20bb-466d-86a0-69e07e0bab91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:54.628862 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-262a3e44-20bb-466d-86a0-69e07e0bab91 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:54.630066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-262a3e44-20bb-466d-86a0-69e07e0bab91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:54.669259 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:54.689847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf78f0e-0a8a-453a-92e9-bbf95d8d35be None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:54.715702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 65/135] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:54.717000 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a3e44-20bb-466d-86a0-69e07e0bab91 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:54.719091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:54.740523 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cea78-68ca-45a1-82cc-f8d159ab33ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:54.741565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cea78-68ca-45a1-82cc-f8d159ab33ae None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:54.741903 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cea78-68ca-45a1-82cc-f8d159ab33ae None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:54.742195 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cea78-68ca-45a1-82cc-f8d159ab33ae None admin] PATH_INFO: `/v3/projects/e6e5887740a34024a565b8aae8b2a3e5` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:54.744832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8cea78-68ca-45a1-82cc-f8d159ab33ae None admin] RBAC: Authorizing `identity:get_project(project_id=e6e5887740a34024a565b8aae8b2a3e5)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:54.747331 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8cea78-68ca-45a1-82cc-f8d159ab33ae None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:54.749140 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 71/136] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:49:54 2022] GET /identity/v3/projects/e6e5887740a34024a565b8aae8b2a3e5 => generated 312 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:54.749913 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cea78-68ca-45a1-82cc-f8d159ab33ae None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:54.924485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cea78-68ca-45a1-82cc-f8d159ab33ae None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:54.932368 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a3e44-20bb-466d-86a0-69e07e0bab91 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:54.961675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d182f7c-5575-4333-8e2d-0741c170152d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:54.962521 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d182f7c-5575-4333-8e2d-0741c170152d None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:54.962811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d182f7c-5575-4333-8e2d-0741c170152d None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:54.963255 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d182f7c-5575-4333-8e2d-0741c170152d None admin] PATH_INFO: `/v3/domains/default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:54.964463 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d182f7c-5575-4333-8e2d-0741c170152d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:54.965982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d182f7c-5575-4333-8e2d-0741c170152d None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:54.967840 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 72/137] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:49:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:54.968582 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d182f7c-5575-4333-8e2d-0741c170152d None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:55.038679 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:55.071029 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b60fc3-6488-490a-9754-7a6eb8acd1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:55.071937 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b60fc3-6488-490a-9754-7a6eb8acd1d2 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:55.072231 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b60fc3-6488-490a-9754-7a6eb8acd1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:55.072514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b60fc3-6488-490a-9754-7a6eb8acd1d2 None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:55.076647 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d182f7c-5575-4333-8e2d-0741c170152d None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:55.079414 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b60fc3-6488-490a-9754-7a6eb8acd1d2 None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:55.081695 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b60fc3-6488-490a-9754-7a6eb8acd1d2 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:55.083914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 66/138] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:49:54 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:55.084903 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b60fc3-6488-490a-9754-7a6eb8acd1d2 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:55.115137 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2cbef-6971-449e-8ea2-24a0ef069c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:55.116129 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2cbef-6971-449e-8ea2-24a0ef069c30 None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:55.116420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2cbef-6971-449e-8ea2-24a0ef069c30 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:55.116704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2cbef-6971-449e-8ea2-24a0ef069c30 None admin] PATH_INFO: `/v3/projects/e6e5887740a34024a565b8aae8b2a3e5/users/85b48f8334a3449caeae652db3f7ae7f/roles/2fd6a3e7df18446993c578dab05dfdce` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:55.118867 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2cbef-6971-449e-8ea2-24a0ef069c30 None admin] RBAC: Authorizing `identity:create_grant(project_id=e6e5887740a34024a565b8aae8b2a3e5, user_id=85b48f8334a3449caeae652db3f7ae7f, role_id=2fd6a3e7df18446993c578dab05dfdce)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:55.120575 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2cbef-6971-449e-8ea2-24a0ef069c30 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:55.141172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 73/139] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:49:54 2022] PUT /identity/v3/projects/e6e5887740a34024a565b8aae8b2a3e5/users/85b48f8334a3449caeae652db3f7ae7f/roles/2fd6a3e7df18446993c578dab05dfdce => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:49:55.142974 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2cbef-6971-449e-8ea2-24a0ef069c30 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:55.159087 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:55.290163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b60fc3-6488-490a-9754-7a6eb8acd1d2 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:55.332571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055145db-c32d-4855-a1ac-460229c73f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:55.333265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055145db-c32d-4855-a1ac-460229c73f2a None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:55.333553 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055145db-c32d-4855-a1ac-460229c73f2a None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:55.333838 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055145db-c32d-4855-a1ac-460229c73f2a None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:55.334608 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055145db-c32d-4855-a1ac-460229c73f2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:55.336862 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055145db-c32d-4855-a1ac-460229c73f2a None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:55.365415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 67/140] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:49:55 2022] POST /identity/v3/users => generated 283 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:49:55.400631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2cbef-6971-449e-8ea2-24a0ef069c30 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:55.427280 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:55.441951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29effb87-b0c7-4dd8-aa03-b099cbb368fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:55.443245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29effb87-b0c7-4dd8-aa03-b099cbb368fb None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:55.443742 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29effb87-b0c7-4dd8-aa03-b099cbb368fb None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:55.444165 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29effb87-b0c7-4dd8-aa03-b099cbb368fb None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18/users/85b48f8334a3449caeae652db3f7ae7f/roles/964e02e925f147e69a9e02e02a69612b` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:55.446646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29effb87-b0c7-4dd8-aa03-b099cbb368fb None admin] RBAC: Authorizing `identity:create_grant(project_id=d1c47666fb654b3e9538d26cdb9bdd18, user_id=85b48f8334a3449caeae652db3f7ae7f, role_id=964e02e925f147e69a9e02e02a69612b)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:55.448305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29effb87-b0c7-4dd8-aa03-b099cbb368fb None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:55.471570 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 74/141] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:49:55 2022] PUT /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18/users/85b48f8334a3449caeae652db3f7ae7f/roles/964e02e925f147e69a9e02e02a69612b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:49:55.775765 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84a9f2-8969-46aa-9583-ae3644123881 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:55.776137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84a9f2-8969-46aa-9583-ae3644123881 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:55.776345 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84a9f2-8969-46aa-9583-ae3644123881 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:55.778184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 68/142] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:55.784504 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7226a-1582-4413-a0d1-f0b6065e97e1 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:55.785016 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7226a-1582-4413-a0d1-f0b6065e97e1 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:55.785126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7226a-1582-4413-a0d1-f0b6065e97e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:55.808010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-78c7226a-1582-4413-a0d1-f0b6065e97e1 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:55.810350 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-78c7226a-1582-4413-a0d1-f0b6065e97e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:55.902856 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 75/143] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:49:55.927256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84a9f2-8969-46aa-9583-ae3644123881 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:56.024764 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:56.042831 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84a9f2-8969-46aa-9583-ae3644123881 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:56.078150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9164e7-2ee6-4c6c-b7d8-281703280371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:56.078989 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9164e7-2ee6-4c6c-b7d8-281703280371 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:56.079321 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9164e7-2ee6-4c6c-b7d8-281703280371 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:56.079701 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9164e7-2ee6-4c6c-b7d8-281703280371 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:56.090477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9164e7-2ee6-4c6c-b7d8-281703280371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-1e9164e7-2ee6-4c6c-b7d8-281703280371 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:56.098625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:56.099644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:56.100548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:56.100548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:56.100548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:56.100548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:56.100548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:56.100548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:56.100548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:56.100548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:56.100548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:56.100548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:56.100548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:49:56.100548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:49:56.100548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 69/144] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:49:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:49:56.105677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7226a-1582-4413-a0d1-f0b6065e97e1 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:56.223232 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7226a-1582-4413-a0d1-f0b6065e97e1 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:56.246388 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:56.277954 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac6962-6e38-4d09-9992-7336ffe66d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:56.278598 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac6962-6e38-4d09-9992-7336ffe66d22 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:56.278898 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac6962-6e38-4d09-9992-7336ffe66d22 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:56.279177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac6962-6e38-4d09-9992-7336ffe66d22 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:56.279951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac6962-6e38-4d09-9992-7336ffe66d22 None admin] RBAC: Adding query filter params (name=member) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:56.280262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac6962-6e38-4d09-9992-7336ffe66d22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:56.281660 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac6962-6e38-4d09-9992-7336ffe66d22 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:56.294406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 76/145] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:49:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:56.300595 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9164e7-2ee6-4c6c-b7d8-281703280371 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:56.526577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9164e7-2ee6-4c6c-b7d8-281703280371 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:56.564515 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408122c-053a-428a-b847-40a3c81a15cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:56.565418 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408122c-053a-428a-b847-40a3c81a15cd None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:56.565699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408122c-053a-428a-b847-40a3c81a15cd None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:56.565972 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408122c-053a-428a-b847-40a3c81a15cd None admin] PATH_INFO: `/v3/users/85b48f8334a3449caeae652db3f7ae7f` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:56.567270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408122c-053a-428a-b847-40a3c81a15cd None admin] RBAC: Authorizing `identity:get_user(user_id=85b48f8334a3449caeae652db3f7ae7f)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:56.568748 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408122c-053a-428a-b847-40a3c81a15cd None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:56.570449 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 70/146] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:56 2022] GET /identity/v3/users/85b48f8334a3449caeae652db3f7ae7f => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:56.576345 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac6962-6e38-4d09-9992-7336ffe66d22 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:56.688012 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac6962-6e38-4d09-9992-7336ffe66d22 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:56.731913 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e559f-8e54-4c87-b025-7876302ab729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:56.732587 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e559f-8e54-4c87-b025-7876302ab729 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:56.732872 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e559f-8e54-4c87-b025-7876302ab729 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:56.733151 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e559f-8e54-4c87-b025-7876302ab729 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:56.751036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7971a8d-fa7c-4780-8e64-528a20621759 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:56.751484 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7971a8d-fa7c-4780-8e64-528a20621759 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:56.751916 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7971a8d-fa7c-4780-8e64-528a20621759 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:56.753633 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 71/147] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:49:56.764573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03562cdb-65e9-4e8b-9560-24b49de0870f None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:56.764875 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03562cdb-65e9-4e8b-9560-24b49de0870f None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:56.765130 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03562cdb-65e9-4e8b-9560-24b49de0870f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:56.788649 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-03562cdb-65e9-4e8b-9560-24b49de0870f None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:56.789865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-03562cdb-65e9-4e8b-9560-24b49de0870f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:56.808755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:56.825467 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:56.866090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e559f-8e54-4c87-b025-7876302ab729 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:56.867633 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e559f-8e54-4c87-b025-7876302ab729 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:56.908027 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 72/148] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:49:56.937107 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03562cdb-65e9-4e8b-9560-24b49de0870f None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:57.076441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 77/149] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:49:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:49:57.083723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e559f-8e54-4c87-b025-7876302ab729 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:57.151868 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03562cdb-65e9-4e8b-9560-24b49de0870f None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:57.241075 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6e2e1-eedc-4565-b6b7-9ba3c9fc97f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:57.242315 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6e2e1-eedc-4565-b6b7-9ba3c9fc97f3 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:57.242844 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6e2e1-eedc-4565-b6b7-9ba3c9fc97f3 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:57.243321 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6e2e1-eedc-4565-b6b7-9ba3c9fc97f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:57.257904 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:57.269634 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6e2e1-eedc-4565-b6b7-9ba3c9fc97f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-60d6e2e1-eedc-4565-b6b7-9ba3c9fc97f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:57.277812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:57.278867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:57.279763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:57.279763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:57.279763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:57.279763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:57.279763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:57.279763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:57.279763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:57.279763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:57.279763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:57.279763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:57.279763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:49:57.279763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:49:57.296608 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 73/150] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:49:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:49:57.304628 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6e2e1-eedc-4565-b6b7-9ba3c9fc97f3 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:57.309448 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:57.327875 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e559f-8e54-4c87-b025-7876302ab729 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:57.423156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8d118-4282-45f7-adda-f7f8c575b53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:57.425350 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8d118-4282-45f7-adda-f7f8c575b53a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:57.426126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8d118-4282-45f7-adda-f7f8c575b53a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:57.427581 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8d118-4282-45f7-adda-f7f8c575b53a None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:57.430842 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8d118-4282-45f7-adda-f7f8c575b53a None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:57.432754 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8d118-4282-45f7-adda-f7f8c575b53a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:57.436806 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 78/151] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:49:57 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:57.445051 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8d118-4282-45f7-adda-f7f8c575b53a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:57.553071 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6e2e1-eedc-4565-b6b7-9ba3c9fc97f3 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:57.613820 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8d118-4282-45f7-adda-f7f8c575b53a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:57.629556 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d5fa4-1937-4f3b-944d-808e55245886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:57.630638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d5fa4-1937-4f3b-944d-808e55245886 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:57.631153 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d5fa4-1937-4f3b-944d-808e55245886 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:57.631650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d5fa4-1937-4f3b-944d-808e55245886 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:57.632616 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d5fa4-1937-4f3b-944d-808e55245886 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:57.633126 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d5fa4-1937-4f3b-944d-808e55245886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:57.634773 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d5fa4-1937-4f3b-944d-808e55245886 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:57.649836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 74/152] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:49:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:57.651526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecce5ae-0f8e-4f05-8971-a6048a99d4fd None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:57.652048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecce5ae-0f8e-4f05-8971-a6048a99d4fd None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:57.652526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecce5ae-0f8e-4f05-8971-a6048a99d4fd None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:57.652646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafd9d6-a62f-4ad4-af23-68e21c32d5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:57.653210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafd9d6-a62f-4ad4-af23-68e21c32d5ba None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:57.653501 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafd9d6-a62f-4ad4-af23-68e21c32d5ba None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:57.653781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafd9d6-a62f-4ad4-af23-68e21c32d5ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:57.654664 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bafd9d6-a62f-4ad4-af23-68e21c32d5ba None admin] RBAC: Adding query filter params (role.id=2fd6a3e7df18446993c578dab05dfdce, scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18, user.id=85b48f8334a3449caeae652db3f7ae7f) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:57.654975 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bafd9d6-a62f-4ad4-af23-68e21c32d5ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:57.655244 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 75/153] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:49:57.656138 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecce5ae-0f8e-4f05-8971-a6048a99d4fd None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:57.656431 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bafd9d6-a62f-4ad4-af23-68e21c32d5ba None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:57.663353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 79/154] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:49:57 2022] GET /identity/v3/role_assignments?user.id=85b48f8334a3449caeae652db3f7ae7f&scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18&role.id=2fd6a3e7df18446993c578dab05dfdce => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:57.664594 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45b20b-3841-48b0-9758-ed5b84932c4e None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:57.664883 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45b20b-3841-48b0-9758-ed5b84932c4e None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:57.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45b20b-3841-48b0-9758-ed5b84932c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:57.682120 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:57.694124 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-4f45b20b-3841-48b0-9758-ed5b84932c4e None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:57.695884 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-4f45b20b-3841-48b0-9758-ed5b84932c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:57.847124 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:57.873203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 80/155] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:49:57.909389 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45b20b-3841-48b0-9758-ed5b84932c4e None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:57.928042 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecce5ae-0f8e-4f05-8971-a6048a99d4fd None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:58.003014 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d100590-07a5-4ce6-8f0a-f61c761243fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:58.004388 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d100590-07a5-4ce6-8f0a-f61c761243fb None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:58.004895 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d100590-07a5-4ce6-8f0a-f61c761243fb None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:58.005382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d100590-07a5-4ce6-8f0a-f61c761243fb None admin] PATH_INFO: `/v3/users/d4d97de3ef7545e8b15afe6452141ad3` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:58.006918 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d100590-07a5-4ce6-8f0a-f61c761243fb None admin] RBAC: Authorizing `identity:get_user(user_id=d4d97de3ef7545e8b15afe6452141ad3)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:58.008601 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d100590-07a5-4ce6-8f0a-f61c761243fb None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:58.010842 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 76/156] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:57 2022] GET /identity/v3/users/d4d97de3ef7545e8b15afe6452141ad3 => generated 289 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:58.015998 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d100590-07a5-4ce6-8f0a-f61c761243fb None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:58.101159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45b20b-3841-48b0-9758-ed5b84932c4e None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:58.145569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d100590-07a5-4ce6-8f0a-f61c761243fb None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:58.156161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9fa3e-9067-431e-9fbc-d759928d6b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:58.156982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9fa3e-9067-431e-9fbc-d759928d6b35 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:58.157268 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9fa3e-9067-431e-9fbc-d759928d6b35 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:58.157611 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9fa3e-9067-431e-9fbc-d759928d6b35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:58.164906 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:58.179827 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9fa3e-9067-431e-9fbc-d759928d6b35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:58.186725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11c284-fea2-47e6-9fe2-5b44e33de7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:58.187609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11c284-fea2-47e6-9fe2-5b44e33de7ff None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:58.187894 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11c284-fea2-47e6-9fe2-5b44e33de7ff None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:58.188272 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11c284-fea2-47e6-9fe2-5b44e33de7ff None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:58.189924 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11c284-fea2-47e6-9fe2-5b44e33de7ff None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:58.191427 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11c284-fea2-47e6-9fe2-5b44e33de7ff None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-1df9fa3e-9067-431e-9fbc-d759928d6b35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:58.191687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:58.192646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:58.193951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:58.193951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:58.193951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:58.193951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:58.193951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:58.193951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:58.193951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:58.193951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:58.193951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:58.193951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:58.193951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:49:58.193951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:49:58.193951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 81/157] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:49:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:49:58.195604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 77/158] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:49:58 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:58.199159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9fa3e-9067-431e-9fbc-d759928d6b35 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:58.200393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11c284-fea2-47e6-9fe2-5b44e33de7ff None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:58.205708 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:58.317563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11c284-fea2-47e6-9fe2-5b44e33de7ff None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:58.357493 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ace7e-2cad-488f-936d-f283907ad5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:58.359511 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ace7e-2cad-488f-936d-f283907ad5bd None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:58.359839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ace7e-2cad-488f-936d-f283907ad5bd None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:58.360189 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ace7e-2cad-488f-936d-f283907ad5bd None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18/users/d4d97de3ef7545e8b15afe6452141ad3/roles/1e48f1df36584bb08d55fde909d54738` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:58.362697 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ace7e-2cad-488f-936d-f283907ad5bd None admin] RBAC: Authorizing `identity:create_grant(project_id=d1c47666fb654b3e9538d26cdb9bdd18, user_id=d4d97de3ef7545e8b15afe6452141ad3, role_id=1e48f1df36584bb08d55fde909d54738)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:58.364386 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ace7e-2cad-488f-936d-f283907ad5bd None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:58.369794 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9fa3e-9067-431e-9fbc-d759928d6b35 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:58.385345 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 78/159] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:49:58 2022] PUT /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18/users/d4d97de3ef7545e8b15afe6452141ad3/roles/1e48f1df36584bb08d55fde909d54738 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:49:58.429977 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb472cc1-6d1e-429e-85ed-99a9b5d930c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:58.430620 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb472cc1-6d1e-429e-85ed-99a9b5d930c8 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:58.430902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb472cc1-6d1e-429e-85ed-99a9b5d930c8 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:58.431176 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb472cc1-6d1e-429e-85ed-99a9b5d930c8 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:58.431960 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb472cc1-6d1e-429e-85ed-99a9b5d930c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:58.432269 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb472cc1-6d1e-429e-85ed-99a9b5d930c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:58.433705 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb472cc1-6d1e-429e-85ed-99a9b5d930c8 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:58.443695 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 82/160] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:49:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:49:58.449391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ace7e-2cad-488f-936d-f283907ad5bd None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:58.550296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:58.598853 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ace7e-2cad-488f-936d-f283907ad5bd None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:58.635822 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7f3bc-abd4-4a3e-9ff2-e8962a0a7040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:58.637039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7f3bc-abd4-4a3e-9ff2-e8962a0a7040 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:58.637584 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7f3bc-abd4-4a3e-9ff2-e8962a0a7040 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:58.638061 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7f3bc-abd4-4a3e-9ff2-e8962a0a7040 None admin] PATH_INFO: `/v3/users/15d8f48fa0c344f1aed458217ad0c20b` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:58.639580 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7f3bc-abd4-4a3e-9ff2-e8962a0a7040 None admin] RBAC: Authorizing `identity:get_user(user_id=15d8f48fa0c344f1aed458217ad0c20b)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:58.641402 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7f3bc-abd4-4a3e-9ff2-e8962a0a7040 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:58.643538 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 79/161] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:58 2022] GET /identity/v3/users/15d8f48fa0c344f1aed458217ad0c20b => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:58.649145 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb472cc1-6d1e-429e-85ed-99a9b5d930c8 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:58.726326 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:58.784596 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb472cc1-6d1e-429e-85ed-99a9b5d930c8 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:58.830169 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c6cd2-a3b9-4217-9859-e52792b83bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:58.830887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c6cd2-a3b9-4217-9859-e52792b83bd0 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:58.831174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c6cd2-a3b9-4217-9859-e52792b83bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:58.831456 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c6cd2-a3b9-4217-9859-e52792b83bd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:58.924127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c6cd2-a3b9-4217-9859-e52792b83bd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:58.925627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c6cd2-a3b9-4217-9859-e52792b83bd0 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:58.997855 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:59.046173 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 83/162] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:49:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:49:59.052268 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7f3bc-abd4-4a3e-9ff2-e8962a0a7040 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:59.132078 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958fffb-0bd1-45fa-ba4b-9b569a98340d None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:59.132410 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958fffb-0bd1-45fa-ba4b-9b569a98340d None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:59.132644 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958fffb-0bd1-45fa-ba4b-9b569a98340d None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:59.134027 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 84/163] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:49:59.139102 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadd62f-d579-4169-a7eb-36c1bd58ba13 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:59.139388 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadd62f-d579-4169-a7eb-36c1bd58ba13 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:59.139668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadd62f-d579-4169-a7eb-36c1bd58ba13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:59.179258 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-1fadd62f-d579-4169-a7eb-36c1bd58ba13 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:59.180465 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-1fadd62f-d579-4169-a7eb-36c1bd58ba13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:49:59.185240 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7f3bc-abd4-4a3e-9ff2-e8962a0a7040 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:59.237220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e7675-c5a7-403e-a65f-fd45c8729bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:59.238938 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e7675-c5a7-403e-a65f-fd45c8729bcd None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:59.239436 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e7675-c5a7-403e-a65f-fd45c8729bcd None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:59.246283 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e7675-c5a7-403e-a65f-fd45c8729bcd None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:59.248252 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e7675-c5a7-403e-a65f-fd45c8729bcd None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:59.249965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e7675-c5a7-403e-a65f-fd45c8729bcd None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:59.252239 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 80/164] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:49:59 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:59.254473 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:59.258009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e7675-c5a7-403e-a65f-fd45c8729bcd None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:59.311715 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 85/165] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:49:59.347360 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadd62f-d579-4169-a7eb-36c1bd58ba13 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:59.409407 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e7675-c5a7-403e-a65f-fd45c8729bcd None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:59.480768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be687f8e-6919-446d-bb31-1d3d4c0bffff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:59.481846 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be687f8e-6919-446d-bb31-1d3d4c0bffff None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:59.482419 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be687f8e-6919-446d-bb31-1d3d4c0bffff None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:59.482934 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be687f8e-6919-446d-bb31-1d3d4c0bffff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:59.484082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be687f8e-6919-446d-bb31-1d3d4c0bffff None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18, user.id=15d8f48fa0c344f1aed458217ad0c20b) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:59.486309 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be687f8e-6919-446d-bb31-1d3d4c0bffff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:59.491637 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be687f8e-6919-446d-bb31-1d3d4c0bffff None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:59.502352 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 81/166] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:49:59 2022] GET /identity/v3/role_assignments?user.id=15d8f48fa0c344f1aed458217ad0c20b&scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:59.525797 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:59.553976 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadd62f-d579-4169-a7eb-36c1bd58ba13 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:59.610250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897ecf5-7d1b-42c2-affa-fe97176d0f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:59.611389 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897ecf5-7d1b-42c2-affa-fe97176d0f1b None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:59.611389 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897ecf5-7d1b-42c2-affa-fe97176d0f1b None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:59.611630 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897ecf5-7d1b-42c2-affa-fe97176d0f1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:59.629576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7897ecf5-7d1b-42c2-affa-fe97176d0f1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-7897ecf5-7d1b-42c2-affa-fe97176d0f1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:49:59.637445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:49:59.638588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:49:59.639543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:49:59.639543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:49:59.639543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:49:59.639543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:49:59.639543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:49:59.639543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:49:59.639543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:49:59.639543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:49:59.639543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:49:59.639543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:49:59.639543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:49:59.639543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:49:59.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 86/167] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:49:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:49:59.650196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be687f8e-6919-446d-bb31-1d3d4c0bffff None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:59.766793 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:59.811945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be687f8e-6919-446d-bb31-1d3d4c0bffff None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:49:59.858319 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ce5fe-f479-4237-9ecd-51bf93eeeea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:49:59.859010 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ce5fe-f479-4237-9ecd-51bf93eeeea6 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:59.859291 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ce5fe-f479-4237-9ecd-51bf93eeeea6 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:59.859559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ce5fe-f479-4237-9ecd-51bf93eeeea6 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:59.860328 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ce5fe-f479-4237-9ecd-51bf93eeeea6 None admin] RBAC: Adding query filter params (name=member) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:49:59.860698 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ce5fe-f479-4237-9ecd-51bf93eeeea6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:49:59.862129 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ce5fe-f479-4237-9ecd-51bf93eeeea6 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:49:59.871957 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 82/168] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:49:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:59.877368 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897ecf5-7d1b-42c2-affa-fe97176d0f1b None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:49:59.885315 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218bdbc2-0ad7-4c4f-b704-e3a242fe3a4f None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:59.885647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218bdbc2-0ad7-4c4f-b704-e3a242fe3a4f None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:59.885889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218bdbc2-0ad7-4c4f-b704-e3a242fe3a4f None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:59.887599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 83/169] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:49:59.900105 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b666e-0296-4fe7-9589-137ab5440242 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:49:59.900456 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b666e-0296-4fe7-9589-137ab5440242 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:49:59.900669 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b666e-0296-4fe7-9589-137ab5440242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:49:59.926460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-c19b666e-0296-4fe7-9589-137ab5440242 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:49:59.927862 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-c19b666e-0296-4fe7-9589-137ab5440242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:00.049132 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 84/170] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:00.056134 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897ecf5-7d1b-42c2-affa-fe97176d0f1b None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:00.091679 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f00c99-34d2-46c1-8609-a319d3ad9633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:00.092971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f00c99-34d2-46c1-8609-a319d3ad9633 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:00.093518 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f00c99-34d2-46c1-8609-a319d3ad9633 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:00.094011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f00c99-34d2-46c1-8609-a319d3ad9633 None admin] PATH_INFO: `/v3/users/85b48f8334a3449caeae652db3f7ae7f` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:00.095642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f00c99-34d2-46c1-8609-a319d3ad9633 None admin] RBAC: Authorizing `identity:get_user(user_id=85b48f8334a3449caeae652db3f7ae7f)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:00.097365 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f00c99-34d2-46c1-8609-a319d3ad9633 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:00.099634 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 87/171] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:49:59 2022] GET /identity/v3/users/85b48f8334a3449caeae652db3f7ae7f => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:00.105197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b666e-0296-4fe7-9589-137ab5440242 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:00.112146 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f00c99-34d2-46c1-8609-a319d3ad9633 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:00.150672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:00.200925 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:00.271924 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b666e-0296-4fe7-9589-137ab5440242 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:00.278687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f00c99-34d2-46c1-8609-a319d3ad9633 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:00.312323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d44a78-f02e-48e8-b2e3-1c9c68f72304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:00.312893 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d44a78-f02e-48e8-b2e3-1c9c68f72304 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:00.313179 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d44a78-f02e-48e8-b2e3-1c9c68f72304 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:00.313457 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d44a78-f02e-48e8-b2e3-1c9c68f72304 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:00.343011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19d150-31e0-4837-a374-dcc7eb7ea348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:00.343857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19d150-31e0-4837-a374-dcc7eb7ea348 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:00.344149 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19d150-31e0-4837-a374-dcc7eb7ea348 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:00.344429 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19d150-31e0-4837-a374-dcc7eb7ea348 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:00.359806 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19d150-31e0-4837-a374-dcc7eb7ea348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-0c19d150-31e0-4837-a374-dcc7eb7ea348 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:00.376956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:00.378193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:00.379131 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:00.379131 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:00.379131 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:00.379131 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:00.379131 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:00.379131 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:00.379131 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:00.379131 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:00.379131 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:00.379131 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:00.379131 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:00.379131 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:00.386713 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 88/172] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:50:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:00.392725 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19d150-31e0-4837-a374-dcc7eb7ea348 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:00.394366 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d44a78-f02e-48e8-b2e3-1c9c68f72304 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:00.396018 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d44a78-f02e-48e8-b2e3-1c9c68f72304 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:00.458666 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:00.506574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 85/173] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:50:00.512012 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d44a78-f02e-48e8-b2e3-1c9c68f72304 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:00.603233 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:00.624649 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d44a78-f02e-48e8-b2e3-1c9c68f72304 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:00.652905 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19d150-31e0-4837-a374-dcc7eb7ea348 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:00.662990 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab3414-91cc-42ae-8b14-713017513d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:00.664225 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab3414-91cc-42ae-8b14-713017513d07 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:00.664723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab3414-91cc-42ae-8b14-713017513d07 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:00.665207 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab3414-91cc-42ae-8b14-713017513d07 None admin] PATH_INFO: `/v3/projects/e6e5887740a34024a565b8aae8b2a3e5` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:00.666692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab3414-91cc-42ae-8b14-713017513d07 None admin] RBAC: Authorizing `identity:get_project(project_id=e6e5887740a34024a565b8aae8b2a3e5)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:00.668505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab3414-91cc-42ae-8b14-713017513d07 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:00.670589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 86/174] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:00 2022] GET /identity/v3/projects/e6e5887740a34024a565b8aae8b2a3e5 => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:00.672179 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0047fd3-60a1-4253-8792-1bf71b6dbf65 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:00.672674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0047fd3-60a1-4253-8792-1bf71b6dbf65 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:00.673347 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0047fd3-60a1-4253-8792-1bf71b6dbf65 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:00.675327 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 87/175] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:00.680953 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbfd55-5ba6-4780-b26a-45dcc7883814 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:00.681517 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbfd55-5ba6-4780-b26a-45dcc7883814 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:00.683019 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbfd55-5ba6-4780-b26a-45dcc7883814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:00.716887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-efdbfd55-5ba6-4780-b26a-45dcc7883814 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:00.718980 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-efdbfd55-5ba6-4780-b26a-45dcc7883814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:00.724614 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c8ca2-08b8-4fb3-9979-8e51380fd50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:00.725305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c8ca2-08b8-4fb3-9979-8e51380fd50d None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:00.725590 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c8ca2-08b8-4fb3-9979-8e51380fd50d None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:00.725868 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c8ca2-08b8-4fb3-9979-8e51380fd50d None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:00.726672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c8ca2-08b8-4fb3-9979-8e51380fd50d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:00.726985 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c8ca2-08b8-4fb3-9979-8e51380fd50d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:00.728383 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c8ca2-08b8-4fb3-9979-8e51380fd50d None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:00.744171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 89/176] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:00.745180 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c8ca2-08b8-4fb3-9979-8e51380fd50d None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:00.755898 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:00.870821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c8ca2-08b8-4fb3-9979-8e51380fd50d None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:00.892023 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 88/177] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:00.892835 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbfd55-5ba6-4780-b26a-45dcc7883814 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:00.921301 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6f2db-e4e6-43ee-9bc3-61856ea05e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:00.922160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6f2db-e4e6-43ee-9bc3-61856ea05e38 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:00.922476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6f2db-e4e6-43ee-9bc3-61856ea05e38 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:00.922766 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6f2db-e4e6-43ee-9bc3-61856ea05e38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:00.923995 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6f2db-e4e6-43ee-9bc3-61856ea05e38 None admin] RBAC: Adding query filter params (role.id=2fd6a3e7df18446993c578dab05dfdce, scope.project.id=e6e5887740a34024a565b8aae8b2a3e5, user.id=85b48f8334a3449caeae652db3f7ae7f) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:00.924517 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6f2db-e4e6-43ee-9bc3-61856ea05e38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:00.926037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6f2db-e4e6-43ee-9bc3-61856ea05e38 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:00.938857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 90/178] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:00 2022] GET /identity/v3/role_assignments?user.id=85b48f8334a3449caeae652db3f7ae7f&scope.project.id=e6e5887740a34024a565b8aae8b2a3e5&role.id=2fd6a3e7df18446993c578dab05dfdce => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:00.943808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6f2db-e4e6-43ee-9bc3-61856ea05e38 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:00.947476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:01.076627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6f2db-e4e6-43ee-9bc3-61856ea05e38 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:01.086768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbfd55-5ba6-4780-b26a-45dcc7883814 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:01.124146 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f982b-8534-4c40-b402-cbe1f21db01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:01.125225 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f982b-8534-4c40-b402-cbe1f21db01a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:01.125668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f982b-8534-4c40-b402-cbe1f21db01a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:01.126182 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f982b-8534-4c40-b402-cbe1f21db01a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:01.141017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8f982b-8534-4c40-b402-cbe1f21db01a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-ac8f982b-8534-4c40-b402-cbe1f21db01a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:01.151767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:01.153090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:01.154279 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:01.154279 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:01.154279 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:01.154279 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:01.154279 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:01.154279 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:01.154279 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:01.154279 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:01.154279 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:01.154279 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:01.154279 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:01.154279 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:01.155445 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d1593-8155-45de-87a0-8caeae585f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:01.156347 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d1593-8155-45de-87a0-8caeae585f4a None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:01.156631 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d1593-8155-45de-87a0-8caeae585f4a None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:01.156910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d1593-8155-45de-87a0-8caeae585f4a None admin] PATH_INFO: `/v3/users/85b48f8334a3449caeae652db3f7ae7f` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:01.157542 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 91/179] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:50:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:01.158215 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d1593-8155-45de-87a0-8caeae585f4a None admin] RBAC: Authorizing `identity:get_user(user_id=85b48f8334a3449caeae652db3f7ae7f)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:01.162466 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d1593-8155-45de-87a0-8caeae585f4a None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:01.162466 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 89/180] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:00 2022] GET /identity/v3/users/85b48f8334a3449caeae652db3f7ae7f => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:01.163108 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed1f03-eba5-4129-94ea-8e9eff1bc888 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:01.163382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed1f03-eba5-4129-94ea-8e9eff1bc888 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:01.163660 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed1f03-eba5-4129-94ea-8e9eff1bc888 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:01.164422 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5dbc6-c25a-4123-962a-c5470ff31a5b None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:01.164882 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5dbc6-c25a-4123-962a-c5470ff31a5b None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:01.164993 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 90/181] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:01.165394 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5dbc6-c25a-4123-962a-c5470ff31a5b None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:01.167241 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 92/182] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:01.169091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5dbc6-c25a-4123-962a-c5470ff31a5b None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:01.169235 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed1f03-eba5-4129-94ea-8e9eff1bc888 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:01.197244 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:01.265116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:01.317944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed1f03-eba5-4129-94ea-8e9eff1bc888 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:01.325059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5dbc6-c25a-4123-962a-c5470ff31a5b None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:01.354902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a3378-a12f-461a-8d9b-dbb150bef957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:01.355636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a3378-a12f-461a-8d9b-dbb150bef957 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:01.355927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a3378-a12f-461a-8d9b-dbb150bef957 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:01.356357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a3378-a12f-461a-8d9b-dbb150bef957 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:01.357250 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a3378-a12f-461a-8d9b-dbb150bef957 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:01.357631 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a3378-a12f-461a-8d9b-dbb150bef957 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:01.359044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a3378-a12f-461a-8d9b-dbb150bef957 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:01.361072 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b3c80-0188-4cbc-a41b-23cc94cb656e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:01.361731 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b3c80-0188-4cbc-a41b-23cc94cb656e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:01.362017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b3c80-0188-4cbc-a41b-23cc94cb656e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:01.362317 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b3c80-0188-4cbc-a41b-23cc94cb656e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:01.373783 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 91/183] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:01.373783 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1535fad-3c12-4eb9-98a5-c8e8f0be04eb None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:01.373783 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1535fad-3c12-4eb9-98a5-c8e8f0be04eb None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:01.373783 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1535fad-3c12-4eb9-98a5-c8e8f0be04eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:01.389706 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-a1535fad-3c12-4eb9-98a5-c8e8f0be04eb None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:01.390848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-a1535fad-3c12-4eb9-98a5-c8e8f0be04eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:01.433373 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b3c80-0188-4cbc-a41b-23cc94cb656e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:01.435259 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b3c80-0188-4cbc-a41b-23cc94cb656e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:01.460296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 92/184] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:01.461426 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2da68b-6a8c-4af6-bc08-6157d28336b3 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:01.461722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2da68b-6a8c-4af6-bc08-6157d28336b3 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:01.462002 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2da68b-6a8c-4af6-bc08-6157d28336b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:01.479339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-bd2da68b-6a8c-4af6-bc08-6157d28336b3 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:01.480341 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-bd2da68b-6a8c-4af6-bc08-6157d28336b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:01.488792 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:01.523426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:01.566456 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 93/185] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:01.566456 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2da68b-6a8c-4af6-bc08-6157d28336b3 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:01.570894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 93/186] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:01.570894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b3c80-0188-4cbc-a41b-23cc94cb656e None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:01.681311 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2da68b-6a8c-4af6-bc08-6157d28336b3 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:01.689875 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b3c80-0188-4cbc-a41b-23cc94cb656e None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:01.727602 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6cb33-4187-4a15-8460-0aca42c4be0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:01.728058 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac797c68-0e84-4c4b-8b33-9698885844de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:01.728499 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6cb33-4187-4a15-8460-0aca42c4be0e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:01.728786 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6cb33-4187-4a15-8460-0aca42c4be0e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:01.729063 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6cb33-4187-4a15-8460-0aca42c4be0e None admin] PATH_INFO: `/v3/users/5f7cc5d676544ed6999cd3ca0e0f0782` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:01.729768 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac797c68-0e84-4c4b-8b33-9698885844de None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:01.730253 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac797c68-0e84-4c4b-8b33-9698885844de None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:01.730768 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac797c68-0e84-4c4b-8b33-9698885844de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:01.741327 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:01.742747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac797c68-0e84-4c4b-8b33-9698885844de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:01.744786 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6cb33-4187-4a15-8460-0aca42c4be0e None admin] RBAC: Authorizing `identity:get_user(user_id=5f7cc5d676544ed6999cd3ca0e0f0782)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:01.746774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6cb33-4187-4a15-8460-0aca42c4be0e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:01.748695 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 94/187] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:01 2022] GET /identity/v3/users/5f7cc5d676544ed6999cd3ca0e0f0782 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-ac797c68-0e84-4c4b-8b33-9698885844de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:01.750902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:01.751961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:01.752870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:01.752870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:01.752870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:01.752870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:01.752870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:01.752870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:01.752870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:01.752870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:01.752870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:01.752870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:01.752870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:01.752870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:01.752870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 94/188] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:50:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:01.753912 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac797c68-0e84-4c4b-8b33-9698885844de None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:01.759647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6cb33-4187-4a15-8460-0aca42c4be0e None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:01.784351 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:01.863887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6cb33-4187-4a15-8460-0aca42c4be0e None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:01.866726 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac797c68-0e84-4c4b-8b33-9698885844de None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:01.899219 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd59436-3dd7-4d2e-96d2-2119730aab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:01.900097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd59436-3dd7-4d2e-96d2-2119730aab78 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:01.900377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd59436-3dd7-4d2e-96d2-2119730aab78 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:01.900686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd59436-3dd7-4d2e-96d2-2119730aab78 None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:01.901815 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd59436-3dd7-4d2e-96d2-2119730aab78 None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:01.903328 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd59436-3dd7-4d2e-96d2-2119730aab78 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:01.903557 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee3f68-ddc8-4ec3-82f3-9cc07e00b51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:01.904356 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee3f68-ddc8-4ec3-82f3-9cc07e00b51b None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:01.904640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee3f68-ddc8-4ec3-82f3-9cc07e00b51b None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:01.904916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee3f68-ddc8-4ec3-82f3-9cc07e00b51b None admin] PATH_INFO: `/v3/domains/default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:01.905018 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 95/189] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:01 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:01.905773 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd59436-3dd7-4d2e-96d2-2119730aab78 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:01.911288 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee3f68-ddc8-4ec3-82f3-9cc07e00b51b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:01.912806 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee3f68-ddc8-4ec3-82f3-9cc07e00b51b None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:01.914642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 95/190] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:01.919119 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee3f68-ddc8-4ec3-82f3-9cc07e00b51b None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:01.983580 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:02.016298 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:02.025612 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd59436-3dd7-4d2e-96d2-2119730aab78 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:02.028568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee3f68-ddc8-4ec3-82f3-9cc07e00b51b None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:02.062792 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100e2f8-2a2b-43e7-892f-c37752db8f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:02.063595 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100e2f8-2a2b-43e7-892f-c37752db8f68 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:02.063882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100e2f8-2a2b-43e7-892f-c37752db8f68 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:02.064159 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100e2f8-2a2b-43e7-892f-c37752db8f68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:02.065203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887035a-2d6a-419b-bfac-50e288837683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:02.065888 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887035a-2d6a-419b-bfac-50e288837683 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:02.066177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887035a-2d6a-419b-bfac-50e288837683 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:02.066486 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887035a-2d6a-419b-bfac-50e288837683 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:02.067245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887035a-2d6a-419b-bfac-50e288837683 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:02.068285 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887035a-2d6a-419b-bfac-50e288837683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:02.068954 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887035a-2d6a-419b-bfac-50e288837683 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:02.077806 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 96/191] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:02.078503 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887035a-2d6a-419b-bfac-50e288837683 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:02.146045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100e2f8-2a2b-43e7-892f-c37752db8f68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:02.148075 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100e2f8-2a2b-43e7-892f-c37752db8f68 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:02.182892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887035a-2d6a-419b-bfac-50e288837683 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:02.214437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:02.216151 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf6178-88c7-4cc9-a84c-c7bef75fc9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:02.216813 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf6178-88c7-4cc9-a84c-c7bef75fc9e1 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:02.217127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf6178-88c7-4cc9-a84c-c7bef75fc9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:02.217403 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf6178-88c7-4cc9-a84c-c7bef75fc9e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:02.218251 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf6178-88c7-4cc9-a84c-c7bef75fc9e1 None admin] RBAC: Adding query filter params (role.id=964e02e925f147e69a9e02e02a69612b, scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18, user.id=85b48f8334a3449caeae652db3f7ae7f) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:02.218596 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf6178-88c7-4cc9-a84c-c7bef75fc9e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:02.220097 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf6178-88c7-4cc9-a84c-c7bef75fc9e1 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:02.226542 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 97/192] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:02 2022] GET /identity/v3/role_assignments?user.id=85b48f8334a3449caeae652db3f7ae7f&scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18&role.id=964e02e925f147e69a9e02e02a69612b => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:02.227306 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf6178-88c7-4cc9-a84c-c7bef75fc9e1 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:02.246719 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:02.256589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 96/193] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:02.260103 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100e2f8-2a2b-43e7-892f-c37752db8f68 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:02.345151 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf6178-88c7-4cc9-a84c-c7bef75fc9e1 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:02.375667 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100e2f8-2a2b-43e7-892f-c37752db8f68 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:02.386810 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834449f1-6e4f-4981-ab51-fc148c80bdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:02.387507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834449f1-6e4f-4981-ab51-fc148c80bdbc None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:02.387794 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834449f1-6e4f-4981-ab51-fc148c80bdbc None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:02.388273 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834449f1-6e4f-4981-ab51-fc148c80bdbc None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:02.388810 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834449f1-6e4f-4981-ab51-fc148c80bdbc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:02.390343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834449f1-6e4f-4981-ab51-fc148c80bdbc None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:02.415254 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed839eb-5b62-49c6-a379-c6225f4654c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:02.416565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed839eb-5b62-49c6-a379-c6225f4654c5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:02.416850 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed839eb-5b62-49c6-a379-c6225f4654c5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:02.417274 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed839eb-5b62-49c6-a379-c6225f4654c5 None admin] PATH_INFO: `/v3/users/5f7cc5d676544ed6999cd3ca0e0f0782` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:02.418415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed839eb-5b62-49c6-a379-c6225f4654c5 None admin] RBAC: Authorizing `identity:get_user(user_id=5f7cc5d676544ed6999cd3ca0e0f0782)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:02.420657 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed839eb-5b62-49c6-a379-c6225f4654c5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:02.426369 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 98/194] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:50:02 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:50:02.426592 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 97/195] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:02 2022] GET /identity/v3/users/5f7cc5d676544ed6999cd3ca0e0f0782 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:02.426684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834449f1-6e4f-4981-ab51-fc148c80bdbc None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:02.426781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ba9b8-5505-4064-a5e7-c5142194f080 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:02.426781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ba9b8-5505-4064-a5e7-c5142194f080 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:02.426781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ba9b8-5505-4064-a5e7-c5142194f080 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:02.430457 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 98/196] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:02.431482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ba9b8-5505-4064-a5e7-c5142194f080 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:02.476743 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:02.529492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:02.566054 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ba9b8-5505-4064-a5e7-c5142194f080 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:02.569863 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834449f1-6e4f-4981-ab51-fc148c80bdbc None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:02.605316 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c322e-181f-449b-beaa-f49c74e79105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:02.605994 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c322e-181f-449b-beaa-f49c74e79105 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:02.606307 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c322e-181f-449b-beaa-f49c74e79105 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:02.606583 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c322e-181f-449b-beaa-f49c74e79105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:02.612223 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4494da-763d-46d8-9b41-9a62d09cd594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:02.613117 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4494da-763d-46d8-9b41-9a62d09cd594 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:02.613401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4494da-763d-46d8-9b41-9a62d09cd594 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:02.613674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4494da-763d-46d8-9b41-9a62d09cd594 None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:02.614973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4494da-763d-46d8-9b41-9a62d09cd594 None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:02.616928 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4494da-763d-46d8-9b41-9a62d09cd594 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:02.619082 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 99/197] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:02 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:02.620359 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0087e-57be-4728-a4e8-d389eca68fa5 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:02.620647 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0087e-57be-4728-a4e8-d389eca68fa5 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:02.620927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0087e-57be-4728-a4e8-d389eca68fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:02.638813 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-40c0087e-57be-4728-a4e8-d389eca68fa5 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:02.640030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-40c0087e-57be-4728-a4e8-d389eca68fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:02.691705 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c322e-181f-449b-beaa-f49c74e79105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:02.693279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c322e-181f-449b-beaa-f49c74e79105 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:02.742360 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 100/198] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:02.742360 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0087e-57be-4728-a4e8-d389eca68fa5 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:02.754738 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:02.821583 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:02.852911 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 99/199] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:02.854044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c322e-181f-449b-beaa-f49c74e79105 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:02.861647 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0087e-57be-4728-a4e8-d389eca68fa5 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:02.905478 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d0f1b-d9dc-4473-b978-b12f1760cdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:02.906158 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d0f1b-d9dc-4473-b978-b12f1760cdca None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:02.906479 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d0f1b-d9dc-4473-b978-b12f1760cdca None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:02.907270 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d0f1b-d9dc-4473-b978-b12f1760cdca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:02.908152 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d0f1b-d9dc-4473-b978-b12f1760cdca None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=c06a45678b434f629eb93e70012c1738, user.id=5f7cc5d676544ed6999cd3ca0e0f0782) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:02.908474 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d0f1b-d9dc-4473-b978-b12f1760cdca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:02.910830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d0f1b-d9dc-4473-b978-b12f1760cdca None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:02.918644 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 101/200] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:02 2022] GET /identity/v3/role_assignments?user.id=5f7cc5d676544ed6999cd3ca0e0f0782&scope.project.id=c06a45678b434f629eb93e70012c1738&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:02.919473 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d0f1b-d9dc-4473-b978-b12f1760cdca None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:03.005782 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c322e-181f-449b-beaa-f49c74e79105 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:03.067052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df504c-f817-4305-9a1d-9e3b6c935ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:03.067898 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df504c-f817-4305-9a1d-9e3b6c935ff3 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:03.068186 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df504c-f817-4305-9a1d-9e3b6c935ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:03.068470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df504c-f817-4305-9a1d-9e3b6c935ff3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:03.083800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:03.105042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d0f1b-d9dc-4473-b978-b12f1760cdca None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:03.111370 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df504c-f817-4305-9a1d-9e3b6c935ff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-88df504c-f817-4305-9a1d-9e3b6c935ff3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:03.120220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:03.121415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:03.122529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:03.122529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:03.122529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:03.122529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:03.122529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:03.122529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:03.122529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:03.122529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:03.122529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:03.122529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:03.122529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:50:03.122529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:03.125706 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 100/201] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:03.132790 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df504c-f817-4305-9a1d-9e3b6c935ff3 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:03.168744 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41116723-8332-474a-b920-50b7e6157827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:03.169403 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41116723-8332-474a-b920-50b7e6157827 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:03.169696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41116723-8332-474a-b920-50b7e6157827 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:03.169973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41116723-8332-474a-b920-50b7e6157827 None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:03.171362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41116723-8332-474a-b920-50b7e6157827 None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:03.173197 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41116723-8332-474a-b920-50b7e6157827 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:03.175317 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 102/202] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:02 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:03.184458 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41116723-8332-474a-b920-50b7e6157827 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:03.200304 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:03.326589 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41116723-8332-474a-b920-50b7e6157827 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:03.340623 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df504c-f817-4305-9a1d-9e3b6c935ff3 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:03.370166 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189b149-63df-4379-a0d3-7090c293ef4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:03.370869 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189b149-63df-4379-a0d3-7090c293ef4e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:03.371298 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189b149-63df-4379-a0d3-7090c293ef4e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:03.371784 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189b149-63df-4379-a0d3-7090c293ef4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:03.373080 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189b149-63df-4379-a0d3-7090c293ef4e None admin] RBAC: Adding query filter params (role.id=964e02e925f147e69a9e02e02a69612b, scope.project.id=c06a45678b434f629eb93e70012c1738, user.id=5f7cc5d676544ed6999cd3ca0e0f0782) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:03.373400 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189b149-63df-4379-a0d3-7090c293ef4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:03.374927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189b149-63df-4379-a0d3-7090c293ef4e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:03.382038 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 103/203] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:03 2022] GET /identity/v3/role_assignments?user.id=5f7cc5d676544ed6999cd3ca0e0f0782&scope.project.id=c06a45678b434f629eb93e70012c1738&role.id=964e02e925f147e69a9e02e02a69612b => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:03.436287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188eb3f-9eb6-4fe3-a3ca-8289b828558e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:03.437090 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188eb3f-9eb6-4fe3-a3ca-8289b828558e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:03.437278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188eb3f-9eb6-4fe3-a3ca-8289b828558e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:03.437536 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188eb3f-9eb6-4fe3-a3ca-8289b828558e None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:03.438417 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188eb3f-9eb6-4fe3-a3ca-8289b828558e None admin] RBAC: Adding query filter params (name=reader) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:03.438733 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188eb3f-9eb6-4fe3-a3ca-8289b828558e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:03.440401 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188eb3f-9eb6-4fe3-a3ca-8289b828558e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:03.462551 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 101/204] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:03.468875 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189b149-63df-4379-a0d3-7090c293ef4e None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:03.561476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:03.638580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189b149-63df-4379-a0d3-7090c293ef4e None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:03.696828 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e88c9-7a57-475a-b952-24ef972b4188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:03.697720 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e88c9-7a57-475a-b952-24ef972b4188 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:03.698001 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e88c9-7a57-475a-b952-24ef972b4188 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:03.698303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e88c9-7a57-475a-b952-24ef972b4188 None admin] PATH_INFO: `/v3/users/d4d97de3ef7545e8b15afe6452141ad3` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:03.700071 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e88c9-7a57-475a-b952-24ef972b4188 None admin] RBAC: Authorizing `identity:get_user(user_id=d4d97de3ef7545e8b15afe6452141ad3)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:03.701592 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e88c9-7a57-475a-b952-24ef972b4188 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:03.705196 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 104/205] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:03 2022] GET /identity/v3/users/d4d97de3ef7545e8b15afe6452141ad3 => generated 289 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:03.709514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188eb3f-9eb6-4fe3-a3ca-8289b828558e None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:03.846756 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:03.919441 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188eb3f-9eb6-4fe3-a3ca-8289b828558e None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:03.973732 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870fbe50-12a2-434d-b8e4-f0fd4e8de3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:03.974842 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870fbe50-12a2-434d-b8e4-f0fd4e8de3dd None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:03.975337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870fbe50-12a2-434d-b8e4-f0fd4e8de3dd None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:03.975672 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870fbe50-12a2-434d-b8e4-f0fd4e8de3dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:04.098731 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870fbe50-12a2-434d-b8e4-f0fd4e8de3dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:04.100242 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870fbe50-12a2-434d-b8e4-f0fd4e8de3dd None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:04.183146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:04.234408 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 102/206] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:04.243375 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e88c9-7a57-475a-b952-24ef972b4188 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:04.379232 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e88c9-7a57-475a-b952-24ef972b4188 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:04.428759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7347cb-3705-4915-bc80-2e7a358dd82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:04.429652 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7347cb-3705-4915-bc80-2e7a358dd82c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:04.429939 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7347cb-3705-4915-bc80-2e7a358dd82c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:04.430215 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7347cb-3705-4915-bc80-2e7a358dd82c None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:04.431458 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7347cb-3705-4915-bc80-2e7a358dd82c None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:04.432923 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7347cb-3705-4915-bc80-2e7a358dd82c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:04.434732 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 105/207] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:04 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:04.440918 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870fbe50-12a2-434d-b8e4-f0fd4e8de3dd None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:04.556550 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870fbe50-12a2-434d-b8e4-f0fd4e8de3dd None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:04.591536 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0eb8a8-17ae-4a91-91e8-803c7804c410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:04.592206 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0eb8a8-17ae-4a91-91e8-803c7804c410 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:04.592489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0eb8a8-17ae-4a91-91e8-803c7804c410 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:04.592765 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0eb8a8-17ae-4a91-91e8-803c7804c410 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:04.593617 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0eb8a8-17ae-4a91-91e8-803c7804c410 None admin] RBAC: Adding query filter params (role.id=1e48f1df36584bb08d55fde909d54738, scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18, user.id=d4d97de3ef7545e8b15afe6452141ad3) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:04.593932 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0eb8a8-17ae-4a91-91e8-803c7804c410 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:04.595581 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0eb8a8-17ae-4a91-91e8-803c7804c410 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:04.602499 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 103/208] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:04 2022] GET /identity/v3/role_assignments?user.id=d4d97de3ef7545e8b15afe6452141ad3&scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18&role.id=1e48f1df36584bb08d55fde909d54738 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:05.391034 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0da460-b05a-4bed-8e43-052eb0107114 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:05.391610 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0da460-b05a-4bed-8e43-052eb0107114 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:05.391610 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0da460-b05a-4bed-8e43-052eb0107114 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:05.393327 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 106/209] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:05.399866 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b054c-cbbe-4cc2-b011-274bdaefab2a None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:05.400169 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b054c-cbbe-4cc2-b011-274bdaefab2a None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:05.400420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b054c-cbbe-4cc2-b011-274bdaefab2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:05.419789 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-f15b054c-cbbe-4cc2-b011-274bdaefab2a None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:05.421009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-f15b054c-cbbe-4cc2-b011-274bdaefab2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:05.438736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:05.496692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0417b3-5f21-42ea-bd85-7e141b974ae3 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:05.497084 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0417b3-5f21-42ea-bd85-7e141b974ae3 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:05.497210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 104/210] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:05.497648 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0417b3-5f21-42ea-bd85-7e141b974ae3 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:05.499722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 107/211] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:05.506108 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e2ce4-6938-4d93-9fc5-be3f8db7bb41 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:05.506450 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e2ce4-6938-4d93-9fc5-be3f8db7bb41 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:05.506734 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e2ce4-6938-4d93-9fc5-be3f8db7bb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:05.524412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-423e2ce4-6938-4d93-9fc5-be3f8db7bb41 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:05.525328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0417b3-5f21-42ea-bd85-7e141b974ae3 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:05.525661 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-423e2ce4-6938-4d93-9fc5-be3f8db7bb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:05.528668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:05.599889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 105/212] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:05.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e2ce4-6938-4d93-9fc5-be3f8db7bb41 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:05.654247 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0417b3-5f21-42ea-bd85-7e141b974ae3 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:05.697405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c889a3-34fd-4f97-bdcd-207f760a19d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:05.698353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c889a3-34fd-4f97-bdcd-207f760a19d1 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:05.698640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c889a3-34fd-4f97-bdcd-207f760a19d1 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:05.699251 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c889a3-34fd-4f97-bdcd-207f760a19d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:05.700101 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c889a3-34fd-4f97-bdcd-207f760a19d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:05.701586 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c889a3-34fd-4f97-bdcd-207f760a19d1 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:05.705525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 108/213] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:05.705525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcf51a-07c7-48bb-a746-6152900e8d0a None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:05.705525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcf51a-07c7-48bb-a746-6152900e8d0a None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:05.705525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcf51a-07c7-48bb-a746-6152900e8d0a None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:05.706788 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 109/214] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:05.711893 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfcf51a-07c7-48bb-a746-6152900e8d0a None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:05.725348 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:05.746134 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e2ce4-6938-4d93-9fc5-be3f8db7bb41 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:05.787319 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847514f2-adb3-4c95-95c4-6123dc2aaeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:05.788149 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847514f2-adb3-4c95-95c4-6123dc2aaeed None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:05.788432 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847514f2-adb3-4c95-95c4-6123dc2aaeed None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:05.788707 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847514f2-adb3-4c95-95c4-6123dc2aaeed None admin] PATH_INFO: `/v3/roles/member` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:05.793866 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:05.802611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847514f2-adb3-4c95-95c4-6123dc2aaeed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-847514f2-adb3-4c95-95c4-6123dc2aaeed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:05.811125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:05.812170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:05.813150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:05.813150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:05.813150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:05.813150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:05.813150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:05.813150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:05.813150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:05.813150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:05.813150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:05.813150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:05.813150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:05.813150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:05.813150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 106/215] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:05.813964 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfdcdc-48ac-4d2d-8ee4-342b0b3521a8 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:05.813964 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfdcdc-48ac-4d2d-8ee4-342b0b3521a8 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:05.814947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfdcdc-48ac-4d2d-8ee4-342b0b3521a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:05.832174 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-bbdfdcdc-48ac-4d2d-8ee4-342b0b3521a8 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:05.833367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-bbdfdcdc-48ac-4d2d-8ee4-342b0b3521a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:05.835681 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfcf51a-07c7-48bb-a746-6152900e8d0a None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:05.872831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa1f26-8825-4d48-80bd-c95042e7fc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:05.873524 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa1f26-8825-4d48-80bd-c95042e7fc6f None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:05.873807 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa1f26-8825-4d48-80bd-c95042e7fc6f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:05.874080 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa1f26-8825-4d48-80bd-c95042e7fc6f None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:05.875254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa1f26-8825-4d48-80bd-c95042e7fc6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:05.876917 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa1f26-8825-4d48-80bd-c95042e7fc6f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:05.899951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 110/216] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:50:05 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:50:05.899951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa1f26-8825-4d48-80bd-c95042e7fc6f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:05.917309 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 107/217] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:05.940822 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfdcdc-48ac-4d2d-8ee4-342b0b3521a8 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:06.017975 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:06.035095 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa1f26-8825-4d48-80bd-c95042e7fc6f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:06.052860 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfdcdc-48ac-4d2d-8ee4-342b0b3521a8 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:06.063693 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:06.074976 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ce32e-6a6c-44bc-a594-a404c25d0c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:06.075744 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ce32e-6a6c-44bc-a594-a404c25d0c83 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:06.076070 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ce32e-6a6c-44bc-a594-a404c25d0c83 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:06.076351 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ce32e-6a6c-44bc-a594-a404c25d0c83 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:06.077124 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ce32e-6a6c-44bc-a594-a404c25d0c83 None admin] RBAC: Adding query filter params (name=member) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:06.077431 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ce32e-6a6c-44bc-a594-a404c25d0c83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:06.078947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ce32e-6a6c-44bc-a594-a404c25d0c83 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:06.088836 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 111/218] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:06.090003 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce258c88-cc9f-42ea-b1a3-131af61dbe1f None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:06.090314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce258c88-cc9f-42ea-b1a3-131af61dbe1f None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:06.090581 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce258c88-cc9f-42ea-b1a3-131af61dbe1f None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:06.092112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 112/219] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:06.094739 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47cbf7-9e0b-48e4-a3b5-439a5526f527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:06.095572 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47cbf7-9e0b-48e4-a3b5-439a5526f527 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:06.095857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47cbf7-9e0b-48e4-a3b5-439a5526f527 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:06.096894 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47cbf7-9e0b-48e4-a3b5-439a5526f527 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:06.098375 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce258c88-cc9f-42ea-b1a3-131af61dbe1f None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:06.108937 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47cbf7-9e0b-48e4-a3b5-439a5526f527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-8a47cbf7-9e0b-48e4-a3b5-439a5526f527 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:06.117017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:06.118076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:06.119060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:06.119060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:06.119060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:06.119060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:06.119060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:06.119060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:06.119060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:06.119060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:06.119060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:06.119060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:06.119060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:06.119060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:06.119747 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 108/220] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:50:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:06.120527 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6cd65-d2b7-41de-b06a-cd164abe7424 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:06.120649 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6cd65-d2b7-41de-b06a-cd164abe7424 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:06.120887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6cd65-d2b7-41de-b06a-cd164abe7424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:06.139925 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-65a6cd65-d2b7-41de-b06a-cd164abe7424 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:06.141131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-65a6cd65-d2b7-41de-b06a-cd164abe7424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:06.220245 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 109/221] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:06.221275 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6cd65-d2b7-41de-b06a-cd164abe7424 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:06.226159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce258c88-cc9f-42ea-b1a3-131af61dbe1f None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:06.263720 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b19d8-f485-4192-af4b-3e96a3c08507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:06.264619 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b19d8-f485-4192-af4b-3e96a3c08507 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:06.264902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b19d8-f485-4192-af4b-3e96a3c08507 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:06.265180 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b19d8-f485-4192-af4b-3e96a3c08507 None admin] PATH_INFO: `/v3/users/8cf3a976f1e5478d8d453091853dd877` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:06.282743 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b19d8-f485-4192-af4b-3e96a3c08507 None admin] RBAC: Authorizing `identity:get_user(user_id=8cf3a976f1e5478d8d453091853dd877)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:06.284554 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b19d8-f485-4192-af4b-3e96a3c08507 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:06.286402 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 113/222] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:06 2022] GET /identity/v3/users/8cf3a976f1e5478d8d453091853dd877 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:06.287166 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b19d8-f485-4192-af4b-3e96a3c08507 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:06.296268 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:06.340294 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6cd65-d2b7-41de-b06a-cd164abe7424 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:06.352200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:06.379134 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfcd24e-6878-49e9-b425-22f68535fd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:06.380012 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcd24e-6878-49e9-b425-22f68535fd48 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:06.380240 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcd24e-6878-49e9-b425-22f68535fd48 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:06.380467 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcd24e-6878-49e9-b425-22f68535fd48 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:06.381200 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfcd24e-6878-49e9-b425-22f68535fd48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:06.381510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfcd24e-6878-49e9-b425-22f68535fd48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:06.382937 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfcd24e-6878-49e9-b425-22f68535fd48 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:06.392532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 110/223] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:06.393370 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfcd24e-6878-49e9-b425-22f68535fd48 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:06.423867 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b19d8-f485-4192-af4b-3e96a3c08507 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:06.467821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5970181-94eb-44e0-a128-e857eb037b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:06.468949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5970181-94eb-44e0-a128-e857eb037b35 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:06.469520 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5970181-94eb-44e0-a128-e857eb037b35 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:06.471074 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5970181-94eb-44e0-a128-e857eb037b35 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:06.489672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5970181-94eb-44e0-a128-e857eb037b35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-e5970181-94eb-44e0-a128-e857eb037b35 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:06.502010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:06.503096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:06.504015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:06.504015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:06.504015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:06.504015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:06.504015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:06.504015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:06.504015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:06.504015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:06.504015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:06.504015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:06.504015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:06.504015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:06.504701 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfcd24e-6878-49e9-b425-22f68535fd48 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:06.508533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 114/224] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:50:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:06.509659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5970181-94eb-44e0-a128-e857eb037b35 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:06.533039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:06.548201 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c54073-5fae-4051-817f-7fb4ce61b69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:06.549407 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c54073-5fae-4051-817f-7fb4ce61b69c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:06.550655 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c54073-5fae-4051-817f-7fb4ce61b69c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:06.550934 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c54073-5fae-4051-817f-7fb4ce61b69c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:06.624607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5970181-94eb-44e0-a128-e857eb037b35 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:06.633065 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:06.639894 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c54073-5fae-4051-817f-7fb4ce61b69c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:06.642043 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c54073-5fae-4051-817f-7fb4ce61b69c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:06.663997 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1b4fd-50a7-4fc9-ba6d-bc3b09cfed7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:06.664997 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1b4fd-50a7-4fc9-ba6d-bc3b09cfed7a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:06.665282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1b4fd-50a7-4fc9-ba6d-bc3b09cfed7a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:06.665560 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1b4fd-50a7-4fc9-ba6d-bc3b09cfed7a None admin] PATH_INFO: `/v3/users/5f7cc5d676544ed6999cd3ca0e0f0782` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:06.666911 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1b4fd-50a7-4fc9-ba6d-bc3b09cfed7a None admin] RBAC: Authorizing `identity:get_user(user_id=5f7cc5d676544ed6999cd3ca0e0f0782)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:06.668397 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1b4fd-50a7-4fc9-ba6d-bc3b09cfed7a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:06.670493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 115/225] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:06 2022] GET /identity/v3/users/5f7cc5d676544ed6999cd3ca0e0f0782 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:06.671392 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1b4fd-50a7-4fc9-ba6d-bc3b09cfed7a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:06.772832 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 111/226] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:06.774193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c54073-5fae-4051-817f-7fb4ce61b69c None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:06.806542 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:06.809006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1b4fd-50a7-4fc9-ba6d-bc3b09cfed7a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:06.848807 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124a85b-c3fb-45a5-ac36-82bea2e47970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:06.849892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124a85b-c3fb-45a5-ac36-82bea2e47970 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:06.850453 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124a85b-c3fb-45a5-ac36-82bea2e47970 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:06.850939 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124a85b-c3fb-45a5-ac36-82bea2e47970 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:06.851913 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124a85b-c3fb-45a5-ac36-82bea2e47970 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:06.852451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124a85b-c3fb-45a5-ac36-82bea2e47970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:06.854052 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124a85b-c3fb-45a5-ac36-82bea2e47970 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:06.862796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 116/227] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:06.864033 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124a85b-c3fb-45a5-ac36-82bea2e47970 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:06.893926 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c54073-5fae-4051-817f-7fb4ce61b69c None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:06.900722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:06.934538 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e374b-cc6e-4af9-a42a-734ffde3558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:06.935130 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e374b-cc6e-4af9-a42a-734ffde3558b None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:06.935412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e374b-cc6e-4af9-a42a-734ffde3558b None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:06.935685 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e374b-cc6e-4af9-a42a-734ffde3558b None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:06.936911 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e374b-cc6e-4af9-a42a-734ffde3558b None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:06.938883 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e374b-cc6e-4af9-a42a-734ffde3558b None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:06.941167 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 112/228] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:06 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:06.942146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e374b-cc6e-4af9-a42a-734ffde3558b None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:06.971888 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124a85b-c3fb-45a5-ac36-82bea2e47970 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:07.007362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a53002-e24b-4869-b015-a21e6673835e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:07.008647 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a53002-e24b-4869-b015-a21e6673835e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:07.009160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a53002-e24b-4869-b015-a21e6673835e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:07.009656 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a53002-e24b-4869-b015-a21e6673835e None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:07.011088 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a53002-e24b-4869-b015-a21e6673835e None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:07.012789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a53002-e24b-4869-b015-a21e6673835e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:07.015134 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 117/229] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:06 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:07.016083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a53002-e24b-4869-b015-a21e6673835e None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:07.047692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:07.058255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e374b-cc6e-4af9-a42a-734ffde3558b None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:07.095737 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdfd10-7ced-48cc-94ef-7319256d4ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:07.096692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdfd10-7ced-48cc-94ef-7319256d4ef9 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:07.096977 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdfd10-7ced-48cc-94ef-7319256d4ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:07.097252 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdfd10-7ced-48cc-94ef-7319256d4ef9 None admin] PATH_INFO: `/v3/users/5f7cc5d676544ed6999cd3ca0e0f0782` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:07.098853 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdfd10-7ced-48cc-94ef-7319256d4ef9 None admin] RBAC: Authorizing `identity:get_user(user_id=5f7cc5d676544ed6999cd3ca0e0f0782)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:07.100335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdfd10-7ced-48cc-94ef-7319256d4ef9 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:07.102532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 113/230] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:06 2022] GET /identity/v3/users/5f7cc5d676544ed6999cd3ca0e0f0782 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:07.103584 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdfd10-7ced-48cc-94ef-7319256d4ef9 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:07.129433 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a53002-e24b-4869-b015-a21e6673835e None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:07.132538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:07.167233 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32300355-47e0-4301-881a-e947206b5247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:07.168146 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32300355-47e0-4301-881a-e947206b5247 None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:07.168429 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32300355-47e0-4301-881a-e947206b5247 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:07.168704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32300355-47e0-4301-881a-e947206b5247 None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738/users/5f7cc5d676544ed6999cd3ca0e0f0782/roles/2cd6d3dd613c4824a26cacf53fb6c07f` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:07.170919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32300355-47e0-4301-881a-e947206b5247 None admin] RBAC: Authorizing `identity:create_grant(project_id=c06a45678b434f629eb93e70012c1738, user_id=5f7cc5d676544ed6999cd3ca0e0f0782, role_id=2cd6d3dd613c4824a26cacf53fb6c07f)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:07.172398 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32300355-47e0-4301-881a-e947206b5247 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:07.189306 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 118/231] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:50:07 2022] PUT /identity/v3/projects/c06a45678b434f629eb93e70012c1738/users/5f7cc5d676544ed6999cd3ca0e0f0782/roles/2cd6d3dd613c4824a26cacf53fb6c07f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:07.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32300355-47e0-4301-881a-e947206b5247 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:07.218146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdfd10-7ced-48cc-94ef-7319256d4ef9 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:07.261591 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab724af5-25d4-415b-9f4d-b27cd05ed469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:07.262835 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab724af5-25d4-415b-9f4d-b27cd05ed469 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:07.263346 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab724af5-25d4-415b-9f4d-b27cd05ed469 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:07.263837 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab724af5-25d4-415b-9f4d-b27cd05ed469 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:07.265225 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab724af5-25d4-415b-9f4d-b27cd05ed469 None admin] RBAC: Adding query filter params (role.id=2fd6a3e7df18446993c578dab05dfdce, scope.project.id=c06a45678b434f629eb93e70012c1738, user.id=8cf3a976f1e5478d8d453091853dd877) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:07.265838 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab724af5-25d4-415b-9f4d-b27cd05ed469 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:07.267824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab724af5-25d4-415b-9f4d-b27cd05ed469 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:07.275289 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 114/232] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:07 2022] GET /identity/v3/role_assignments?user.id=8cf3a976f1e5478d8d453091853dd877&scope.project.id=c06a45678b434f629eb93e70012c1738&role.id=2fd6a3e7df18446993c578dab05dfdce => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:07.310158 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32300355-47e0-4301-881a-e947206b5247 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:07.349518 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c9919-5fec-4873-84af-c30075cbede7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:07.350452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c9919-5fec-4873-84af-c30075cbede7 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:07.350738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c9919-5fec-4873-84af-c30075cbede7 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:07.351013 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c9919-5fec-4873-84af-c30075cbede7 None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:07.352258 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c9919-5fec-4873-84af-c30075cbede7 None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:07.353779 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c9919-5fec-4873-84af-c30075cbede7 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:07.355832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 119/233] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:07 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:07.361883 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab724af5-25d4-415b-9f4d-b27cd05ed469 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:07.421696 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:07.503922 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab724af5-25d4-415b-9f4d-b27cd05ed469 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:07.545574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc67cc5-2931-4c94-ae3a-f3bfc02648f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:07.547432 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc67cc5-2931-4c94-ae3a-f3bfc02648f8 None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:07.548096 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc67cc5-2931-4c94-ae3a-f3bfc02648f8 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:07.548670 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc67cc5-2931-4c94-ae3a-f3bfc02648f8 None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738/users/5f7cc5d676544ed6999cd3ca0e0f0782/roles/964e02e925f147e69a9e02e02a69612b` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:07.551601 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc67cc5-2931-4c94-ae3a-f3bfc02648f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c06a45678b434f629eb93e70012c1738, user_id=5f7cc5d676544ed6999cd3ca0e0f0782, role_id=964e02e925f147e69a9e02e02a69612b)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:07.553379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc67cc5-2931-4c94-ae3a-f3bfc02648f8 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:07.572614 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 115/234] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:50:07 2022] PUT /identity/v3/projects/c06a45678b434f629eb93e70012c1738/users/5f7cc5d676544ed6999cd3ca0e0f0782/roles/964e02e925f147e69a9e02e02a69612b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:08.536092 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381076bc-2a3e-4f29-b205-376ed93b47cf None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:08.537162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381076bc-2a3e-4f29-b205-376ed93b47cf None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:08.537700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381076bc-2a3e-4f29-b205-376ed93b47cf None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:08.539650 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 120/235] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:08.545192 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e8c91-3bbc-493d-bf40-f5c3197a4a44 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:08.545757 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e8c91-3bbc-493d-bf40-f5c3197a4a44 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:08.546664 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e8c91-3bbc-493d-bf40-f5c3197a4a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:08.568213 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-452e8c91-3bbc-493d-bf40-f5c3197a4a44 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:08.569777 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-452e8c91-3bbc-493d-bf40-f5c3197a4a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:08.578147 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead8981-69c5-49ed-a6ed-41812946797e None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:08.578752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead8981-69c5-49ed-a6ed-41812946797e None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:08.579277 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead8981-69c5-49ed-a6ed-41812946797e None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:08.580906 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 121/236] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:50:08.586621 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77d00e-551b-4304-86a0-7046aa632762 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:08.587216 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77d00e-551b-4304-86a0-7046aa632762 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:08.587738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77d00e-551b-4304-86a0-7046aa632762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:08.619495 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-4b77d00e-551b-4304-86a0-7046aa632762 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:08.621623 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-4b77d00e-551b-4304-86a0-7046aa632762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:08.678631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:08.709921 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 116/237] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:08.736451 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e8c91-3bbc-493d-bf40-f5c3197a4a44 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:08.755611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:08.764920 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 122/238] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:08.805089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77d00e-551b-4304-86a0-7046aa632762 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:08.842463 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e8c91-3bbc-493d-bf40-f5c3197a4a44 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:08.879513 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460166ed-0755-4e86-a199-cb8effa6901f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:08.880700 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460166ed-0755-4e86-a199-cb8effa6901f None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:08.881300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460166ed-0755-4e86-a199-cb8effa6901f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:08.881794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460166ed-0755-4e86-a199-cb8effa6901f None admin] PATH_INFO: `/v3/domains/default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:08.883217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460166ed-0755-4e86-a199-cb8effa6901f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:08.885045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460166ed-0755-4e86-a199-cb8effa6901f None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:08.887570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 117/239] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:08.893631 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460166ed-0755-4e86-a199-cb8effa6901f None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:08.994760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:09.004768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460166ed-0755-4e86-a199-cb8effa6901f None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:09.047471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3912e6-799e-4750-b249-85a36f0bb166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:09.048552 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3912e6-799e-4750-b249-85a36f0bb166 None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:09.049066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3912e6-799e-4750-b249-85a36f0bb166 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:09.049541 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3912e6-799e-4750-b249-85a36f0bb166 None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:09.050493 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3912e6-799e-4750-b249-85a36f0bb166 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:09.052152 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3912e6-799e-4750-b249-85a36f0bb166 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:09.089841 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 118/240] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:50:08 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:50:09.094398 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77d00e-551b-4304-86a0-7046aa632762 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:09.146477 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134bc2e4-63f9-440c-9109-d96335d8d2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:09.147675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bc2e4-63f9-440c-9109-d96335d8d2e5 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:09.148167 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bc2e4-63f9-440c-9109-d96335d8d2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:09.148652 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bc2e4-63f9-440c-9109-d96335d8d2e5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:09.161162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134bc2e4-63f9-440c-9109-d96335d8d2e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-134bc2e4-63f9-440c-9109-d96335d8d2e5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:09.169700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:09.170824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:09.171752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:09.171752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:09.171752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:09.171752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:09.171752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:09.171752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:09.171752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:09.171752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:09.171752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:09.171752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:09.171752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:50:09.171752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:09.174782 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 123/241] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:09.180403 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3912e6-799e-4750-b249-85a36f0bb166 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:09.328789 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3912e6-799e-4750-b249-85a36f0bb166 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:09.375792 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac0851-5160-4cf0-81e5-ef95aaa4883d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:09.376838 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac0851-5160-4cf0-81e5-ef95aaa4883d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:09.377343 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac0851-5160-4cf0-81e5-ef95aaa4883d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:09.377823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac0851-5160-4cf0-81e5-ef95aaa4883d None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:09.378843 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac0851-5160-4cf0-81e5-ef95aaa4883d None admin] RBAC: Adding query filter params (name=reader) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:09.379378 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac0851-5160-4cf0-81e5-ef95aaa4883d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:09.380992 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac0851-5160-4cf0-81e5-ef95aaa4883d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:09.386855 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:09.393807 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 119/242] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:09.401369 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134bc2e4-63f9-440c-9109-d96335d8d2e5 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:09.429068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:09.511137 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134bc2e4-63f9-440c-9109-d96335d8d2e5 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:09.552659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e9b45-f76b-4ed1-8510-83b67a5dd250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:09.553561 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e9b45-f76b-4ed1-8510-83b67a5dd250 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:09.553845 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e9b45-f76b-4ed1-8510-83b67a5dd250 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:09.554123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e9b45-f76b-4ed1-8510-83b67a5dd250 None admin] PATH_INFO: `/v3/users/f1af6f751696447fa5fab4ac872cf8df` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:09.569401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e9b45-f76b-4ed1-8510-83b67a5dd250 None admin] RBAC: Authorizing `identity:get_user(user_id=f1af6f751696447fa5fab4ac872cf8df)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:09.570951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e9b45-f76b-4ed1-8510-83b67a5dd250 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:09.577270 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 124/243] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:09 2022] GET /identity/v3/users/f1af6f751696447fa5fab4ac872cf8df => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:09.578761 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac0851-5160-4cf0-81e5-ef95aaa4883d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:09.722900 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac0851-5160-4cf0-81e5-ef95aaa4883d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:09.763637 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d793c19-05ff-4518-801a-5dbb1a40e111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:09.764308 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d793c19-05ff-4518-801a-5dbb1a40e111 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:09.764592 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d793c19-05ff-4518-801a-5dbb1a40e111 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:09.764870 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d793c19-05ff-4518-801a-5dbb1a40e111 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:09.835889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:09.838474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d793c19-05ff-4518-801a-5dbb1a40e111 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:09.840331 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d793c19-05ff-4518-801a-5dbb1a40e111 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:09.960540 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 120/244] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:09.965961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e9b45-f76b-4ed1-8510-83b67a5dd250 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:10.055314 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4101ac-6ee9-46ab-96e0-4ca76bfc0257 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:10.055674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4101ac-6ee9-46ab-96e0-4ca76bfc0257 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:10.055870 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4101ac-6ee9-46ab-96e0-4ca76bfc0257 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:10.057356 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 121/245] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:10.059419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:10.062940 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf0924-283d-4cc2-9c93-36352a482743 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:10.063664 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf0924-283d-4cc2-9c93-36352a482743 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:10.063792 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf0924-283d-4cc2-9c93-36352a482743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:10.082004 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e9b45-f76b-4ed1-8510-83b67a5dd250 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:10.082968 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-6ddf0924-283d-4cc2-9c93-36352a482743 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:10.084288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-6ddf0924-283d-4cc2-9c93-36352a482743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:10.118587 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd947b3-4cb2-4f0d-a3c6-038fa9ae3cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:10.119855 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd947b3-4cb2-4f0d-a3c6-038fa9ae3cba None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:10.120349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd947b3-4cb2-4f0d-a3c6-038fa9ae3cba None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:10.120873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd947b3-4cb2-4f0d-a3c6-038fa9ae3cba None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:10.122312 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd947b3-4cb2-4f0d-a3c6-038fa9ae3cba None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:10.124011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd947b3-4cb2-4f0d-a3c6-038fa9ae3cba None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:10.126429 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 125/246] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:09 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:10.135538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd947b3-4cb2-4f0d-a3c6-038fa9ae3cba None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:10.160407 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 122/247] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:10.205081 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf0924-283d-4cc2-9c93-36352a482743 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:10.250568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd947b3-4cb2-4f0d-a3c6-038fa9ae3cba None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:10.270674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:10.302894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73e14d-6c31-47e9-b195-5cf9c8ff2f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:10.304138 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73e14d-6c31-47e9-b195-5cf9c8ff2f27 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:10.305831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73e14d-6c31-47e9-b195-5cf9c8ff2f27 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:10.306428 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73e14d-6c31-47e9-b195-5cf9c8ff2f27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:10.307628 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73e14d-6c31-47e9-b195-5cf9c8ff2f27 None admin] RBAC: Adding query filter params (role.id=1e48f1df36584bb08d55fde909d54738, scope.project.id=c06a45678b434f629eb93e70012c1738, user.id=f1af6f751696447fa5fab4ac872cf8df) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:10.309039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73e14d-6c31-47e9-b195-5cf9c8ff2f27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:10.313438 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73e14d-6c31-47e9-b195-5cf9c8ff2f27 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:10.325469 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 126/248] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:10 2022] GET /identity/v3/role_assignments?user.id=f1af6f751696447fa5fab4ac872cf8df&scope.project.id=c06a45678b434f629eb93e70012c1738&role.id=1e48f1df36584bb08d55fde909d54738 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:10.327780 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689408b5-bfa6-43d7-add5-70b1fbfb25ca None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:10.328629 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689408b5-bfa6-43d7-add5-70b1fbfb25ca None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:10.329999 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689408b5-bfa6-43d7-add5-70b1fbfb25ca None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:10.331952 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 127/249] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:10.339727 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e98581-9c0f-40a9-9a85-578f53df5f9f None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:10.340461 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e98581-9c0f-40a9-9a85-578f53df5f9f None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:10.340966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e98581-9c0f-40a9-9a85-578f53df5f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:10.371294 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-33e98581-9c0f-40a9-9a85-578f53df5f9f None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:10.373556 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-33e98581-9c0f-40a9-9a85-578f53df5f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:10.391690 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:10.415138 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf0924-283d-4cc2-9c93-36352a482743 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:10.499281 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de4c4a-a8c6-4622-804f-406092f114df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:10.500526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de4c4a-a8c6-4622-804f-406092f114df None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:10.502581 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de4c4a-a8c6-4622-804f-406092f114df None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:10.503103 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de4c4a-a8c6-4622-804f-406092f114df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:10.526141 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de4c4a-a8c6-4622-804f-406092f114df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:10.527856 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 128/250] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-23de4c4a-a8c6-4622-804f-406092f114df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:10.536298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:10.537397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:10.540420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:10.540420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:10.540420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:10.540420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:10.540420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:10.540420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:10.540420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:10.540420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:10.540420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:10.540420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:10.540420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:10.540420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:10.541202 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f2708-bead-4f7e-8e66-2f1f07cdc959 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:10.541202 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f2708-bead-4f7e-8e66-2f1f07cdc959 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:10.541202 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f2708-bead-4f7e-8e66-2f1f07cdc959 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:10.541202 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 129/251] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:10.543569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 123/252] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:50:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:10.550313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa886a-d27f-4e52-a853-8036137c092e None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:10.550313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa886a-d27f-4e52-a853-8036137c092e None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:10.550313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa886a-d27f-4e52-a853-8036137c092e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:10.551355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de4c4a-a8c6-4622-804f-406092f114df None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:10.580606 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-aaaa886a-d27f-4e52-a853-8036137c092e None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:10.583250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-aaaa886a-d27f-4e52-a853-8036137c092e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:10.659321 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 130/253] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:10.660068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa886a-d27f-4e52-a853-8036137c092e None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:10.697017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:10.709275 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de4c4a-a8c6-4622-804f-406092f114df None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:10.779754 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f9c39-f416-4ca6-8d6f-224572954d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:10.780348 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f9c39-f416-4ca6-8d6f-224572954d64 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:10.780632 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f9c39-f416-4ca6-8d6f-224572954d64 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:10.780904 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f9c39-f416-4ca6-8d6f-224572954d64 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:10.781671 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f9c39-f416-4ca6-8d6f-224572954d64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:10.782116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f9c39-f416-4ca6-8d6f-224572954d64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:10.783722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f9c39-f416-4ca6-8d6f-224572954d64 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:10.801826 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 124/254] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:10.802542 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f9c39-f416-4ca6-8d6f-224572954d64 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:10.816652 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:10.870941 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa886a-d27f-4e52-a853-8036137c092e None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:10.934165 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8470448-8345-4357-af74-ab81c3c1a02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:10.935000 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8470448-8345-4357-af74-ab81c3c1a02f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:10.935279 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8470448-8345-4357-af74-ab81c3c1a02f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:10.935553 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8470448-8345-4357-af74-ab81c3c1a02f None admin] PATH_INFO: `/v3/roles/member` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:10.946586 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8470448-8345-4357-af74-ab81c3c1a02f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-e8470448-8345-4357-af74-ab81c3c1a02f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:10.954672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:10.955688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:10.956631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:10.956631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:10.956631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:10.956631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:10.956631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:10.956631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:10.956631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:10.956631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:10.956631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:10.956631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:10.956631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:10.956631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:10.956631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 131/255] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:10.957375 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8470448-8345-4357-af74-ab81c3c1a02f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:10.982499 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f9c39-f416-4ca6-8d6f-224572954d64 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:11.017101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e90bfb-ee4c-4c9b-99cb-14435ecb22a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:11.017862 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e90bfb-ee4c-4c9b-99cb-14435ecb22a8 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:11.018151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e90bfb-ee4c-4c9b-99cb-14435ecb22a8 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:11.018464 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e90bfb-ee4c-4c9b-99cb-14435ecb22a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:11.029524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e90bfb-ee4c-4c9b-99cb-14435ecb22a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-c9e90bfb-ee4c-4c9b-99cb-14435ecb22a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:11.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:11.045564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:11.046534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:11.046534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:11.046534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:11.046534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:11.046534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:11.046534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:11.046534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:11.046534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:11.046534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:11.046534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:11.046534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:11.046534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:11.046534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 125/256] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:50:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:11.047282 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e90bfb-ee4c-4c9b-99cb-14435ecb22a8 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:11.061234 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8470448-8345-4357-af74-ab81c3c1a02f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:11.096310 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88318cab-51f6-4e64-968c-318e68ff329c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:11.096991 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:11.097198 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88318cab-51f6-4e64-968c-318e68ff329c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:11.097801 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88318cab-51f6-4e64-968c-318e68ff329c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:11.098075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88318cab-51f6-4e64-968c-318e68ff329c None admin] PATH_INFO: `/v3/users/5f7cc5d676544ed6999cd3ca0e0f0782` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:11.099888 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88318cab-51f6-4e64-968c-318e68ff329c None admin] RBAC: Authorizing `identity:get_user(user_id=5f7cc5d676544ed6999cd3ca0e0f0782)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:11.101468 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88318cab-51f6-4e64-968c-318e68ff329c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:11.103427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 132/257] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:10 2022] GET /identity/v3/users/5f7cc5d676544ed6999cd3ca0e0f0782 => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:11.104834 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88318cab-51f6-4e64-968c-318e68ff329c None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:11.108736 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:11.208360 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88318cab-51f6-4e64-968c-318e68ff329c None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:11.249094 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603ec9a-4005-4f19-9fb7-7d6a0411861e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:11.249775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603ec9a-4005-4f19-9fb7-7d6a0411861e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:11.250294 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603ec9a-4005-4f19-9fb7-7d6a0411861e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:11.250573 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603ec9a-4005-4f19-9fb7-7d6a0411861e None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:11.251385 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603ec9a-4005-4f19-9fb7-7d6a0411861e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:11.251685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603ec9a-4005-4f19-9fb7-7d6a0411861e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:11.253140 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603ec9a-4005-4f19-9fb7-7d6a0411861e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:11.273494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 133/258] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:11.274172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603ec9a-4005-4f19-9fb7-7d6a0411861e None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:11.293908 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e90bfb-ee4c-4c9b-99cb-14435ecb22a8 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:11.358096 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cdb4e-ece6-4317-9de6-7936551db195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:11.358736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cdb4e-ece6-4317-9de6-7936551db195 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:11.359019 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cdb4e-ece6-4317-9de6-7936551db195 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:11.359292 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cdb4e-ece6-4317-9de6-7936551db195 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:11.360117 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cdb4e-ece6-4317-9de6-7936551db195 None admin] RBAC: Adding query filter params (name=member) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:11.360450 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cdb4e-ece6-4317-9de6-7936551db195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:11.366309 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cdb4e-ece6-4317-9de6-7936551db195 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:11.372856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 126/259] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:11.373533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cdb4e-ece6-4317-9de6-7936551db195 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:11.406648 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:11.453081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603ec9a-4005-4f19-9fb7-7d6a0411861e None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:11.489110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d61869-b3d2-4e8d-8581-42e463185503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:11.489861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d61869-b3d2-4e8d-8581-42e463185503 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:11.490146 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d61869-b3d2-4e8d-8581-42e463185503 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:11.490552 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d61869-b3d2-4e8d-8581-42e463185503 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:11.504304 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:11.553430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cdb4e-ece6-4317-9de6-7936551db195 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:11.569297 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d61869-b3d2-4e8d-8581-42e463185503 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:11.570901 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d61869-b3d2-4e8d-8581-42e463185503 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:11.627104 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08fb7f-111f-4b83-b3b7-e6106558e30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:11.627933 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08fb7f-111f-4b83-b3b7-e6106558e30b None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:11.628222 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08fb7f-111f-4b83-b3b7-e6106558e30b None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:11.628505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08fb7f-111f-4b83-b3b7-e6106558e30b None admin] PATH_INFO: `/v3/users/5f7cc5d676544ed6999cd3ca0e0f0782` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:11.629776 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08fb7f-111f-4b83-b3b7-e6106558e30b None admin] RBAC: Authorizing `identity:get_user(user_id=5f7cc5d676544ed6999cd3ca0e0f0782)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:11.631290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08fb7f-111f-4b83-b3b7-e6106558e30b None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:11.634315 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 127/260] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:11 2022] GET /identity/v3/users/5f7cc5d676544ed6999cd3ca0e0f0782 => generated 283 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:11.634653 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:11.635095 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08fb7f-111f-4b83-b3b7-e6106558e30b None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:11.686746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 134/261] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:11.687660 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d61869-b3d2-4e8d-8581-42e463185503 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:11.829133 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d61869-b3d2-4e8d-8581-42e463185503 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:11.845183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08fb7f-111f-4b83-b3b7-e6106558e30b None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:11.882798 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9976c-e1a2-4aa1-99ed-28d58ff809ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:11.882798 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9976c-e1a2-4aa1-99ed-28d58ff809ea None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:11.883073 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9976c-e1a2-4aa1-99ed-28d58ff809ea None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:11.883296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9976c-e1a2-4aa1-99ed-28d58ff809ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:11.910007 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33e68b-d9be-4b8c-923e-3218880e6c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:11.910923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33e68b-d9be-4b8c-923e-3218880e6c45 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:11.911204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33e68b-d9be-4b8c-923e-3218880e6c45 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:11.911477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33e68b-d9be-4b8c-923e-3218880e6c45 None admin] PATH_INFO: `/v3/users/8cf3a976f1e5478d8d453091853dd877` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:11.916967 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33e68b-d9be-4b8c-923e-3218880e6c45 None admin] RBAC: Authorizing `identity:get_user(user_id=8cf3a976f1e5478d8d453091853dd877)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:11.918494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33e68b-d9be-4b8c-923e-3218880e6c45 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:11.920503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 128/262] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:11 2022] GET /identity/v3/users/8cf3a976f1e5478d8d453091853dd877 => generated 297 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:11.921059 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33e68b-d9be-4b8c-923e-3218880e6c45 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:11.946652 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:11.955904 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:11.989023 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9976c-e1a2-4aa1-99ed-28d58ff809ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:11.990680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9976c-e1a2-4aa1-99ed-28d58ff809ea None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:12.094689 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 135/263] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:12.095440 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9976c-e1a2-4aa1-99ed-28d58ff809ea None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:12.121023 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33e68b-d9be-4b8c-923e-3218880e6c45 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:12.175691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d64f97-2603-4520-be7f-665b153f3bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:12.176582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d64f97-2603-4520-be7f-665b153f3bdb None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:12.176867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d64f97-2603-4520-be7f-665b153f3bdb None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:12.177143 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d64f97-2603-4520-be7f-665b153f3bdb None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:12.183115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d64f97-2603-4520-be7f-665b153f3bdb None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:12.184610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d64f97-2603-4520-be7f-665b153f3bdb None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:12.186610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 129/264] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:11 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:12.187292 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d64f97-2603-4520-be7f-665b153f3bdb None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:12.190662 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:12.199837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9976c-e1a2-4aa1-99ed-28d58ff809ea None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:12.236346 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d1e34-1ec7-4b2c-a373-905ae7158f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:12.237219 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d1e34-1ec7-4b2c-a373-905ae7158f9e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:12.237499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d1e34-1ec7-4b2c-a373-905ae7158f9e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:12.237771 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d1e34-1ec7-4b2c-a373-905ae7158f9e None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:12.238988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d1e34-1ec7-4b2c-a373-905ae7158f9e None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:12.240497 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d1e34-1ec7-4b2c-a373-905ae7158f9e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:12.242596 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 136/265] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:12 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:12.243501 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d1e34-1ec7-4b2c-a373-905ae7158f9e None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:12.295253 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:12.338937 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d64f97-2603-4520-be7f-665b153f3bdb None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:12.358505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d1e34-1ec7-4b2c-a373-905ae7158f9e None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:12.397000 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:12.403739 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb21de-d5b4-4175-a254-78460f9371b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:12.404427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb21de-d5b4-4175-a254-78460f9371b5 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:12.404709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb21de-d5b4-4175-a254-78460f9371b5 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:12.404988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb21de-d5b4-4175-a254-78460f9371b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:12.405847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb21de-d5b4-4175-a254-78460f9371b5 None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=c06a45678b434f629eb93e70012c1738, user.id=5f7cc5d676544ed6999cd3ca0e0f0782) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:12.406159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb21de-d5b4-4175-a254-78460f9371b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:12.407795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb21de-d5b4-4175-a254-78460f9371b5 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:12.410712 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c10343-c06d-49dc-a3c8-9b21fd6e740a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:12.411466 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c10343-c06d-49dc-a3c8-9b21fd6e740a None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:12.411748 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c10343-c06d-49dc-a3c8-9b21fd6e740a None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:12.412020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c10343-c06d-49dc-a3c8-9b21fd6e740a None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:12.413273 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c10343-c06d-49dc-a3c8-9b21fd6e740a None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:12.414834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c10343-c06d-49dc-a3c8-9b21fd6e740a None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:12.416747 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 130/266] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:12 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:12.418562 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c10343-c06d-49dc-a3c8-9b21fd6e740a None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:12.424528 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 137/267] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:12 2022] GET /identity/v3/role_assignments?user.id=5f7cc5d676544ed6999cd3ca0e0f0782&scope.project.id=c06a45678b434f629eb93e70012c1738&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:12.428752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb21de-d5b4-4175-a254-78460f9371b5 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:12.545079 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c10343-c06d-49dc-a3c8-9b21fd6e740a None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:12.582801 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21465ab-1336-4a94-a2ae-26bdb5a964ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:12.583712 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21465ab-1336-4a94-a2ae-26bdb5a964ab None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:12.583995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21465ab-1336-4a94-a2ae-26bdb5a964ab None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:12.584273 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21465ab-1336-4a94-a2ae-26bdb5a964ab None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738/users/8cf3a976f1e5478d8d453091853dd877/roles/2fd6a3e7df18446993c578dab05dfdce` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:12.586331 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21465ab-1336-4a94-a2ae-26bdb5a964ab None admin] RBAC: Authorizing `identity:create_grant(project_id=c06a45678b434f629eb93e70012c1738, user_id=8cf3a976f1e5478d8d453091853dd877, role_id=2fd6a3e7df18446993c578dab05dfdce)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:12.587814 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21465ab-1336-4a94-a2ae-26bdb5a964ab None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:12.613641 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 131/268] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:50:12 2022] PUT /identity/v3/projects/c06a45678b434f629eb93e70012c1738/users/8cf3a976f1e5478d8d453091853dd877/roles/2fd6a3e7df18446993c578dab05dfdce => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:12.628925 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b8581-f9bf-4522-ae00-b23095c3fac1 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:12.629272 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b8581-f9bf-4522-ae00-b23095c3fac1 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:12.629473 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b8581-f9bf-4522-ae00-b23095c3fac1 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:12.632310 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 132/269] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:50:12.638781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb21de-d5b4-4175-a254-78460f9371b5 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:12.647219 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db4443-59d7-4b48-9ebf-acd576d97f4b None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:12.647574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db4443-59d7-4b48-9ebf-acd576d97f4b None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:12.647777 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db4443-59d7-4b48-9ebf-acd576d97f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:12.674192 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:12.684211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b716176-f5a9-4211-9de8-9937d84983bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:12.685351 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b716176-f5a9-4211-9de8-9937d84983bb None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:12.685911 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b716176-f5a9-4211-9de8-9937d84983bb None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:12.686574 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b716176-f5a9-4211-9de8-9937d84983bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:12.687649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b716176-f5a9-4211-9de8-9937d84983bb None admin] RBAC: Adding query filter params (role.id=964e02e925f147e69a9e02e02a69612b, scope.project.id=c06a45678b434f629eb93e70012c1738, user.id=5f7cc5d676544ed6999cd3ca0e0f0782) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:12.688237 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b716176-f5a9-4211-9de8-9937d84983bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:12.690110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b716176-f5a9-4211-9de8-9937d84983bb None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:12.691343 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-e1db4443-59d7-4b48-9ebf-acd576d97f4b None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:12.692585 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-e1db4443-59d7-4b48-9ebf-acd576d97f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:12.705312 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 138/270] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:12 2022] GET /identity/v3/role_assignments?user.id=5f7cc5d676544ed6999cd3ca0e0f0782&scope.project.id=c06a45678b434f629eb93e70012c1738&role.id=964e02e925f147e69a9e02e02a69612b => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:12.788893 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 133/271] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:12.831428 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b716176-f5a9-4211-9de8-9937d84983bb None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:12.908730 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:12.967088 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b716176-f5a9-4211-9de8-9937d84983bb None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:13.012376 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d1e0a-0dd7-46e0-b86e-b2c99195cee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:13.013601 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d1e0a-0dd7-46e0-b86e-b2c99195cee0 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:13.014095 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d1e0a-0dd7-46e0-b86e-b2c99195cee0 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:13.014619 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d1e0a-0dd7-46e0-b86e-b2c99195cee0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:13.016081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d1e0a-0dd7-46e0-b86e-b2c99195cee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:13.018427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d1e0a-0dd7-46e0-b86e-b2c99195cee0 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:13.020782 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 139/272] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:13.026458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db4443-59d7-4b48-9ebf-acd576d97f4b None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:13.088861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177dbbb-305a-4c5e-8a83-702ff61a2512 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:13.089528 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177dbbb-305a-4c5e-8a83-702ff61a2512 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:13.090015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177dbbb-305a-4c5e-8a83-702ff61a2512 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:13.091690 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 140/273] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:13.097862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37741b-915d-4fb1-802d-4ca986066583 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:13.098192 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37741b-915d-4fb1-802d-4ca986066583 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:13.098463 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37741b-915d-4fb1-802d-4ca986066583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:13.118068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-fe37741b-915d-4fb1-802d-4ca986066583 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:13.119685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-fe37741b-915d-4fb1-802d-4ca986066583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:13.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db4443-59d7-4b48-9ebf-acd576d97f4b None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:13.193645 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade353b4-ae2c-44bb-9301-8fa4c1dc5749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:13.194473 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade353b4-ae2c-44bb-9301-8fa4c1dc5749 None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:13.194829 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade353b4-ae2c-44bb-9301-8fa4c1dc5749 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:13.195105 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade353b4-ae2c-44bb-9301-8fa4c1dc5749 None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:13.195839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade353b4-ae2c-44bb-9301-8fa4c1dc5749 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:13.197306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade353b4-ae2c-44bb-9301-8fa4c1dc5749 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:13.205748 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:13.208680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 141/274] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:13.210564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bf938-0ca8-4bf7-8c3a-773c46198851 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:13.211153 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bf938-0ca8-4bf7-8c3a-773c46198851 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:13.211716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bf938-0ca8-4bf7-8c3a-773c46198851 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:13.213518 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 142/275] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:13.220246 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a2dbb-24f5-4c73-a976-4687bef82294 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:13.220509 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a2dbb-24f5-4c73-a976-4687bef82294 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:13.220752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a2dbb-24f5-4c73-a976-4687bef82294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:13.233279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 134/276] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:50:13 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:50:13.238422 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-af3a2dbb-24f5-4c73-a976-4687bef82294 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:13.239691 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-af3a2dbb-24f5-4c73-a976-4687bef82294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:13.251493 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade353b4-ae2c-44bb-9301-8fa4c1dc5749 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:13.317199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:13.352347 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 143/277] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:13.371552 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a2dbb-24f5-4c73-a976-4687bef82294 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:13.377441 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade353b4-ae2c-44bb-9301-8fa4c1dc5749 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:13.429555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c3952-21ae-4633-b309-33f1dd671dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:13.430421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c3952-21ae-4633-b309-33f1dd671dab None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:13.430755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c3952-21ae-4633-b309-33f1dd671dab None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:13.431030 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c3952-21ae-4633-b309-33f1dd671dab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:13.449014 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c3952-21ae-4633-b309-33f1dd671dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-ce4c3952-21ae-4633-b309-33f1dd671dab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:13.460131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:13.461198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:13.462173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:13.462173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:13.462173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:13.462173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:13.462173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:13.462173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:13.462173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:13.462173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:13.462173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:13.462173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:13.462173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:50:13.462173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:13.462173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 135/278] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:13.471115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c3952-21ae-4633-b309-33f1dd671dab None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:13.496414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a2dbb-24f5-4c73-a976-4687bef82294 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:13.534794 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9783c21-230e-40af-ad92-67d14d72d8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:13.535987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9783c21-230e-40af-ad92-67d14d72d8c4 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:13.536457 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9783c21-230e-40af-ad92-67d14d72d8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:13.536927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9783c21-230e-40af-ad92-67d14d72d8c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:13.541583 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:13.553907 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9783c21-230e-40af-ad92-67d14d72d8c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-c9783c21-230e-40af-ad92-67d14d72d8c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:13.562668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:13.563811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:13.564761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:13.564761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:13.564761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:13.564761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:13.564761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:13.564761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:13.564761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:13.564761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:13.564761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:13.564761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:13.564761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:13.564761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:13.572477 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 144/279] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:13.573393 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9783c21-230e-40af-ad92-67d14d72d8c4 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:13.605385 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c3952-21ae-4633-b309-33f1dd671dab None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:13.632472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:13.649334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96012d2-65c4-4b2f-98c1-e27627c50f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:13.650013 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96012d2-65c4-4b2f-98c1-e27627c50f47 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:13.650319 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96012d2-65c4-4b2f-98c1-e27627c50f47 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:13.650605 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96012d2-65c4-4b2f-98c1-e27627c50f47 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:13.651548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96012d2-65c4-4b2f-98c1-e27627c50f47 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:13.651870 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96012d2-65c4-4b2f-98c1-e27627c50f47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:13.653879 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96012d2-65c4-4b2f-98c1-e27627c50f47 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:13.666868 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 136/280] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:13.675068 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96012d2-65c4-4b2f-98c1-e27627c50f47 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:13.717164 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9783c21-230e-40af-ad92-67d14d72d8c4 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:13.762699 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75581d-4e24-4376-bcde-f7ab8d9cd584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:13.763386 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75581d-4e24-4376-bcde-f7ab8d9cd584 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:13.763670 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75581d-4e24-4376-bcde-f7ab8d9cd584 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:13.763945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75581d-4e24-4376-bcde-f7ab8d9cd584 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:13.764718 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75581d-4e24-4376-bcde-f7ab8d9cd584 None admin] RBAC: Adding query filter params (name=member) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:13.765026 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75581d-4e24-4376-bcde-f7ab8d9cd584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:13.767524 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75581d-4e24-4376-bcde-f7ab8d9cd584 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:13.781395 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 145/281] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:13.794750 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75581d-4e24-4376-bcde-f7ab8d9cd584 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:13.806748 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96012d2-65c4-4b2f-98c1-e27627c50f47 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:13.822569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:13.846223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e343c2-64ec-4a9a-99bb-4552fffd4616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:13.847151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e343c2-64ec-4a9a-99bb-4552fffd4616 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:13.847845 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e343c2-64ec-4a9a-99bb-4552fffd4616 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:13.848154 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e343c2-64ec-4a9a-99bb-4552fffd4616 None admin] PATH_INFO: `/v3/users/f1af6f751696447fa5fab4ac872cf8df` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:13.849391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e343c2-64ec-4a9a-99bb-4552fffd4616 None admin] RBAC: Authorizing `identity:get_user(user_id=f1af6f751696447fa5fab4ac872cf8df)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:13.850981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e343c2-64ec-4a9a-99bb-4552fffd4616 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:13.853178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 137/282] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:13 2022] GET /identity/v3/users/f1af6f751696447fa5fab4ac872cf8df => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:13.858939 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e343c2-64ec-4a9a-99bb-4552fffd4616 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:13.961196 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75581d-4e24-4376-bcde-f7ab8d9cd584 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:13.980860 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:14.013454 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e343c2-64ec-4a9a-99bb-4552fffd4616 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:14.038339 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96355e6-8d34-44a8-9f67-f18a3f21f7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:14.039285 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96355e6-8d34-44a8-9f67-f18a3f21f7ca None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:14.039569 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96355e6-8d34-44a8-9f67-f18a3f21f7ca None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:14.039856 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96355e6-8d34-44a8-9f67-f18a3f21f7ca None admin] PATH_INFO: `/v3/users/623fc05e72754249b07bd11514011666` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:14.063703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96355e6-8d34-44a8-9f67-f18a3f21f7ca None admin] RBAC: Authorizing `identity:get_user(user_id=623fc05e72754249b07bd11514011666)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:14.065653 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96355e6-8d34-44a8-9f67-f18a3f21f7ca None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:14.070829 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 146/283] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:13 2022] GET /identity/v3/users/623fc05e72754249b07bd11514011666 => generated 293 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:14.074188 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288ccc4-6189-4609-abc3-aa99731f8775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:14.075171 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288ccc4-6189-4609-abc3-aa99731f8775 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:14.075450 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288ccc4-6189-4609-abc3-aa99731f8775 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:14.075723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288ccc4-6189-4609-abc3-aa99731f8775 None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:14.077088 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96355e6-8d34-44a8-9f67-f18a3f21f7ca None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:14.078503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288ccc4-6189-4609-abc3-aa99731f8775 None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:14.082143 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288ccc4-6189-4609-abc3-aa99731f8775 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:14.089724 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 138/284] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:13 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:14.091458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288ccc4-6189-4609-abc3-aa99731f8775 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:14.190179 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:14.248576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96355e6-8d34-44a8-9f67-f18a3f21f7ca None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:14.283096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eef873-a474-4d65-94e9-1fc215214e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:14.283763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eef873-a474-4d65-94e9-1fc215214e8f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:14.284042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eef873-a474-4d65-94e9-1fc215214e8f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:14.284314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eef873-a474-4d65-94e9-1fc215214e8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:14.285157 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eef873-a474-4d65-94e9-1fc215214e8f None admin] RBAC: Adding query filter params (role.id=2fd6a3e7df18446993c578dab05dfdce, user.id=623fc05e72754249b07bd11514011666, scope.system=all) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:14.285463 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eef873-a474-4d65-94e9-1fc215214e8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:14.288140 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eef873-a474-4d65-94e9-1fc215214e8f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:14.302738 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288ccc4-6189-4609-abc3-aa99731f8775 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:14.304146 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 147/285] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 19:50:14 2022] GET /identity/v3/role_assignments?user.id=623fc05e72754249b07bd11514011666&scope.system=all&role.id=2fd6a3e7df18446993c578dab05dfdce => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:14.367901 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a0662-374e-49ed-95c3-faef907da18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:14.368905 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a0662-374e-49ed-95c3-faef907da18e None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:14.369206 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a0662-374e-49ed-95c3-faef907da18e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:14.369492 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a0662-374e-49ed-95c3-faef907da18e None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738/users/f1af6f751696447fa5fab4ac872cf8df/roles/1e48f1df36584bb08d55fde909d54738` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:14.371762 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a0662-374e-49ed-95c3-faef907da18e None admin] RBAC: Authorizing `identity:create_grant(project_id=c06a45678b434f629eb93e70012c1738, user_id=f1af6f751696447fa5fab4ac872cf8df, role_id=1e48f1df36584bb08d55fde909d54738)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:14.373594 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a0662-374e-49ed-95c3-faef907da18e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:14.396804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 139/286] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:50:14 2022] PUT /identity/v3/projects/c06a45678b434f629eb93e70012c1738/users/f1af6f751696447fa5fab4ac872cf8df/roles/1e48f1df36584bb08d55fde909d54738 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:15.848231 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fabf3-1468-443f-b0b3-9fd63a201116 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:15.849259 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fabf3-1468-443f-b0b3-9fd63a201116 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:15.849792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fabf3-1468-443f-b0b3-9fd63a201116 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:15.851783 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 148/287] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:15.857576 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f185420-91fe-4c0a-99ca-185a6dda0ec5 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:15.858159 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f185420-91fe-4c0a-99ca-185a6dda0ec5 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:15.858159 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f185420-91fe-4c0a-99ca-185a6dda0ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:15.881836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-4f185420-91fe-4c0a-99ca-185a6dda0ec5 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:15.883172 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-4f185420-91fe-4c0a-99ca-185a6dda0ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:15.954833 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 140/288] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:15.978405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fabf3-1468-443f-b0b3-9fd63a201116 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:16.010228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:16.071200 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ed2d1-ef23-406e-9c55-4aeab385962b None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:16.071493 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ed2d1-ef23-406e-9c55-4aeab385962b None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:16.071985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ed2d1-ef23-406e-9c55-4aeab385962b None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:16.074540 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 141/289] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:16.084273 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b5ced-7bbb-4eac-a232-3fe2e8351d99 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:16.084909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b5ced-7bbb-4eac-a232-3fe2e8351d99 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:16.085525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b5ced-7bbb-4eac-a232-3fe2e8351d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:16.089828 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fabf3-1468-443f-b0b3-9fd63a201116 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:16.111400 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-178b5ced-7bbb-4eac-a232-3fe2e8351d99 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:16.112623 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-178b5ced-7bbb-4eac-a232-3fe2e8351d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:16.145990 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8c012-a14c-4ce6-8a96-efd648110ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:16.146852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8c012-a14c-4ce6-8a96-efd648110ded None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:16.147140 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8c012-a14c-4ce6-8a96-efd648110ded None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:16.147417 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8c012-a14c-4ce6-8a96-efd648110ded None admin] PATH_INFO: `/v3/roles/member` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:16.158773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8c012-a14c-4ce6-8a96-efd648110ded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-3ee8c012-a14c-4ce6-8a96-efd648110ded None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:16.166568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:16.167564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:16.168494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:16.168494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:16.168494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:16.168494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:16.168494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:16.168494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:16.168494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:16.168494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:16.168494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:16.168494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:16.168494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:16.168494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:16.168494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 149/290] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:16.169249 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53639281-eda6-4a97-9bcf-af62e35c7492 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:16.169322 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53639281-eda6-4a97-9bcf-af62e35c7492 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:16.169566 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53639281-eda6-4a97-9bcf-af62e35c7492 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:16.171048 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 150/291] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:16.176734 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4e8fe-29aa-45d6-ac9a-d2e003c0c360 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:16.177023 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4e8fe-29aa-45d6-ac9a-d2e003c0c360 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:16.177307 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4e8fe-29aa-45d6-ac9a-d2e003c0c360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:16.178035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:16.195959 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-6da4e8fe-29aa-45d6-ac9a-d2e003c0c360 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:16.197150 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-6da4e8fe-29aa-45d6-ac9a-d2e003c0c360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:16.223924 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 142/292] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:16.224801 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b5ced-7bbb-4eac-a232-3fe2e8351d99 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:16.268707 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 151/293] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:16.269602 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4e8fe-29aa-45d6-ac9a-d2e003c0c360 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:16.303910 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:16.370415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b5ced-7bbb-4eac-a232-3fe2e8351d99 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:16.391499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4e8fe-29aa-45d6-ac9a-d2e003c0c360 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:16.414835 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb87b38-a3e9-4dc8-a8c6-cefd00a98525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:16.415514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb87b38-a3e9-4dc8-a8c6-cefd00a98525 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:16.415800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb87b38-a3e9-4dc8-a8c6-cefd00a98525 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:16.416073 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb87b38-a3e9-4dc8-a8c6-cefd00a98525 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:16.416840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb87b38-a3e9-4dc8-a8c6-cefd00a98525 None admin] RBAC: Adding query filter params (name=member) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:16.417148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb87b38-a3e9-4dc8-a8c6-cefd00a98525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:16.418562 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb87b38-a3e9-4dc8-a8c6-cefd00a98525 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:16.427608 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 143/294] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:16.428442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb87b38-a3e9-4dc8-a8c6-cefd00a98525 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:16.440897 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a890a-d0a2-4dac-8fc0-2e480363c8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:16.441712 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a890a-d0a2-4dac-8fc0-2e480363c8e5 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:16.441990 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a890a-d0a2-4dac-8fc0-2e480363c8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:16.442281 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a890a-d0a2-4dac-8fc0-2e480363c8e5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:16.456559 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a890a-d0a2-4dac-8fc0-2e480363c8e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:16.464544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-2c2a890a-d0a2-4dac-8fc0-2e480363c8e5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:16.468673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:16.469789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:16.471206 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:16.471206 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:16.471206 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:16.471206 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:16.471206 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:16.471206 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:16.471206 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:16.471206 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:16.471206 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:16.471206 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:16.471206 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:50:16.471206 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:16.474331 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 152/295] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:16.475406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a890a-d0a2-4dac-8fc0-2e480363c8e5 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:16.555863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb87b38-a3e9-4dc8-a8c6-cefd00a98525 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:16.579757 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:16.602130 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a890a-d0a2-4dac-8fc0-2e480363c8e5 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:16.602658 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fff12f-0a84-48d4-9e21-5e1756dfbeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:16.603475 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fff12f-0a84-48d4-9e21-5e1756dfbeb0 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:16.604048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fff12f-0a84-48d4-9e21-5e1756dfbeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:16.604329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fff12f-0a84-48d4-9e21-5e1756dfbeb0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:16.605786 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fff12f-0a84-48d4-9e21-5e1756dfbeb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:16.608197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fff12f-0a84-48d4-9e21-5e1756dfbeb0 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:16.614756 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 144/296] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:16.614756 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fff12f-0a84-48d4-9e21-5e1756dfbeb0 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:16.652425 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d53ae-cdeb-45d7-9c97-43b8d45d3bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:16.654184 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d53ae-cdeb-45d7-9c97-43b8d45d3bf5 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:16.654513 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d53ae-cdeb-45d7-9c97-43b8d45d3bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:16.654791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d53ae-cdeb-45d7-9c97-43b8d45d3bf5 None admin] PATH_INFO: `/v3/users/8cf3a976f1e5478d8d453091853dd877` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:16.656004 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d53ae-cdeb-45d7-9c97-43b8d45d3bf5 None admin] RBAC: Authorizing `identity:get_user(user_id=8cf3a976f1e5478d8d453091853dd877)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:16.657671 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d53ae-cdeb-45d7-9c97-43b8d45d3bf5 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:16.662346 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 153/297] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:16 2022] GET /identity/v3/users/8cf3a976f1e5478d8d453091853dd877 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:16.663389 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d53ae-cdeb-45d7-9c97-43b8d45d3bf5 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:16.739752 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:16.744156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fff12f-0a84-48d4-9e21-5e1756dfbeb0 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:16.775587 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d53ae-cdeb-45d7-9c97-43b8d45d3bf5 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:16.791838 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397ae61-6dc4-483f-a350-dcf5e238f2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:16.792741 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397ae61-6dc4-483f-a350-dcf5e238f2fa None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:16.793030 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397ae61-6dc4-483f-a350-dcf5e238f2fa None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:16.793308 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397ae61-6dc4-483f-a350-dcf5e238f2fa None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:16.794093 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397ae61-6dc4-483f-a350-dcf5e238f2fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:16.794951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397ae61-6dc4-483f-a350-dcf5e238f2fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:16.796717 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397ae61-6dc4-483f-a350-dcf5e238f2fa None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:16.810517 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 145/298] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:16.811748 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397ae61-6dc4-483f-a350-dcf5e238f2fa None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:16.825519 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07c83d-e9e4-427f-9c05-c9954d51dbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:16.826611 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07c83d-e9e4-427f-9c05-c9954d51dbcb None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:16.826920 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07c83d-e9e4-427f-9c05-c9954d51dbcb None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:16.827198 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07c83d-e9e4-427f-9c05-c9954d51dbcb None admin] PATH_INFO: `/v3/groups` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:16.827934 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07c83d-e9e4-427f-9c05-c9954d51dbcb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:16.829703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07c83d-e9e4-427f-9c05-c9954d51dbcb None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:16.854313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 154/299] 10.222.0.84 () {60 vars in 1134 bytes} [Thu May 19 19:50:16 2022] POST /identity/v3/groups => generated 229 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:50:16.854313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07c83d-e9e4-427f-9c05-c9954d51dbcb None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:16.885392 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:16.989617 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397ae61-6dc4-483f-a350-dcf5e238f2fa None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:17.032987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07c83d-e9e4-427f-9c05-c9954d51dbcb None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:17.036470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a41d2a-f733-4569-923b-cc8c40c61a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:17.037479 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a41d2a-f733-4569-923b-cc8c40c61a8d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:17.037981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a41d2a-f733-4569-923b-cc8c40c61a8d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:17.038621 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a41d2a-f733-4569-923b-cc8c40c61a8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:17.073265 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948e31c-7828-4ea6-be0a-8a1a65dbf54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:17.074096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948e31c-7828-4ea6-be0a-8a1a65dbf54c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:17.074621 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948e31c-7828-4ea6-be0a-8a1a65dbf54c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:17.074736 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948e31c-7828-4ea6-be0a-8a1a65dbf54c None admin] PATH_INFO: `/v3/users/48340d751e3a4652a60279f1172d470d` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:17.077398 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:17.091763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948e31c-7828-4ea6-be0a-8a1a65dbf54c None admin] RBAC: Authorizing `identity:get_user(user_id=48340d751e3a4652a60279f1172d470d)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:17.093228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948e31c-7828-4ea6-be0a-8a1a65dbf54c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:17.095992 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 155/300] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:16 2022] GET /identity/v3/users/48340d751e3a4652a60279f1172d470d => generated 293 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:17.099495 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feecda6-8a51-4f27-bf8b-fa7266f6a316 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:17.099854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feecda6-8a51-4f27-bf8b-fa7266f6a316 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:17.100196 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feecda6-8a51-4f27-bf8b-fa7266f6a316 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:17.102020 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 156/301] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:17.106475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feecda6-8a51-4f27-bf8b-fa7266f6a316 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:17.129863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a41d2a-f733-4569-923b-cc8c40c61a8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:17.131393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a41d2a-f733-4569-923b-cc8c40c61a8d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:17.212523 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:17.255666 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feecda6-8a51-4f27-bf8b-fa7266f6a316 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:17.271032 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 146/302] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:17.272745 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9c874-1007-4e8b-a61e-78ec66954c83 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:17.273041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9c874-1007-4e8b-a61e-78ec66954c83 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:17.273319 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9c874-1007-4e8b-a61e-78ec66954c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:17.295370 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-98f9c874-1007-4e8b-a61e-78ec66954c83 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:17.296689 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-98f9c874-1007-4e8b-a61e-78ec66954c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:17.296832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8c1ea-b3b5-4f92-b3ca-35b2f8b3e6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:17.297466 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8c1ea-b3b5-4f92-b3ca-35b2f8b3e6f7 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:17.297771 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8c1ea-b3b5-4f92-b3ca-35b2f8b3e6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:17.298190 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8c1ea-b3b5-4f92-b3ca-35b2f8b3e6f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:17.299188 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8c1ea-b3b5-4f92-b3ca-35b2f8b3e6f7 None admin] RBAC: Adding query filter params (role.id=1e48f1df36584bb08d55fde909d54738, user.id=48340d751e3a4652a60279f1172d470d, scope.system=all) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:17.299503 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8c1ea-b3b5-4f92-b3ca-35b2f8b3e6f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:17.301011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8c1ea-b3b5-4f92-b3ca-35b2f8b3e6f7 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:17.314410 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 157/303] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 19:50:17 2022] GET /identity/v3/role_assignments?user.id=48340d751e3a4652a60279f1172d470d&scope.system=all&role.id=1e48f1df36584bb08d55fde909d54738 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:17.315516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8c1ea-b3b5-4f92-b3ca-35b2f8b3e6f7 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:17.377773 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:17.390107 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 147/304] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:17.400795 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f37212-9c69-4e1f-9bc1-b1155970444b None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:17.401136 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f37212-9c69-4e1f-9bc1-b1155970444b None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:17.401362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f37212-9c69-4e1f-9bc1-b1155970444b None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:17.402807 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 148/305] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:50:17.410256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a5e45-dd5c-4671-a657-c4fb5675c842 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:17.410634 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a5e45-dd5c-4671-a657-c4fb5675c842 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:17.414289 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a5e45-dd5c-4671-a657-c4fb5675c842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:17.442301 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-3d4a5e45-dd5c-4671-a657-c4fb5675c842 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:17.442643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-3d4a5e45-dd5c-4671-a657-c4fb5675c842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:17.454444 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8c1ea-b3b5-4f92-b3ca-35b2f8b3e6f7 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:17.493339 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae53294-fb66-4e5d-bec0-274756ca1d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:17.494237 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae53294-fb66-4e5d-bec0-274756ca1d1c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:17.494647 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae53294-fb66-4e5d-bec0-274756ca1d1c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:17.494931 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae53294-fb66-4e5d-bec0-274756ca1d1c None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:17.496239 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae53294-fb66-4e5d-bec0-274756ca1d1c None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:17.497855 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae53294-fb66-4e5d-bec0-274756ca1d1c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:17.500044 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 158/306] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:17 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:17.500733 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:17.506252 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae53294-fb66-4e5d-bec0-274756ca1d1c None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:17.530413 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 149/307] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:17.531547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a5e45-dd5c-4671-a657-c4fb5675c842 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:17.625647 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae53294-fb66-4e5d-bec0-274756ca1d1c None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:17.641801 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a5e45-dd5c-4671-a657-c4fb5675c842 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:17.669724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4638bb9-1a05-4f4b-8bd7-0360174276af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:17.670694 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638bb9-1a05-4f4b-8bd7-0360174276af None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:17.671044 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638bb9-1a05-4f4b-8bd7-0360174276af None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:17.671389 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638bb9-1a05-4f4b-8bd7-0360174276af None admin] PATH_INFO: `/v3/roles/member` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:17.671590 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:17.684174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4638bb9-1a05-4f4b-8bd7-0360174276af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:17.686647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40329123-0487-41d5-aa79-2b3a59a7656a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:17.687506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40329123-0487-41d5-aa79-2b3a59a7656a None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:17.687975 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40329123-0487-41d5-aa79-2b3a59a7656a None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:17.688413 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40329123-0487-41d5-aa79-2b3a59a7656a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:17.689401 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40329123-0487-41d5-aa79-2b3a59a7656a None admin] RBAC: Adding query filter params (role.id=2fd6a3e7df18446993c578dab05dfdce, scope.project.id=c06a45678b434f629eb93e70012c1738, user.id=8cf3a976f1e5478d8d453091853dd877) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:17.689865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40329123-0487-41d5-aa79-2b3a59a7656a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:17.691520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40329123-0487-41d5-aa79-2b3a59a7656a None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-c4638bb9-1a05-4f4b-8bd7-0360174276af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:17.692685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:17.693879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:17.695040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:17.695040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:17.695040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:17.695040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:17.695040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:17.695040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:17.695040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:17.695040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:17.695040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:17.695040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:17.695040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:17.695040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:17.700654 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 159/308] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:17.701549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 150/309] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:17 2022] GET /identity/v3/role_assignments?user.id=8cf3a976f1e5478d8d453091853dd877&scope.project.id=c06a45678b434f629eb93e70012c1738&role.id=2fd6a3e7df18446993c578dab05dfdce => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:17.701681 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4638bb9-1a05-4f4b-8bd7-0360174276af None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:17.708428 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40329123-0487-41d5-aa79-2b3a59a7656a None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:17.810593 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:17.866835 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4638bb9-1a05-4f4b-8bd7-0360174276af None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:17.882509 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40329123-0487-41d5-aa79-2b3a59a7656a None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:17.908103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a71c30-9ea0-4ba0-9a6e-3feafc02e57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:17.909406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a71c30-9ea0-4ba0-9a6e-3feafc02e57f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:17.909982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a71c30-9ea0-4ba0-9a6e-3feafc02e57f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:17.910477 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a71c30-9ea0-4ba0-9a6e-3feafc02e57f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:17.925702 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a71c30-9ea0-4ba0-9a6e-3feafc02e57f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-08a71c30-9ea0-4ba0-9a6e-3feafc02e57f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:17.935336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:17.936564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:17.937476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:17.937476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:17.937476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:17.937476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:17.937476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:17.937476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:17.937476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:17.937476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:17.937476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:17.937476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:17.937476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:50:17.937476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:17.940947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 160/310] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:17.946611 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a71c30-9ea0-4ba0-9a6e-3feafc02e57f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:17.951262 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75d19f-a61a-4610-969c-d3e97d6f5dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:17.952709 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75d19f-a61a-4610-969c-d3e97d6f5dd4 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:17.952709 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75d19f-a61a-4610-969c-d3e97d6f5dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:17.952709 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75d19f-a61a-4610-969c-d3e97d6f5dd4 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:17.953547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75d19f-a61a-4610-969c-d3e97d6f5dd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:17.953916 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75d19f-a61a-4610-969c-d3e97d6f5dd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:17.955402 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75d19f-a61a-4610-969c-d3e97d6f5dd4 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:17.971452 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 151/311] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:17.977964 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75d19f-a61a-4610-969c-d3e97d6f5dd4 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:18.063267 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a71c30-9ea0-4ba0-9a6e-3feafc02e57f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:18.082055 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:18.087051 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:18.111783 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a4b3f-2fc9-4d72-9839-d265ea02e68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:18.113757 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a4b3f-2fc9-4d72-9839-d265ea02e68e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:18.114229 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a4b3f-2fc9-4d72-9839-d265ea02e68e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:18.114762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a4b3f-2fc9-4d72-9839-d265ea02e68e None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:18.115939 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a4b3f-2fc9-4d72-9839-d265ea02e68e None admin] RBAC: Adding query filter params (name=reader) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:18.116828 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a4b3f-2fc9-4d72-9839-d265ea02e68e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:18.119702 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a4b3f-2fc9-4d72-9839-d265ea02e68e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:18.140487 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 161/312] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:18.145980 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a4b3f-2fc9-4d72-9839-d265ea02e68e None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:18.146366 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75d19f-a61a-4610-969c-d3e97d6f5dd4 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:18.192210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828bbe47-2c32-415a-8d2e-dd07d4ce279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:18.193029 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828bbe47-2c32-415a-8d2e-dd07d4ce279d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:18.193311 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828bbe47-2c32-415a-8d2e-dd07d4ce279d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:18.193587 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828bbe47-2c32-415a-8d2e-dd07d4ce279d None admin] PATH_INFO: `/v3/users/623fc05e72754249b07bd11514011666` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:18.194879 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828bbe47-2c32-415a-8d2e-dd07d4ce279d None admin] RBAC: Authorizing `identity:get_user(user_id=623fc05e72754249b07bd11514011666)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:18.196323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828bbe47-2c32-415a-8d2e-dd07d4ce279d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:18.198716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 152/313] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:17 2022] GET /identity/v3/users/623fc05e72754249b07bd11514011666 => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:18.204634 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828bbe47-2c32-415a-8d2e-dd07d4ce279d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:18.269833 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a4b3f-2fc9-4d72-9839-d265ea02e68e None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:18.318079 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae481769-a23c-436c-9a9d-9ab7fff4326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:18.319707 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae481769-a23c-436c-9a9d-9ab7fff4326a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:18.320224 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae481769-a23c-436c-9a9d-9ab7fff4326a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:18.320705 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae481769-a23c-436c-9a9d-9ab7fff4326a None admin] PATH_INFO: `/v3/users/f1af6f751696447fa5fab4ac872cf8df` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:18.322861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae481769-a23c-436c-9a9d-9ab7fff4326a None admin] RBAC: Authorizing `identity:get_user(user_id=f1af6f751696447fa5fab4ac872cf8df)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:18.324800 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae481769-a23c-436c-9a9d-9ab7fff4326a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:18.328250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 162/314] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:18 2022] GET /identity/v3/users/f1af6f751696447fa5fab4ac872cf8df => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:18.334608 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae481769-a23c-436c-9a9d-9ab7fff4326a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:18.337536 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828bbe47-2c32-415a-8d2e-dd07d4ce279d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:18.356787 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:18.377798 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4f04d-3300-419b-b7ee-0eb442c80134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:18.378758 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4f04d-3300-419b-b7ee-0eb442c80134 None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:18.379044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4f04d-3300-419b-b7ee-0eb442c80134 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:18.379580 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4f04d-3300-419b-b7ee-0eb442c80134 None admin] PATH_INFO: `/v3/system/users/623fc05e72754249b07bd11514011666/roles/2fd6a3e7df18446993c578dab05dfdce` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:18.381927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4f04d-3300-419b-b7ee-0eb442c80134 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=623fc05e72754249b07bd11514011666, role_id=2fd6a3e7df18446993c578dab05dfdce)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:18.383363 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4f04d-3300-419b-b7ee-0eb442c80134 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:18.394540 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 153/315] 10.222.0.84 () {58 vars in 1334 bytes} [Thu May 19 19:50:18 2022] PUT /identity/v3/system/users/623fc05e72754249b07bd11514011666/roles/2fd6a3e7df18446993c578dab05dfdce => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:18.398749 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:18.468599 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae481769-a23c-436c-9a9d-9ab7fff4326a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:18.509596 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452cb99-7001-4bfa-bb40-b15adaaba368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:18.510676 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452cb99-7001-4bfa-bb40-b15adaaba368 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:18.511211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452cb99-7001-4bfa-bb40-b15adaaba368 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:18.511708 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452cb99-7001-4bfa-bb40-b15adaaba368 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:18.591511 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452cb99-7001-4bfa-bb40-b15adaaba368 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:18.592990 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452cb99-7001-4bfa-bb40-b15adaaba368 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:18.642894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:18.696283 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 163/316] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:50:18.702584 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4f04d-3300-419b-b7ee-0eb442c80134 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:18.818843 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4f04d-3300-419b-b7ee-0eb442c80134 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:18.855321 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e34863-d26f-46b5-80a2-47fe3d603e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:18.856199 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e34863-d26f-46b5-80a2-47fe3d603e22 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:18.856483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e34863-d26f-46b5-80a2-47fe3d603e22 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:18.856758 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e34863-d26f-46b5-80a2-47fe3d603e22 None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:18.857952 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e34863-d26f-46b5-80a2-47fe3d603e22 None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:18.859438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e34863-d26f-46b5-80a2-47fe3d603e22 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:18.861506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 154/317] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:18 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:18.867045 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452cb99-7001-4bfa-bb40-b15adaaba368 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:18.969515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452cb99-7001-4bfa-bb40-b15adaaba368 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:19.008727 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4523c5-a0c7-4f26-bfd4-2da003c9bbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:19.009816 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4523c5-a0c7-4f26-bfd4-2da003c9bbe9 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:19.010338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4523c5-a0c7-4f26-bfd4-2da003c9bbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:19.010818 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4523c5-a0c7-4f26-bfd4-2da003c9bbe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:19.011885 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4523c5-a0c7-4f26-bfd4-2da003c9bbe9 None admin] RBAC: Adding query filter params (role.id=1e48f1df36584bb08d55fde909d54738, scope.project.id=c06a45678b434f629eb93e70012c1738, user.id=f1af6f751696447fa5fab4ac872cf8df) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:19.012418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4523c5-a0c7-4f26-bfd4-2da003c9bbe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:19.014086 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4523c5-a0c7-4f26-bfd4-2da003c9bbe9 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:19.021421 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 164/318] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:18 2022] GET /identity/v3/role_assignments?user.id=f1af6f751696447fa5fab4ac872cf8df&scope.project.id=c06a45678b434f629eb93e70012c1738&role.id=1e48f1df36584bb08d55fde909d54738 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:19.384605 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd70c5-66cc-476f-b5c4-77a22106021a None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:19.385167 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd70c5-66cc-476f-b5c4-77a22106021a None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:19.385167 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd70c5-66cc-476f-b5c4-77a22106021a None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:19.386726 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 155/319] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:19.391966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483829a3-1666-45a7-8485-8c2cdf22f492 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:19.392272 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483829a3-1666-45a7-8485-8c2cdf22f492 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:19.392510 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483829a3-1666-45a7-8485-8c2cdf22f492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:19.410319 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-483829a3-1666-45a7-8485-8c2cdf22f492 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:19.411560 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-483829a3-1666-45a7-8485-8c2cdf22f492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:19.419860 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:19.484038 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 165/320] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:19.507141 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd70c5-66cc-476f-b5c4-77a22106021a None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:19.565007 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:19.623432 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd70c5-66cc-476f-b5c4-77a22106021a None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:19.659546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead22f1-26d2-4f00-91b0-1eca73e364c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:19.660701 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead22f1-26d2-4f00-91b0-1eca73e364c2 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:19.661129 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead22f1-26d2-4f00-91b0-1eca73e364c2 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:19.661557 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead22f1-26d2-4f00-91b0-1eca73e364c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:19.662913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead22f1-26d2-4f00-91b0-1eca73e364c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:19.664534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead22f1-26d2-4f00-91b0-1eca73e364c2 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:19.670884 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 156/321] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:19.672246 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483829a3-1666-45a7-8485-8c2cdf22f492 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:19.697350 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23b5aa-7d36-442a-bb3c-2ebd08f9f3bf None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:19.697689 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23b5aa-7d36-442a-bb3c-2ebd08f9f3bf None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:19.697900 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23b5aa-7d36-442a-bb3c-2ebd08f9f3bf None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:19.699358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 157/322] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:19.704469 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e2e73-9e99-47d4-86be-d08a3e6789ee None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:19.704729 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e2e73-9e99-47d4-86be-d08a3e6789ee None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:19.705074 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e2e73-9e99-47d4-86be-d08a3e6789ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:19.723459 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-b30e2e73-9e99-47d4-86be-d08a3e6789ee None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:19.724773 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-b30e2e73-9e99-47d4-86be-d08a3e6789ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:19.787642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483829a3-1666-45a7-8485-8c2cdf22f492 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:19.814314 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 158/323] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:19.826289 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42212f0b-af14-46de-a57c-9771d7237b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:19.826974 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42212f0b-af14-46de-a57c-9771d7237b82 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:19.827226 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42212f0b-af14-46de-a57c-9771d7237b82 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:19.827522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42212f0b-af14-46de-a57c-9771d7237b82 None admin] PATH_INFO: `/v3/groups` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:19.828208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42212f0b-af14-46de-a57c-9771d7237b82 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:19.829670 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42212f0b-af14-46de-a57c-9771d7237b82 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:19.834149 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e2e73-9e99-47d4-86be-d08a3e6789ee None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:19.854299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 166/324] 10.222.0.84 () {60 vars in 1134 bytes} [Thu May 19 19:50:19 2022] POST /identity/v3/groups => generated 226 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:50:19.896420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:19.956394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e2e73-9e99-47d4-86be-d08a3e6789ee None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:19.993113 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69f1d8-54ff-42d0-8f93-e8105e0387cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:19.993927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69f1d8-54ff-42d0-8f93-e8105e0387cf None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:19.994208 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69f1d8-54ff-42d0-8f93-e8105e0387cf None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:19.994508 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69f1d8-54ff-42d0-8f93-e8105e0387cf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:20.006253 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69f1d8-54ff-42d0-8f93-e8105e0387cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-af69f1d8-54ff-42d0-8f93-e8105e0387cf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:20.014182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:20.015297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:20.021691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:20.021691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:20.021691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:20.021691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:20.021691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:20.021691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:20.021691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:20.021691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:20.021691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:20.021691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:20.021691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:50:20.021691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:20.021691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 159/325] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:20.022491 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42212f0b-af14-46de-a57c-9771d7237b82 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:20.025526 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:20.136140 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42212f0b-af14-46de-a57c-9771d7237b82 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:20.171264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca3dc1-7678-4b06-a37d-deb57d249fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:20.171941 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca3dc1-7678-4b06-a37d-deb57d249fdf None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:20.172254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca3dc1-7678-4b06-a37d-deb57d249fdf None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:20.172621 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca3dc1-7678-4b06-a37d-deb57d249fdf None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:20.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca3dc1-7678-4b06-a37d-deb57d249fdf None admin] RBAC: Adding query filter params (name=reader) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:20.173697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca3dc1-7678-4b06-a37d-deb57d249fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:20.175119 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca3dc1-7678-4b06-a37d-deb57d249fdf None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:20.184333 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 167/326] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:20.190377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69f1d8-54ff-42d0-8f93-e8105e0387cf None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:20.299052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69f1d8-54ff-42d0-8f93-e8105e0387cf None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:20.308580 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:20.337295 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be432967-c892-4e52-937f-2ae4e1a116e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:20.338498 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be432967-c892-4e52-937f-2ae4e1a116e1 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:20.339039 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be432967-c892-4e52-937f-2ae4e1a116e1 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:20.339480 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be432967-c892-4e52-937f-2ae4e1a116e1 None admin] PATH_INFO: `/v3/users/48340d751e3a4652a60279f1172d470d` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:20.340959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be432967-c892-4e52-937f-2ae4e1a116e1 None admin] RBAC: Authorizing `identity:get_user(user_id=48340d751e3a4652a60279f1172d470d)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:20.342594 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be432967-c892-4e52-937f-2ae4e1a116e1 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:20.345087 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 160/327] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:20 2022] GET /identity/v3/users/48340d751e3a4652a60279f1172d470d => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:20.351483 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca3dc1-7678-4b06-a37d-deb57d249fdf None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:20.418252 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:20.458784 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca3dc1-7678-4b06-a37d-deb57d249fdf None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:20.494657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3d4ee-5e69-46db-9c70-779e8b82ae81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:20.495818 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3d4ee-5e69-46db-9c70-779e8b82ae81 None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:20.496252 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3d4ee-5e69-46db-9c70-779e8b82ae81 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:20.496678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3d4ee-5e69-46db-9c70-779e8b82ae81 None admin] PATH_INFO: `/v3/system/users/48340d751e3a4652a60279f1172d470d/roles/1e48f1df36584bb08d55fde909d54738` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:20.498367 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec3d4ee-5e69-46db-9c70-779e8b82ae81 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=48340d751e3a4652a60279f1172d470d, role_id=1e48f1df36584bb08d55fde909d54738)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:20.499920 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec3d4ee-5e69-46db-9c70-779e8b82ae81 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:20.512443 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 168/328] 10.222.0.84 () {58 vars in 1334 bytes} [Thu May 19 19:50:20 2022] PUT /identity/v3/system/users/48340d751e3a4652a60279f1172d470d/roles/1e48f1df36584bb08d55fde909d54738 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:50:20.754855 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16954042-a65f-45ee-b353-0b9e88986e4e None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:20.755391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16954042-a65f-45ee-b353-0b9e88986e4e None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:20.755636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16954042-a65f-45ee-b353-0b9e88986e4e None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:20.757175 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 161/329] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:20.763048 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ca95c-a1ea-4450-968c-cc6fcef16ef8 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:20.763340 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ca95c-a1ea-4450-968c-cc6fcef16ef8 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:20.763621 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ca95c-a1ea-4450-968c-cc6fcef16ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:20.784279 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-4d3ca95c-a1ea-4450-968c-cc6fcef16ef8 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:20.785808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-4d3ca95c-a1ea-4450-968c-cc6fcef16ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:20.860462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 169/330] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:20.885794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16954042-a65f-45ee-b353-0b9e88986e4e None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:21.014382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16954042-a65f-45ee-b353-0b9e88986e4e None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:21.059882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b06001-065c-4c1b-ac2f-90e1f41fae95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:21.060692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b06001-065c-4c1b-ac2f-90e1f41fae95 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:21.060977 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b06001-065c-4c1b-ac2f-90e1f41fae95 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:21.061251 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b06001-065c-4c1b-ac2f-90e1f41fae95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:21.079485 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b06001-065c-4c1b-ac2f-90e1f41fae95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-f7b06001-065c-4c1b-ac2f-90e1f41fae95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:21.087286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:21.088279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:21.089217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:21.089217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:21.089217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:21.089217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:21.089217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:21.089217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:21.089217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:21.089217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:21.089217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:21.089217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:21.089217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:21.089217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:21.089217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 162/331] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:21.096610 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ca95c-a1ea-4450-968c-cc6fcef16ef8 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:21.179605 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:21.234532 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ca95c-a1ea-4450-968c-cc6fcef16ef8 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:21.290800 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fdfc2-f7e4-4265-8eab-13143f0e7993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:21.291962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fdfc2-f7e4-4265-8eab-13143f0e7993 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:21.292465 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fdfc2-f7e4-4265-8eab-13143f0e7993 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:21.292947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fdfc2-f7e4-4265-8eab-13143f0e7993 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:21.293933 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fdfc2-f7e4-4265-8eab-13143f0e7993 None admin] RBAC: Adding query filter params (name=member) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:21.294476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fdfc2-f7e4-4265-8eab-13143f0e7993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:21.296205 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fdfc2-f7e4-4265-8eab-13143f0e7993 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:21.306549 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 170/332] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:21.311931 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b06001-065c-4c1b-ac2f-90e1f41fae95 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:21.377111 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:21.453975 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b06001-065c-4c1b-ac2f-90e1f41fae95 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:21.496034 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a570bb-58f3-49a7-a33f-71461b958700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:21.496920 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a570bb-58f3-49a7-a33f-71461b958700 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:21.497199 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a570bb-58f3-49a7-a33f-71461b958700 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:21.497471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a570bb-58f3-49a7-a33f-71461b958700 None admin] PATH_INFO: `/v3/users/623fc05e72754249b07bd11514011666` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:21.500291 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a570bb-58f3-49a7-a33f-71461b958700 None admin] RBAC: Authorizing `identity:get_user(user_id=623fc05e72754249b07bd11514011666)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:21.501993 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a570bb-58f3-49a7-a33f-71461b958700 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:21.504158 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 163/333] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:21 2022] GET /identity/v3/users/623fc05e72754249b07bd11514011666 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:21.509944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fdfc2-f7e4-4265-8eab-13143f0e7993 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:21.633881 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fdfc2-f7e4-4265-8eab-13143f0e7993 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:21.659416 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:21.679272 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d33e5a-7a52-46d6-8ef0-478cf9ae16ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:21.679939 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d33e5a-7a52-46d6-8ef0-478cf9ae16ef None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:21.680220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d33e5a-7a52-46d6-8ef0-478cf9ae16ef None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:21.680491 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d33e5a-7a52-46d6-8ef0-478cf9ae16ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:21.681337 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d33e5a-7a52-46d6-8ef0-478cf9ae16ef None admin] RBAC: Adding query filter params (role.id=2fd6a3e7df18446993c578dab05dfdce, user.id=623fc05e72754249b07bd11514011666, scope.system=all) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:21.681646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d33e5a-7a52-46d6-8ef0-478cf9ae16ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:21.683132 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d33e5a-7a52-46d6-8ef0-478cf9ae16ef None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:21.701205 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 171/334] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 19:50:21 2022] GET /identity/v3/role_assignments?user.id=623fc05e72754249b07bd11514011666&scope.system=all&role.id=2fd6a3e7df18446993c578dab05dfdce => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:23.013400 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7673f-a32e-419b-80a6-9785c8feee72 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.014455 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7673f-a32e-419b-80a6-9785c8feee72 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.014962 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7673f-a32e-419b-80a6-9785c8feee72 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.016940 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 164/335] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:23.032395 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32325695-1fab-4afd-b60b-28668a4d5e55 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.033410 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32325695-1fab-4afd-b60b-28668a4d5e55 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.033924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32325695-1fab-4afd-b60b-28668a4d5e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.053521 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-32325695-1fab-4afd-b60b-28668a4d5e55 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:23.055204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-32325695-1fab-4afd-b60b-28668a4d5e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:23.135233 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 172/336] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:23.166298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7673f-a32e-419b-80a6-9785c8feee72 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:23.280777 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7673f-a32e-419b-80a6-9785c8feee72 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:23.283060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:23.289074 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04b63e-0bf0-47af-a7ee-277ede4c0304 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.289710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04b63e-0bf0-47af-a7ee-277ede4c0304 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.290216 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04b63e-0bf0-47af-a7ee-277ede4c0304 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.292206 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 173/337] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:23.299052 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cf234-0a8d-4ce7-9d00-b80c0fdb9635 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.299785 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cf234-0a8d-4ce7-9d00-b80c0fdb9635 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.300270 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cf234-0a8d-4ce7-9d00-b80c0fdb9635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.320701 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-818cf234-0a8d-4ce7-9d00-b80c0fdb9635 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:23.322357 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-818cf234-0a8d-4ce7-9d00-b80c0fdb9635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:23.344504 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eef43e-8dc4-41f3-bf17-53cc392b4bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:23.345344 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eef43e-8dc4-41f3-bf17-53cc392b4bc4 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.345626 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eef43e-8dc4-41f3-bf17-53cc392b4bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.345900 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eef43e-8dc4-41f3-bf17-53cc392b4bc4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.375367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eef43e-8dc4-41f3-bf17-53cc392b4bc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-e7eef43e-8dc4-41f3-bf17-53cc392b4bc4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:23.388999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:23.390397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:23.391639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:23.391639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:23.391639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:23.391639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:23.391639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:23.391639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:23.391639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:23.391639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:23.391639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:23.391639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:23.391639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:23.391639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:23.394202 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 165/338] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:50:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:23.400032 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eef43e-8dc4-41f3-bf17-53cc392b4bc4 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:23.422286 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 174/339] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:23.451482 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cf234-0a8d-4ce7-9d00-b80c0fdb9635 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:23.527312 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:23.553316 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eef43e-8dc4-41f3-bf17-53cc392b4bc4 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:23.607563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064839a6-4353-46ba-aa4c-53a5dd412707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:23.608238 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064839a6-4353-46ba-aa4c-53a5dd412707 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.608517 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064839a6-4353-46ba-aa4c-53a5dd412707 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.608789 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064839a6-4353-46ba-aa4c-53a5dd412707 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.609555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064839a6-4353-46ba-aa4c-53a5dd412707 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:23.610061 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064839a6-4353-46ba-aa4c-53a5dd412707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:23.611522 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064839a6-4353-46ba-aa4c-53a5dd412707 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:23.628232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 166/340] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:23.629528 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20413e29-d00f-4ea0-acd4-c03006bfa503 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.629811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20413e29-d00f-4ea0-acd4-c03006bfa503 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.630084 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20413e29-d00f-4ea0-acd4-c03006bfa503 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.631600 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 167/341] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:23.635498 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cf234-0a8d-4ce7-9d00-b80c0fdb9635 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:23.636707 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e9675-0516-4682-a04f-f382343ca7c4 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.636993 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e9675-0516-4682-a04f-f382343ca7c4 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.637269 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e9675-0516-4682-a04f-f382343ca7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.657688 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-f07e9675-0516-4682-a04f-f382343ca7c4 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:23.658905 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-f07e9675-0516-4682-a04f-f382343ca7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:23.683068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4407cfc-bd6e-45fb-aa4e-f4b8d67099d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:23.684413 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:23.685281 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4407cfc-bd6e-45fb-aa4e-f4b8d67099d6 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.685560 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4407cfc-bd6e-45fb-aa4e-f4b8d67099d6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.689001 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4407cfc-bd6e-45fb-aa4e-f4b8d67099d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.705889 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4407cfc-bd6e-45fb-aa4e-f4b8d67099d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-a4407cfc-bd6e-45fb-aa4e-f4b8d67099d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:23.720110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:23.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:23.722156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:23.722156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:23.722156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:23.722156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:23.722156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:23.722156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:23.722156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:23.722156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:23.722156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:23.722156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:23.722156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:23.722156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:23.722156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 175/342] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:23.726177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4407cfc-bd6e-45fb-aa4e-f4b8d67099d6 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:23.782297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 168/343] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:23.783315 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e9675-0516-4682-a04f-f382343ca7c4 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:23.944197 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:23.950118 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4407cfc-bd6e-45fb-aa4e-f4b8d67099d6 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:23.983381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e9675-0516-4682-a04f-f382343ca7c4 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:24.016801 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bb29e-cd1f-468d-869a-223058edd145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:24.016801 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bb29e-cd1f-468d-869a-223058edd145 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:24.016801 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bb29e-cd1f-468d-869a-223058edd145 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:24.016801 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bb29e-cd1f-468d-869a-223058edd145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:24.054272 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45d09c-c566-4932-b84a-93a5de7c9459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:24.055002 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45d09c-c566-4932-b84a-93a5de7c9459 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:24.058294 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45d09c-c566-4932-b84a-93a5de7c9459 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:24.058294 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45d09c-c566-4932-b84a-93a5de7c9459 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:24.058294 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45d09c-c566-4932-b84a-93a5de7c9459 None admin] RBAC: Adding query filter params (name=member) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:24.058294 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45d09c-c566-4932-b84a-93a5de7c9459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:24.058294 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45d09c-c566-4932-b84a-93a5de7c9459 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:24.083248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 169/344] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:24.083914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45d09c-c566-4932-b84a-93a5de7c9459 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:24.132191 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:24.184306 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7bb29e-cd1f-468d-869a-223058edd145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:24.185809 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7bb29e-cd1f-468d-869a-223058edd145 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:24.304515 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45d09c-c566-4932-b84a-93a5de7c9459 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:24.347095 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4236b41-bf87-462d-b4b7-3bdaa4268c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:24.348215 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4236b41-bf87-462d-b4b7-3bdaa4268c19 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:24.348491 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4236b41-bf87-462d-b4b7-3bdaa4268c19 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:24.348763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4236b41-bf87-462d-b4b7-3bdaa4268c19 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:24.362318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4236b41-bf87-462d-b4b7-3bdaa4268c19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-e4236b41-bf87-462d-b4b7-3bdaa4268c19 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:24.374082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:24.375209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:24.380802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:24.380802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:24.380802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:24.380802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:24.380802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:24.380802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:24.380802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:24.380802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:24.380802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:24.380802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:24.380802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:50:24.380802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:24.380802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 170/345] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:24.380802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4236b41-bf87-462d-b4b7-3bdaa4268c19 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:24.438779 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:24.442339 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 176/346] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:24.447648 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bb29e-cd1f-468d-869a-223058edd145 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:24.479556 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:24.507406 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4236b41-bf87-462d-b4b7-3bdaa4268c19 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:24.554917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6a2da-16b6-448b-8e13-5fe383b45fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:24.556492 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6a2da-16b6-448b-8e13-5fe383b45fc5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:24.556836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6a2da-16b6-448b-8e13-5fe383b45fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:24.557172 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6a2da-16b6-448b-8e13-5fe383b45fc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:24.632299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bb29e-cd1f-468d-869a-223058edd145 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:24.642314 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6a2da-16b6-448b-8e13-5fe383b45fc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:24.644272 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6a2da-16b6-448b-8e13-5fe383b45fc5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:24.711429 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7123c7-0509-42d7-a7ac-290046091aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:24.712129 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7123c7-0509-42d7-a7ac-290046091aee None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:24.712411 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7123c7-0509-42d7-a7ac-290046091aee None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:24.723172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7123c7-0509-42d7-a7ac-290046091aee None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:24.724058 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7123c7-0509-42d7-a7ac-290046091aee None admin] RBAC: Adding query filter params (name=reader) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:24.724377 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7123c7-0509-42d7-a7ac-290046091aee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:24.725803 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7123c7-0509-42d7-a7ac-290046091aee None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:24.748511 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 177/347] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:24.750084 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7123c7-0509-42d7-a7ac-290046091aee None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:24.776266 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:24.806992 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 171/348] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:24.808526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a0ca2-fa0d-4216-bf2b-d554e2b31d4c None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:24.808870 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a0ca2-fa0d-4216-bf2b-d554e2b31d4c None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:24.809333 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a0ca2-fa0d-4216-bf2b-d554e2b31d4c None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:24.811037 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 172/349] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:24.811877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a0ca2-fa0d-4216-bf2b-d554e2b31d4c None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:24.845685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:24.898793 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7123c7-0509-42d7-a7ac-290046091aee None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:24.934013 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b918ca-8500-4451-a903-95d7134a73b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:24.934945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b918ca-8500-4451-a903-95d7134a73b6 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:24.935227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b918ca-8500-4451-a903-95d7134a73b6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:24.935515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b918ca-8500-4451-a903-95d7134a73b6 None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:24.936826 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b918ca-8500-4451-a903-95d7134a73b6 None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:24.938951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b918ca-8500-4451-a903-95d7134a73b6 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:24.941067 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 178/350] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:24 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:24.942616 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf717c26-3298-4873-b77d-1ed64f60f43d None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:24.943075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf717c26-3298-4873-b77d-1ed64f60f43d None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:24.943516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf717c26-3298-4873-b77d-1ed64f60f43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:24.958766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a0ca2-fa0d-4216-bf2b-d554e2b31d4c None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:24.961692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-bf717c26-3298-4873-b77d-1ed64f60f43d None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:24.963920 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-bf717c26-3298-4873-b77d-1ed64f60f43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:25.003466 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d3ea2-7a5a-4747-8ecc-c6fa4dc1296e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:25.004226 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d3ea2-7a5a-4747-8ecc-c6fa4dc1296e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:25.004504 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d3ea2-7a5a-4747-8ecc-c6fa4dc1296e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:25.004778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d3ea2-7a5a-4747-8ecc-c6fa4dc1296e None admin] PATH_INFO: `/v3/users/48340d751e3a4652a60279f1172d470d` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:25.005992 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d3ea2-7a5a-4747-8ecc-c6fa4dc1296e None admin] RBAC: Authorizing `identity:get_user(user_id=48340d751e3a4652a60279f1172d470d)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:25.007495 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d3ea2-7a5a-4747-8ecc-c6fa4dc1296e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:25.013776 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 173/351] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:50:24 2022] GET /identity/v3/users/48340d751e3a4652a60279f1172d470d => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:25.013776 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d3ea2-7a5a-4747-8ecc-c6fa4dc1296e None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:25.061516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 179/352] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:25.063042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ed2eb-1dbc-4ec8-a398-ebe44d8e9ade None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:25.063585 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ed2eb-1dbc-4ec8-a398-ebe44d8e9ade None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:25.063865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ed2eb-1dbc-4ec8-a398-ebe44d8e9ade None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:25.065160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 180/353] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:25.065844 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ed2eb-1dbc-4ec8-a398-ebe44d8e9ade None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:25.079814 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:25.144392 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:25.154560 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d3ea2-7a5a-4747-8ecc-c6fa4dc1296e None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:25.195870 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184588e1-07c6-49b5-a837-b05a9aacfd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:25.196242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ed2eb-1dbc-4ec8-a398-ebe44d8e9ade None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:25.196752 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184588e1-07c6-49b5-a837-b05a9aacfd11 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:25.197032 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184588e1-07c6-49b5-a837-b05a9aacfd11 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:25.197304 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184588e1-07c6-49b5-a837-b05a9aacfd11 None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:25.198554 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184588e1-07c6-49b5-a837-b05a9aacfd11 None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:25.200044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184588e1-07c6-49b5-a837-b05a9aacfd11 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:25.202190 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 174/354] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:25 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:25.202650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184588e1-07c6-49b5-a837-b05a9aacfd11 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:25.242038 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9992c2-8b97-479c-beea-cef104e38e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:25.243239 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9992c2-8b97-479c-beea-cef104e38e6a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:25.243678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9992c2-8b97-479c-beea-cef104e38e6a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:25.243955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9992c2-8b97-479c-beea-cef104e38e6a None admin] PATH_INFO: `/v3/groups/89c57e2b55954eb1aafdf8513a450951` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:25.250647 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9992c2-8b97-479c-beea-cef104e38e6a None admin] RBAC: Authorizing `identity:get_group(group_id=89c57e2b55954eb1aafdf8513a450951)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:25.252107 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9992c2-8b97-479c-beea-cef104e38e6a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:25.253976 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 181/355] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:50:25 2022] GET /identity/v3/groups/89c57e2b55954eb1aafdf8513a450951 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:25.255097 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78868a4-d675-407c-b933-950fd9aa4304 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:25.255373 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78868a4-d675-407c-b933-950fd9aa4304 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:25.255645 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78868a4-d675-407c-b933-950fd9aa4304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:25.272437 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-d78868a4-d675-407c-b933-950fd9aa4304 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:25.273647 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-d78868a4-d675-407c-b933-950fd9aa4304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:25.320046 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184588e1-07c6-49b5-a837-b05a9aacfd11 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:25.335849 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:25.354454 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 182/356] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:25.355687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78868a4-d675-407c-b933-950fd9aa4304 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:25.374412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6fbbd-fa2a-426f-81b1-ce6bb8e1df9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:25.375868 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6fbbd-fa2a-426f-81b1-ce6bb8e1df9f None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:25.376305 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6fbbd-fa2a-426f-81b1-ce6bb8e1df9f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:25.376736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6fbbd-fa2a-426f-81b1-ce6bb8e1df9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:25.377915 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6fbbd-fa2a-426f-81b1-ce6bb8e1df9f None admin] RBAC: Adding query filter params (role.id=1e48f1df36584bb08d55fde909d54738, user.id=48340d751e3a4652a60279f1172d470d, scope.system=all) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:25.378626 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6fbbd-fa2a-426f-81b1-ce6bb8e1df9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:25.380093 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6fbbd-fa2a-426f-81b1-ce6bb8e1df9f None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:25.393840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 175/357] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 19:50:25 2022] GET /identity/v3/role_assignments?user.id=48340d751e3a4652a60279f1172d470d&scope.system=all&role.id=1e48f1df36584bb08d55fde909d54738 => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:25.395187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e806bd-2771-4be6-bf21-cd268dcca635 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:25.395480 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e806bd-2771-4be6-bf21-cd268dcca635 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:25.395778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e806bd-2771-4be6-bf21-cd268dcca635 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:25.397060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 176/358] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:25.397800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e806bd-2771-4be6-bf21-cd268dcca635 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:25.421826 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:25.489426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78868a4-d675-407c-b933-950fd9aa4304 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:25.536278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96537175-26a2-4dff-8539-1a0324642d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:25.537101 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96537175-26a2-4dff-8539-1a0324642d70 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:25.537387 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96537175-26a2-4dff-8539-1a0324642d70 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:25.537665 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96537175-26a2-4dff-8539-1a0324642d70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:25.555346 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96537175-26a2-4dff-8539-1a0324642d70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:25.561693 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e806bd-2771-4be6-bf21-cd268dcca635 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-96537175-26a2-4dff-8539-1a0324642d70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:25.563245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:25.564802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:25.565773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:25.565773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:25.565773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:25.565773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:25.565773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:25.565773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:25.565773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:25.565773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:25.565773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:25.565773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:25.565773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:25.565773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:25.565773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 183/359] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:25.565773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96537175-26a2-4dff-8539-1a0324642d70 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:25.615664 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607b415-fe4a-47f9-8ca7-359390203a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:25.616482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607b415-fe4a-47f9-8ca7-359390203a5d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:25.616768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607b415-fe4a-47f9-8ca7-359390203a5d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:25.617040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607b415-fe4a-47f9-8ca7-359390203a5d None admin] PATH_INFO: `/v3/groups/89c57e2b55954eb1aafdf8513a450951` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:25.618433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5607b415-fe4a-47f9-8ca7-359390203a5d None admin] RBAC: Authorizing `identity:get_group(group_id=89c57e2b55954eb1aafdf8513a450951)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:25.620322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5607b415-fe4a-47f9-8ca7-359390203a5d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:25.622351 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 177/360] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:50:25 2022] GET /identity/v3/groups/89c57e2b55954eb1aafdf8513a450951 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:25.623120 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607b415-fe4a-47f9-8ca7-359390203a5d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:25.673345 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:25.705017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96537175-26a2-4dff-8539-1a0324642d70 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:25.721445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:25.747137 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e7a0f-88b9-44bf-b2c8-7199f8ad0e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:25.747953 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e7a0f-88b9-44bf-b2c8-7199f8ad0e00 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:25.748244 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e7a0f-88b9-44bf-b2c8-7199f8ad0e00 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:25.748550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e7a0f-88b9-44bf-b2c8-7199f8ad0e00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:25.749373 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e7a0f-88b9-44bf-b2c8-7199f8ad0e00 None admin] RBAC: Adding query filter params (group.id=89c57e2b55954eb1aafdf8513a450951, role.id=964e02e925f147e69a9e02e02a69612b, scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:25.749688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e7a0f-88b9-44bf-b2c8-7199f8ad0e00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:25.755050 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e7a0f-88b9-44bf-b2c8-7199f8ad0e00 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:25.760001 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607b415-fe4a-47f9-8ca7-359390203a5d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:25.761070 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 184/361] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:50:25 2022] GET /identity/v3/role_assignments?group.id=89c57e2b55954eb1aafdf8513a450951&scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18&role.id=964e02e925f147e69a9e02e02a69612b => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:25.762391 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9724383-ff39-4520-b81c-32f07f2653a9 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:25.762686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9724383-ff39-4520-b81c-32f07f2653a9 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:25.762966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9724383-ff39-4520-b81c-32f07f2653a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:25.783742 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-f9724383-ff39-4520-b81c-32f07f2653a9 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:25.784980 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-f9724383-ff39-4520-b81c-32f07f2653a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:25.816205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bce5a8-960c-4e27-9e5b-6603cda40f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:25.817049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bce5a8-960c-4e27-9e5b-6603cda40f8d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:25.817337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bce5a8-960c-4e27-9e5b-6603cda40f8d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:25.817615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bce5a8-960c-4e27-9e5b-6603cda40f8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:25.833942 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bce5a8-960c-4e27-9e5b-6603cda40f8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-b9bce5a8-960c-4e27-9e5b-6603cda40f8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:25.845611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:25.846854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:25.848111 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:25.848111 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:25.848111 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:25.848111 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:25.848111 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:25.848111 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:25.848111 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:25.848111 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:25.848111 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:25.848111 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:25.848111 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:25.848111 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:25.848111 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 178/362] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:50:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:25.849506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bce5a8-960c-4e27-9e5b-6603cda40f8d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:25.874863 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 185/363] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:25.875735 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9724383-ff39-4520-b81c-32f07f2653a9 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:25.992852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:25.999154 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bce5a8-960c-4e27-9e5b-6603cda40f8d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:26.008414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9724383-ff39-4520-b81c-32f07f2653a9 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:26.018737 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:26.049515 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add3c8e-c930-47d2-844d-676aa554c3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:26.050204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add3c8e-c930-47d2-844d-676aa554c3d6 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:26.050528 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add3c8e-c930-47d2-844d-676aa554c3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:26.050810 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add3c8e-c930-47d2-844d-676aa554c3d6 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:26.051578 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add3c8e-c930-47d2-844d-676aa554c3d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:26.051892 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add3c8e-c930-47d2-844d-676aa554c3d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:26.052133 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73b251-be93-4fb9-a6ba-b4ed875fa164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:26.052801 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73b251-be93-4fb9-a6ba-b4ed875fa164 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:26.053082 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73b251-be93-4fb9-a6ba-b4ed875fa164 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:26.053305 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add3c8e-c930-47d2-844d-676aa554c3d6 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:26.053475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73b251-be93-4fb9-a6ba-b4ed875fa164 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:26.054333 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73b251-be93-4fb9-a6ba-b4ed875fa164 None admin] RBAC: Adding query filter params (group.id=89c57e2b55954eb1aafdf8513a450951, role.id=2fd6a3e7df18446993c578dab05dfdce, scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:26.055506 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73b251-be93-4fb9-a6ba-b4ed875fa164 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:26.056037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73b251-be93-4fb9-a6ba-b4ed875fa164 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:26.062160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 186/364] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:50:25 2022] GET /identity/v3/role_assignments?group.id=89c57e2b55954eb1aafdf8513a450951&scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18&role.id=2fd6a3e7df18446993c578dab05dfdce => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:26.068614 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73b251-be93-4fb9-a6ba-b4ed875fa164 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:26.068856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 179/365] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:26.070140 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add3c8e-c930-47d2-844d-676aa554c3d6 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:26.190075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73b251-be93-4fb9-a6ba-b4ed875fa164 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:26.202883 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add3c8e-c930-47d2-844d-676aa554c3d6 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:26.230069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083877e5-902d-46c4-b4cf-cf51a10b0e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:26.231459 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083877e5-902d-46c4-b4cf-cf51a10b0e32 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:26.231892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083877e5-902d-46c4-b4cf-cf51a10b0e32 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:26.232339 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083877e5-902d-46c4-b4cf-cf51a10b0e32 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:26.233269 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083877e5-902d-46c4-b4cf-cf51a10b0e32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:26.233766 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083877e5-902d-46c4-b4cf-cf51a10b0e32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:26.236295 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083877e5-902d-46c4-b4cf-cf51a10b0e32 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:26.243318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6f36a-06d2-48eb-bc3b-96e8170c188b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:26.244395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6f36a-06d2-48eb-bc3b-96e8170c188b None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:26.244841 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6f36a-06d2-48eb-bc3b-96e8170c188b None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:26.245368 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6f36a-06d2-48eb-bc3b-96e8170c188b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:26.250636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 187/366] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:26.251641 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083877e5-902d-46c4-b4cf-cf51a10b0e32 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:26.260120 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6f36a-06d2-48eb-bc3b-96e8170c188b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-cfe6f36a-06d2-48eb-bc3b-96e8170c188b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:26.269286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:26.270471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:26.271438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:26.271438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:26.271438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:26.271438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:26.271438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:26.271438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:26.271438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:26.271438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:26.271438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:26.271438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:26.271438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:26.271438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:26.274161 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 180/367] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:50:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:26.275491 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6f36a-06d2-48eb-bc3b-96e8170c188b None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:26.288433 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:26.319247 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:26.398701 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083877e5-902d-46c4-b4cf-cf51a10b0e32 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:26.426212 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6f36a-06d2-48eb-bc3b-96e8170c188b None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:26.462699 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4a4cf-29ee-4ebd-aff7-66d52256f186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:26.463714 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4a4cf-29ee-4ebd-aff7-66d52256f186 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:26.464366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4a4cf-29ee-4ebd-aff7-66d52256f186 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:26.464857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4a4cf-29ee-4ebd-aff7-66d52256f186 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:26.474765 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2768bf-ffc2-4dca-9728-8642e67f4d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:26.475142 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2768bf-ffc2-4dca-9728-8642e67f4d8b None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:26.475480 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2768bf-ffc2-4dca-9728-8642e67f4d8b None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:26.475760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2768bf-ffc2-4dca-9728-8642e67f4d8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:26.547936 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4a4cf-29ee-4ebd-aff7-66d52256f186 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:26.549967 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4a4cf-29ee-4ebd-aff7-66d52256f186 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:26.569590 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2768bf-ffc2-4dca-9728-8642e67f4d8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:26.571492 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2768bf-ffc2-4dca-9728-8642e67f4d8b None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:26.586100 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:26.596771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:26.689946 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 188/368] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:26.691230 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4a4cf-29ee-4ebd-aff7-66d52256f186 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:26.696710 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 181/369] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:26.697659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2768bf-ffc2-4dca-9728-8642e67f4d8b None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:26.828232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2768bf-ffc2-4dca-9728-8642e67f4d8b None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:26.835468 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4a4cf-29ee-4ebd-aff7-66d52256f186 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:26.861173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:26.872407 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:26.880662 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ebdca-f109-42af-a410-af45b02c05df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:26.881350 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ebdca-f109-42af-a410-af45b02c05df None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:26.881629 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ebdca-f109-42af-a410-af45b02c05df None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:26.881898 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ebdca-f109-42af-a410-af45b02c05df None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:26.882690 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ebdca-f109-42af-a410-af45b02c05df None admin] RBAC: Adding query filter params (name=admin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:26.882996 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ebdca-f109-42af-a410-af45b02c05df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:26.884395 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ebdca-f109-42af-a410-af45b02c05df None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:26.887866 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1efa81-e259-4e71-bc0b-f4136d8e192d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:26.888754 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1efa81-e259-4e71-bc0b-f4136d8e192d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:26.889038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1efa81-e259-4e71-bc0b-f4136d8e192d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:26.889317 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1efa81-e259-4e71-bc0b-f4136d8e192d None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:26.893009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1efa81-e259-4e71-bc0b-f4136d8e192d None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:26.894253 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 189/370] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:26.894605 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1efa81-e259-4e71-bc0b-f4136d8e192d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:26.895092 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ebdca-f109-42af-a410-af45b02c05df None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:26.896777 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 182/371] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:26 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:26.901852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1efa81-e259-4e71-bc0b-f4136d8e192d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:27.006186 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ebdca-f109-42af-a410-af45b02c05df None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:27.008396 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1efa81-e259-4e71-bc0b-f4136d8e192d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:27.043924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd2aff-af2e-4552-b56d-f5492c142ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:27.044727 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f3764-1257-4438-81c6-5c5a45346c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:27.044951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd2aff-af2e-4552-b56d-f5492c142ebc None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:27.045188 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd2aff-af2e-4552-b56d-f5492c142ebc None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:27.045461 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd2aff-af2e-4552-b56d-f5492c142ebc None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:27.045626 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f3764-1257-4438-81c6-5c5a45346c93 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:27.045919 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f3764-1257-4438-81c6-5c5a45346c93 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:27.046198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f3764-1257-4438-81c6-5c5a45346c93 None admin] PATH_INFO: `/v3/groups/89c57e2b55954eb1aafdf8513a450951` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:27.046898 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd2aff-af2e-4552-b56d-f5492c142ebc None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:27.047485 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f3764-1257-4438-81c6-5c5a45346c93 None admin] RBAC: Authorizing `identity:get_group(group_id=89c57e2b55954eb1aafdf8513a450951)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:27.048493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd2aff-af2e-4552-b56d-f5492c142ebc None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:27.048927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f3764-1257-4438-81c6-5c5a45346c93 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:27.050551 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 190/372] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:26 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:27.050942 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 183/373] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:50:26 2022] GET /identity/v3/groups/89c57e2b55954eb1aafdf8513a450951 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:27.051267 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd2aff-af2e-4552-b56d-f5492c142ebc None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:27.056223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f3764-1257-4438-81c6-5c5a45346c93 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:27.119502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:27.123559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:27.164990 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd2aff-af2e-4552-b56d-f5492c142ebc None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:27.173734 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f3764-1257-4438-81c6-5c5a45346c93 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:27.201658 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36a09c-9f11-4ec5-8ceb-f92b7acef912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:27.202449 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36a09c-9f11-4ec5-8ceb-f92b7acef912 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:27.202797 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36a09c-9f11-4ec5-8ceb-f92b7acef912 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:27.203298 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36a09c-9f11-4ec5-8ceb-f92b7acef912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:27.213244 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47afb2e-bd59-4097-9245-9c4f7ea6729f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:27.214790 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47afb2e-bd59-4097-9245-9c4f7ea6729f None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:27.215341 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47afb2e-bd59-4097-9245-9c4f7ea6729f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:27.215826 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47afb2e-bd59-4097-9245-9c4f7ea6729f None admin] PATH_INFO: `/v3/groups/89c57e2b55954eb1aafdf8513a450951` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:27.217344 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47afb2e-bd59-4097-9245-9c4f7ea6729f None admin] RBAC: Authorizing `identity:get_group(group_id=89c57e2b55954eb1aafdf8513a450951)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:27.219436 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47afb2e-bd59-4097-9245-9c4f7ea6729f None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:27.221965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 184/374] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:50:27 2022] GET /identity/v3/groups/89c57e2b55954eb1aafdf8513a450951 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:27.223469 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47afb2e-bd59-4097-9245-9c4f7ea6729f None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:27.276921 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36a09c-9f11-4ec5-8ceb-f92b7acef912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:27.279559 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36a09c-9f11-4ec5-8ceb-f92b7acef912 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:27.335822 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47afb2e-bd59-4097-9245-9c4f7ea6729f None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:27.347671 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:27.370664 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:27.376387 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21ac2f-e89f-46b4-96d1-8c8ade1e8a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:27.377069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21ac2f-e89f-46b4-96d1-8c8ade1e8a7d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:27.377383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21ac2f-e89f-46b4-96d1-8c8ade1e8a7d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:27.377854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21ac2f-e89f-46b4-96d1-8c8ade1e8a7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:27.378789 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21ac2f-e89f-46b4-96d1-8c8ade1e8a7d None admin] RBAC: Adding query filter params (group.id=89c57e2b55954eb1aafdf8513a450951, role.id=2fd6a3e7df18446993c578dab05dfdce, scope.project.id=c06a45678b434f629eb93e70012c1738) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:27.379113 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21ac2f-e89f-46b4-96d1-8c8ade1e8a7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:27.380596 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21ac2f-e89f-46b4-96d1-8c8ade1e8a7d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:27.387735 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 185/375] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:50:27 2022] GET /identity/v3/role_assignments?group.id=89c57e2b55954eb1aafdf8513a450951&scope.project.id=c06a45678b434f629eb93e70012c1738&role.id=2fd6a3e7df18446993c578dab05dfdce => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:27.388717 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21ac2f-e89f-46b4-96d1-8c8ade1e8a7d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:27.393088 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 191/376] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:27.398838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36a09c-9f11-4ec5-8ceb-f92b7acef912 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:27.515597 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36a09c-9f11-4ec5-8ceb-f92b7acef912 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:27.517342 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21ac2f-e89f-46b4-96d1-8c8ade1e8a7d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:27.556893 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f8b64-5285-467f-9eb5-416acc4ad9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:27.557726 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f8b64-5285-467f-9eb5-416acc4ad9d1 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:27.558016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f8b64-5285-467f-9eb5-416acc4ad9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:27.558345 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f8b64-5285-467f-9eb5-416acc4ad9d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:27.559220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f8b64-5285-467f-9eb5-416acc4ad9d1 None admin] RBAC: Adding query filter params (group.id=89c57e2b55954eb1aafdf8513a450951, role.id=964e02e925f147e69a9e02e02a69612b, scope.project.id=c06a45678b434f629eb93e70012c1738) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:27.559699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f8b64-5285-467f-9eb5-416acc4ad9d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:27.561332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f8b64-5285-467f-9eb5-416acc4ad9d1 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:27.569489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 186/377] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:50:27 2022] GET /identity/v3/role_assignments?group.id=89c57e2b55954eb1aafdf8513a450951&scope.project.id=c06a45678b434f629eb93e70012c1738&role.id=964e02e925f147e69a9e02e02a69612b => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:27.579180 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f298e8-127a-4fa6-86d8-e10b65f48459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:27.580104 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f298e8-127a-4fa6-86d8-e10b65f48459 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:27.580384 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f298e8-127a-4fa6-86d8-e10b65f48459 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:27.580699 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f298e8-127a-4fa6-86d8-e10b65f48459 None admin] PATH_INFO: `/v3/projects/682345a068d445309abcfb72f27e61bb` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:27.582500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f298e8-127a-4fa6-86d8-e10b65f48459 None admin] RBAC: Authorizing `identity:get_project(project_id=682345a068d445309abcfb72f27e61bb)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:27.584059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f298e8-127a-4fa6-86d8-e10b65f48459 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:27.586310 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 192/378] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:27 2022] GET /identity/v3/projects/682345a068d445309abcfb72f27e61bb => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:27.592196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f8b64-5285-467f-9eb5-416acc4ad9d1 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:27.662827 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:27.722768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f8b64-5285-467f-9eb5-416acc4ad9d1 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:27.761027 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1dcbd9-cfbd-4065-8d39-91ed69caebf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:27.761927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dcbd9-cfbd-4065-8d39-91ed69caebf0 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:27.762212 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dcbd9-cfbd-4065-8d39-91ed69caebf0 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:27.762525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dcbd9-cfbd-4065-8d39-91ed69caebf0 None admin] PATH_INFO: `/v3/groups/e3471e8a871441e2a26520c953e8c780` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:27.770410 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1dcbd9-cfbd-4065-8d39-91ed69caebf0 None admin] RBAC: Authorizing `identity:get_group(group_id=e3471e8a871441e2a26520c953e8c780)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:27.771908 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1dcbd9-cfbd-4065-8d39-91ed69caebf0 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:27.774135 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 187/379] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:50:27 2022] GET /identity/v3/groups/e3471e8a871441e2a26520c953e8c780 => generated 229 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:27.780200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f298e8-127a-4fa6-86d8-e10b65f48459 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:27.835043 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:27.916840 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f298e8-127a-4fa6-86d8-e10b65f48459 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:27.960067 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574d7e7-db16-44d6-aeb9-f907a25822f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:27.960740 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574d7e7-db16-44d6-aeb9-f907a25822f3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:27.961018 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574d7e7-db16-44d6-aeb9-f907a25822f3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:27.961288 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574d7e7-db16-44d6-aeb9-f907a25822f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:27.962145 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574d7e7-db16-44d6-aeb9-f907a25822f3 None admin] RBAC: Adding query filter params (group.id=e3471e8a871441e2a26520c953e8c780, role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=682345a068d445309abcfb72f27e61bb) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:27.962496 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574d7e7-db16-44d6-aeb9-f907a25822f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:27.963953 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574d7e7-db16-44d6-aeb9-f907a25822f3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:27.971522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 193/380] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:50:27 2022] GET /identity/v3/role_assignments?group.id=e3471e8a871441e2a26520c953e8c780&scope.project.id=682345a068d445309abcfb72f27e61bb&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:28.652813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a948b72-9b69-4227-a3d4-a0671752f717 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:28.653358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a948b72-9b69-4227-a3d4-a0671752f717 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:28.653358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a948b72-9b69-4227-a3d4-a0671752f717 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:28.655033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 188/381] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:28.661039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b81cc-3cd3-4a6c-8ffb-4016a4f2e639 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:28.661537 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b81cc-3cd3-4a6c-8ffb-4016a4f2e639 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:28.661631 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b81cc-3cd3-4a6c-8ffb-4016a4f2e639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:28.687321 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-2c8b81cc-3cd3-4a6c-8ffb-4016a4f2e639 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:28.688785 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-2c8b81cc-3cd3-4a6c-8ffb-4016a4f2e639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:28.698432 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6ade1-478f-4f7d-9634-04eb60aa9a8f None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:28.698731 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6ade1-478f-4f7d-9634-04eb60aa9a8f None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:28.698986 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6ade1-478f-4f7d-9634-04eb60aa9a8f None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:28.700620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 189/382] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:50:28.709995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816dae62-0b17-4d2b-a141-4da4ad624c5c None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:28.713413 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816dae62-0b17-4d2b-a141-4da4ad624c5c None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:28.713817 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816dae62-0b17-4d2b-a141-4da4ad624c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:28.735198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-816dae62-0b17-4d2b-a141-4da4ad624c5c None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:28.736842 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-816dae62-0b17-4d2b-a141-4da4ad624c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:28.788744 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 194/383] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:28.811093 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b81cc-3cd3-4a6c-8ffb-4016a4f2e639 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:28.836954 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:28.879983 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 190/384] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:28.913384 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b81cc-3cd3-4a6c-8ffb-4016a4f2e639 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:28.923300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816dae62-0b17-4d2b-a141-4da4ad624c5c None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:28.950038 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc4638-9f7a-4068-95d4-ef6c3f99a4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:28.951577 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc4638-9f7a-4068-95d4-ef6c3f99a4f3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:28.952063 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc4638-9f7a-4068-95d4-ef6c3f99a4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:28.952532 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc4638-9f7a-4068-95d4-ef6c3f99a4f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:28.964290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc4638-9f7a-4068-95d4-ef6c3f99a4f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-32dc4638-9f7a-4068-95d4-ef6c3f99a4f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:28.973709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:28.974781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:28.975710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:28.975710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:28.975710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:28.975710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:28.975710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:28.975710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:28.975710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:28.975710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:28.975710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:28.975710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:28.975710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:28.975710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:28.978582 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 195/385] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:50:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:28.986102 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc4638-9f7a-4068-95d4-ef6c3f99a4f3 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:29.040229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:29.104368 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816dae62-0b17-4d2b-a141-4da4ad624c5c None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:29.171290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9154d-299c-448f-a063-3f768a6d9fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:29.172286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9154d-299c-448f-a063-3f768a6d9fcd None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:29.172614 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9154d-299c-448f-a063-3f768a6d9fcd None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:29.172904 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9154d-299c-448f-a063-3f768a6d9fcd None admin] PATH_INFO: `/v3/roles/member` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:29.176570 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:29.192795 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9154d-299c-448f-a063-3f768a6d9fcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-abb9154d-299c-448f-a063-3f768a6d9fcd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:29.200571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:29.201574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:29.202532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:29.202532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:29.202532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:29.202532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:29.202532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:29.202532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:29.202532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:29.202532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:29.202532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:29.202532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:29.202532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:29.202532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:29.208262 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 191/386] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:29.215235 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc4638-9f7a-4068-95d4-ef6c3f99a4f3 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:29.217427 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9154d-299c-448f-a063-3f768a6d9fcd None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:29.276126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7959d37-ce2e-4dd2-87b3-ed0eb6012625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:29.277211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7959d37-ce2e-4dd2-87b3-ed0eb6012625 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:29.277707 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7959d37-ce2e-4dd2-87b3-ed0eb6012625 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:29.278224 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7959d37-ce2e-4dd2-87b3-ed0eb6012625 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:29.279846 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7959d37-ce2e-4dd2-87b3-ed0eb6012625 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:29.280532 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7959d37-ce2e-4dd2-87b3-ed0eb6012625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:29.284288 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7959d37-ce2e-4dd2-87b3-ed0eb6012625 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:29.298921 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 196/387] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:29.305037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7959d37-ce2e-4dd2-87b3-ed0eb6012625 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:29.405313 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9154d-299c-448f-a063-3f768a6d9fcd None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:29.417266 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:29.443514 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7959d37-ce2e-4dd2-87b3-ed0eb6012625 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:29.484412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c25479-3a23-4b9b-b7fe-e37df83525a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:29.486055 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c25479-3a23-4b9b-b7fe-e37df83525a7 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:29.487351 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c25479-3a23-4b9b-b7fe-e37df83525a7 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:29.488343 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c25479-3a23-4b9b-b7fe-e37df83525a7 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:29.489177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bdd74-2eab-447e-b4ea-dc07e97f10d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:29.490180 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c25479-3a23-4b9b-b7fe-e37df83525a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:29.491301 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bdd74-2eab-447e-b4ea-dc07e97f10d4 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:29.491824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c25479-3a23-4b9b-b7fe-e37df83525a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:29.493443 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c25479-3a23-4b9b-b7fe-e37df83525a7 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:29.493933 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bdd74-2eab-447e-b4ea-dc07e97f10d4 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:29.494446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bdd74-2eab-447e-b4ea-dc07e97f10d4 None admin] PATH_INFO: `/v3/groups/89c57e2b55954eb1aafdf8513a450951` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:29.496015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bdd74-2eab-447e-b4ea-dc07e97f10d4 None admin] RBAC: Authorizing `identity:get_group(group_id=89c57e2b55954eb1aafdf8513a450951)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:29.497687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bdd74-2eab-447e-b4ea-dc07e97f10d4 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:29.500226 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 197/388] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:50:29 2022] GET /identity/v3/groups/89c57e2b55954eb1aafdf8513a450951 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:29.505558 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bdd74-2eab-447e-b4ea-dc07e97f10d4 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:29.509779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 192/389] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:29.525911 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c25479-3a23-4b9b-b7fe-e37df83525a7 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:29.528593 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:29.619194 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bdd74-2eab-447e-b4ea-dc07e97f10d4 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:29.655099 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51991be6-c3d8-422f-a6e0-1147975146ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:29.656564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51991be6-c3d8-422f-a6e0-1147975146ea None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:29.657156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51991be6-c3d8-422f-a6e0-1147975146ea None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:29.657694 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51991be6-c3d8-422f-a6e0-1147975146ea None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:29.659189 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51991be6-c3d8-422f-a6e0-1147975146ea None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:29.660970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51991be6-c3d8-422f-a6e0-1147975146ea None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:29.663616 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 198/390] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:29 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:29.670903 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51991be6-c3d8-422f-a6e0-1147975146ea None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:29.699116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c25479-3a23-4b9b-b7fe-e37df83525a7 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:29.765148 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:29.769508 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bf97c-971b-483f-b495-fe69fdb07ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:29.770707 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bf97c-971b-483f-b495-fe69fdb07ea4 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:29.771183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bf97c-971b-483f-b495-fe69fdb07ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:29.771612 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bf97c-971b-483f-b495-fe69fdb07ea4 None admin] PATH_INFO: `/v3/groups/89c57e2b55954eb1aafdf8513a450951` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:29.779713 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bf97c-971b-483f-b495-fe69fdb07ea4 None admin] RBAC: Authorizing `identity:get_group(group_id=89c57e2b55954eb1aafdf8513a450951)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:29.781696 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bf97c-971b-483f-b495-fe69fdb07ea4 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:29.784081 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 193/391] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:50:29 2022] GET /identity/v3/groups/89c57e2b55954eb1aafdf8513a450951 => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:29.792056 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51991be6-c3d8-422f-a6e0-1147975146ea None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:29.796985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bf97c-971b-483f-b495-fe69fdb07ea4 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:29.831647 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912eeced-d3de-493a-baa2-e7c289365261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:29.832818 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912eeced-d3de-493a-baa2-e7c289365261 None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:29.833200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912eeced-d3de-493a-baa2-e7c289365261 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:29.833576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912eeced-d3de-493a-baa2-e7c289365261 None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18/groups/89c57e2b55954eb1aafdf8513a450951/roles/964e02e925f147e69a9e02e02a69612b` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:29.836010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912eeced-d3de-493a-baa2-e7c289365261 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1c47666fb654b3e9538d26cdb9bdd18, group_id=89c57e2b55954eb1aafdf8513a450951, role_id=964e02e925f147e69a9e02e02a69612b)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:29.840286 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912eeced-d3de-493a-baa2-e7c289365261 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:29.856905 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 199/392] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 19:50:29 2022] PUT /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18/groups/89c57e2b55954eb1aafdf8513a450951/roles/964e02e925f147e69a9e02e02a69612b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:29.876855 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:29.987406 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bf97c-971b-483f-b495-fe69fdb07ea4 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:30.036501 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e32ed9-a001-43b0-86ab-3c9d65df31c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:30.037680 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e32ed9-a001-43b0-86ab-3c9d65df31c0 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:30.038131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e32ed9-a001-43b0-86ab-3c9d65df31c0 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:30.038588 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e32ed9-a001-43b0-86ab-3c9d65df31c0 None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:30.040027 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e32ed9-a001-43b0-86ab-3c9d65df31c0 None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:30.041700 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e32ed9-a001-43b0-86ab-3c9d65df31c0 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:30.043642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 194/393] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:29 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:30.050088 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912eeced-d3de-493a-baa2-e7c289365261 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:30.173553 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912eeced-d3de-493a-baa2-e7c289365261 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:30.211043 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab429ff-831a-49ce-810c-396ad242d82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:30.211964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab429ff-831a-49ce-810c-396ad242d82c None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:30.212241 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab429ff-831a-49ce-810c-396ad242d82c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:30.212535 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab429ff-831a-49ce-810c-396ad242d82c None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18/groups/89c57e2b55954eb1aafdf8513a450951/roles/2fd6a3e7df18446993c578dab05dfdce` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:30.214747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab429ff-831a-49ce-810c-396ad242d82c None admin] RBAC: Authorizing `identity:create_grant(project_id=d1c47666fb654b3e9538d26cdb9bdd18, group_id=89c57e2b55954eb1aafdf8513a450951, role_id=2fd6a3e7df18446993c578dab05dfdce)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:30.216250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab429ff-831a-49ce-810c-396ad242d82c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:30.238542 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 200/394] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 19:50:30 2022] PUT /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18/groups/89c57e2b55954eb1aafdf8513a450951/roles/2fd6a3e7df18446993c578dab05dfdce => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:30.434043 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6d8d5-686d-44bb-baa5-7a30217ad80b None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:30.434520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6d8d5-686d-44bb-baa5-7a30217ad80b None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:30.434644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6d8d5-686d-44bb-baa5-7a30217ad80b None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:30.436186 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 195/395] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:30.441597 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6356136-b622-4a54-b3e1-ad5e63799b01 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:30.441884 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6356136-b622-4a54-b3e1-ad5e63799b01 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:30.442160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6356136-b622-4a54-b3e1-ad5e63799b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:30.447364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:30.462487 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-c6356136-b622-4a54-b3e1-ad5e63799b01 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:30.463697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-c6356136-b622-4a54-b3e1-ad5e63799b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:30.478808 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c466893-4142-422c-b8df-cb3511190d02 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:30.479165 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c466893-4142-422c-b8df-cb3511190d02 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:30.479361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c466893-4142-422c-b8df-cb3511190d02 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:30.480986 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 196/396] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:30.487366 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c89a8-66cd-4cd5-ab80-212ac4c9ce4f None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:30.488440 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c89a8-66cd-4cd5-ab80-212ac4c9ce4f None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:30.489027 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c89a8-66cd-4cd5-ab80-212ac4c9ce4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:30.511997 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-518c89a8-66cd-4cd5-ab80-212ac4c9ce4f None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:30.515307 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-518c89a8-66cd-4cd5-ab80-212ac4c9ce4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:30.539612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 201/397] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:30.563709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6356136-b622-4a54-b3e1-ad5e63799b01 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:30.617242 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 197/398] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:30.650894 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c89a8-66cd-4cd5-ab80-212ac4c9ce4f None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:30.700253 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:30.703896 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6356136-b622-4a54-b3e1-ad5e63799b01 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:30.743659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:30.755938 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29f23a-3563-475e-9065-720865da5f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:30.757322 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29f23a-3563-475e-9065-720865da5f7a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:30.758154 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29f23a-3563-475e-9065-720865da5f7a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:30.758832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29f23a-3563-475e-9065-720865da5f7a None admin] PATH_INFO: `/v3/roles/member` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:30.782437 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29f23a-3563-475e-9065-720865da5f7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-eb29f23a-3563-475e-9065-720865da5f7a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:30.799529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:30.800617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:30.801970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:30.801970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:30.801970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:30.801970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:30.801970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:30.801970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:30.801970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:30.801970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:30.801970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:30.801970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:30.801970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:30.801970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:30.801970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 202/399] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:30.810135 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29f23a-3563-475e-9065-720865da5f7a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:30.838538 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c89a8-66cd-4cd5-ab80-212ac4c9ce4f None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:30.881420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac1c48-abc6-4f27-8f57-136f4e8cdd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:30.882831 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac1c48-abc6-4f27-8f57-136f4e8cdd98 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:30.883615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac1c48-abc6-4f27-8f57-136f4e8cdd98 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:30.884396 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac1c48-abc6-4f27-8f57-136f4e8cdd98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:30.899000 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac1c48-abc6-4f27-8f57-136f4e8cdd98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-1fac1c48-abc6-4f27-8f57-136f4e8cdd98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:30.909434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:30.910546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:30.911571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:30.911571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:30.911571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:30.911571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:30.911571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:30.911571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:30.911571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:30.911571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:30.911571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:30.911571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:30.911571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:30.911571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:30.913290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 198/400] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:50:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:30.913290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66759d-9e4b-47cb-833a-2b1026597cbd None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:30.913604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66759d-9e4b-47cb-833a-2b1026597cbd None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:30.913768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66759d-9e4b-47cb-833a-2b1026597cbd None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:30.918533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 199/401] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:30.919422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66759d-9e4b-47cb-833a-2b1026597cbd None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:30.938040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29f23a-3563-475e-9065-720865da5f7a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:30.982609 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee7880-cba5-499a-abe1-f741026169bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:30.983828 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee7880-cba5-499a-abe1-f741026169bf None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:30.984572 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee7880-cba5-499a-abe1-f741026169bf None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:30.985026 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee7880-cba5-499a-abe1-f741026169bf None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:30.986300 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee7880-cba5-499a-abe1-f741026169bf None admin] RBAC: Adding query filter params (name=member) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:30.987050 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee7880-cba5-499a-abe1-f741026169bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:30.989067 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee7880-cba5-499a-abe1-f741026169bf None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:31.001159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 203/402] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:31.002817 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47658ae-680f-4ddd-accd-ae258914e6b7 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:31.003734 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47658ae-680f-4ddd-accd-ae258914e6b7 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:31.004202 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47658ae-680f-4ddd-accd-ae258914e6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:31.023861 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:31.025826 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-f47658ae-680f-4ddd-accd-ae258914e6b7 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:31.027522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-f47658ae-680f-4ddd-accd-ae258914e6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:31.033416 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66759d-9e4b-47cb-833a-2b1026597cbd None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:31.070714 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c4cbc-e0be-4d8e-8aba-e9f107a555d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:31.072366 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c4cbc-e0be-4d8e-8aba-e9f107a555d4 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:31.072366 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c4cbc-e0be-4d8e-8aba-e9f107a555d4 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:31.072366 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c4cbc-e0be-4d8e-8aba-e9f107a555d4 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:31.073329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c4cbc-e0be-4d8e-8aba-e9f107a555d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:31.073759 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c4cbc-e0be-4d8e-8aba-e9f107a555d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:31.075510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c4cbc-e0be-4d8e-8aba-e9f107a555d4 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:31.087801 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 200/403] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:31.088849 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c4cbc-e0be-4d8e-8aba-e9f107a555d4 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:31.091387 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:31.114893 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 204/404] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:31.116089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47658ae-680f-4ddd-accd-ae258914e6b7 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:31.220885 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47658ae-680f-4ddd-accd-ae258914e6b7 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:31.240097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c4cbc-e0be-4d8e-8aba-e9f107a555d4 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:31.256059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6b3b9-d8eb-4e81-9278-0a32ab427cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:31.256859 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6b3b9-d8eb-4e81-9278-0a32ab427cfe None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:31.257139 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6b3b9-d8eb-4e81-9278-0a32ab427cfe None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:31.257414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6b3b9-d8eb-4e81-9278-0a32ab427cfe None admin] PATH_INFO: `/v3/groups/89c57e2b55954eb1aafdf8513a450951` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:31.258644 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e6b3b9-d8eb-4e81-9278-0a32ab427cfe None admin] RBAC: Authorizing `identity:get_group(group_id=89c57e2b55954eb1aafdf8513a450951)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:31.260108 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e6b3b9-d8eb-4e81-9278-0a32ab427cfe None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:31.261759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 205/405] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:50:31 2022] GET /identity/v3/groups/89c57e2b55954eb1aafdf8513a450951 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:31.262560 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6b3b9-d8eb-4e81-9278-0a32ab427cfe None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:31.285308 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b095af8-d387-408b-96ef-f96766a0b1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:31.286766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b095af8-d387-408b-96ef-f96766a0b1b3 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:31.287390 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b095af8-d387-408b-96ef-f96766a0b1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:31.288203 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b095af8-d387-408b-96ef-f96766a0b1b3 None admin] PATH_INFO: `/v3/groups/89c57e2b55954eb1aafdf8513a450951` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:31.289910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b095af8-d387-408b-96ef-f96766a0b1b3 None admin] RBAC: Authorizing `identity:get_group(group_id=89c57e2b55954eb1aafdf8513a450951)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:31.292279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b095af8-d387-408b-96ef-f96766a0b1b3 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:31.295057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 201/406] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:50:31 2022] GET /identity/v3/groups/89c57e2b55954eb1aafdf8513a450951 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:31.296940 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b095af8-d387-408b-96ef-f96766a0b1b3 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:31.329711 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:31.343239 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:31.376265 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6b3b9-d8eb-4e81-9278-0a32ab427cfe None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:31.416203 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b095af8-d387-408b-96ef-f96766a0b1b3 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:31.426110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6e18c-ce4e-4ee5-a069-e45fc4b47b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:31.427649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6e18c-ce4e-4ee5-a069-e45fc4b47b27 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:31.428511 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6e18c-ce4e-4ee5-a069-e45fc4b47b27 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:31.429260 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6e18c-ce4e-4ee5-a069-e45fc4b47b27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:31.444678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb6e18c-ce4e-4ee5-a069-e45fc4b47b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:31.452918 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f80b82-62cb-45d4-863a-75ec161e9a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:31.454016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f80b82-62cb-45d4-863a-75ec161e9a9d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:31.454575 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f80b82-62cb-45d4-863a-75ec161e9a9d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-5fb6e18c-ce4e-4ee5-a069-e45fc4b47b27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:31.454697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:31.455722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:31.456659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:31.456659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:31.456659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:31.456659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:31.456659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:31.456659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:31.456659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:31.456659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:31.456659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:31.456659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:31.456659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:31.456659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:31.457372 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f80b82-62cb-45d4-863a-75ec161e9a9d None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:31.457372 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f80b82-62cb-45d4-863a-75ec161e9a9d None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:31.457530 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 206/407] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:50:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:31.457530 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6e18c-ce4e-4ee5-a069-e45fc4b47b27 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:31.462768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f80b82-62cb-45d4-863a-75ec161e9a9d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:31.464507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 202/408] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:31 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:31.465316 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f80b82-62cb-45d4-863a-75ec161e9a9d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:31.568611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:31.579876 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f80b82-62cb-45d4-863a-75ec161e9a9d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:31.605338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6e18c-ce4e-4ee5-a069-e45fc4b47b27 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:31.619529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39203965-38f2-43d5-8f19-668d6f92abb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:31.620492 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39203965-38f2-43d5-8f19-668d6f92abb5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:31.620784 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39203965-38f2-43d5-8f19-668d6f92abb5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:31.621069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39203965-38f2-43d5-8f19-668d6f92abb5 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:31.622255 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:31.622394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39203965-38f2-43d5-8f19-668d6f92abb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:31.622946 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39203965-38f2-43d5-8f19-668d6f92abb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:31.624884 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39203965-38f2-43d5-8f19-668d6f92abb5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:31.637716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 203/409] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:31.638662 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39203965-38f2-43d5-8f19-668d6f92abb5 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:31.651941 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6eff4-e474-49bb-ada6-6fb1c0ad07c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:31.653080 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6eff4-e474-49bb-ada6-6fb1c0ad07c9 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:31.653512 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6eff4-e474-49bb-ada6-6fb1c0ad07c9 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:31.654000 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6eff4-e474-49bb-ada6-6fb1c0ad07c9 None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:31.655474 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6eff4-e474-49bb-ada6-6fb1c0ad07c9 None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:31.657601 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6eff4-e474-49bb-ada6-6fb1c0ad07c9 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:31.659817 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 207/410] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:31 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:31.660818 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6eff4-e474-49bb-ada6-6fb1c0ad07c9 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:31.760354 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39203965-38f2-43d5-8f19-668d6f92abb5 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:31.785819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6eff4-e474-49bb-ada6-6fb1c0ad07c9 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:31.798746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61e9d1-6576-4e8b-bb0a-686dc367532d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:31.800247 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61e9d1-6576-4e8b-bb0a-686dc367532d None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:31.801034 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61e9d1-6576-4e8b-bb0a-686dc367532d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:31.801537 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61e9d1-6576-4e8b-bb0a-686dc367532d None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738/groups/89c57e2b55954eb1aafdf8513a450951/roles/964e02e925f147e69a9e02e02a69612b` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:31.805088 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61e9d1-6576-4e8b-bb0a-686dc367532d None admin] RBAC: Authorizing `identity:create_grant(project_id=c06a45678b434f629eb93e70012c1738, group_id=89c57e2b55954eb1aafdf8513a450951, role_id=964e02e925f147e69a9e02e02a69612b)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:31.807142 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61e9d1-6576-4e8b-bb0a-686dc367532d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:31.824617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2196a30-94f8-49ca-b2ab-1cc035b43804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:31.825712 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2196a30-94f8-49ca-b2ab-1cc035b43804 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:31.825881 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2196a30-94f8-49ca-b2ab-1cc035b43804 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:31.826087 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 204/411] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 19:50:31 2022] PUT /identity/v3/projects/c06a45678b434f629eb93e70012c1738/groups/89c57e2b55954eb1aafdf8513a450951/roles/964e02e925f147e69a9e02e02a69612b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:31.826370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2196a30-94f8-49ca-b2ab-1cc035b43804 None admin] PATH_INFO: `/v3/groups/e3471e8a871441e2a26520c953e8c780` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:31.827339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61e9d1-6576-4e8b-bb0a-686dc367532d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:31.827890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2196a30-94f8-49ca-b2ab-1cc035b43804 None admin] RBAC: Authorizing `identity:get_group(group_id=e3471e8a871441e2a26520c953e8c780)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:31.829777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2196a30-94f8-49ca-b2ab-1cc035b43804 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:31.832021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 208/412] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:50:31 2022] GET /identity/v3/groups/e3471e8a871441e2a26520c953e8c780 => generated 229 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:31.837845 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2196a30-94f8-49ca-b2ab-1cc035b43804 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:31.856263 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:31.932243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:32.011164 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61e9d1-6576-4e8b-bb0a-686dc367532d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:32.028860 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2196a30-94f8-49ca-b2ab-1cc035b43804 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:32.071711 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04a145-1896-4595-a233-8cf0ef1e10a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:32.073124 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04a145-1896-4595-a233-8cf0ef1e10a1 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:32.073259 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa90114-9632-4a62-9001-b196876e0b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:32.073765 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04a145-1896-4595-a233-8cf0ef1e10a1 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:32.074132 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa90114-9632-4a62-9001-b196876e0b36 None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:32.074456 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa90114-9632-4a62-9001-b196876e0b36 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:32.074789 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa90114-9632-4a62-9001-b196876e0b36 None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738/groups/89c57e2b55954eb1aafdf8513a450951/roles/2fd6a3e7df18446993c578dab05dfdce` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:32.075017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04a145-1896-4595-a233-8cf0ef1e10a1 None admin] PATH_INFO: `/v3/projects/682345a068d445309abcfb72f27e61bb` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:32.076770 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04a145-1896-4595-a233-8cf0ef1e10a1 None admin] RBAC: Authorizing `identity:get_project(project_id=682345a068d445309abcfb72f27e61bb)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:32.076895 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa90114-9632-4a62-9001-b196876e0b36 None admin] RBAC: Authorizing `identity:create_grant(project_id=c06a45678b434f629eb93e70012c1738, group_id=89c57e2b55954eb1aafdf8513a450951, role_id=2fd6a3e7df18446993c578dab05dfdce)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:32.078421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa90114-9632-4a62-9001-b196876e0b36 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:32.079357 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04a145-1896-4595-a233-8cf0ef1e10a1 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:32.082566 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 209/413] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:31 2022] GET /identity/v3/projects/682345a068d445309abcfb72f27e61bb => generated 344 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:32.088186 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04a145-1896-4595-a233-8cf0ef1e10a1 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:32.095642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 205/414] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 19:50:31 2022] PUT /identity/v3/projects/c06a45678b434f629eb93e70012c1738/groups/89c57e2b55954eb1aafdf8513a450951/roles/2fd6a3e7df18446993c578dab05dfdce => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:32.211342 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04a145-1896-4595-a233-8cf0ef1e10a1 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:32.230392 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:32.261134 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06172be2-f056-4439-97c3-0f91e292a9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:32.262065 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06172be2-f056-4439-97c3-0f91e292a9a3 None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:32.262383 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06172be2-f056-4439-97c3-0f91e292a9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:32.262667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06172be2-f056-4439-97c3-0f91e292a9a3 None admin] PATH_INFO: `/v3/projects/682345a068d445309abcfb72f27e61bb/groups/e3471e8a871441e2a26520c953e8c780/roles/2cd6d3dd613c4824a26cacf53fb6c07f` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:32.266333 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06172be2-f056-4439-97c3-0f91e292a9a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=682345a068d445309abcfb72f27e61bb, group_id=e3471e8a871441e2a26520c953e8c780, role_id=2cd6d3dd613c4824a26cacf53fb6c07f)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:32.267859 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06172be2-f056-4439-97c3-0f91e292a9a3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:32.289186 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 210/415] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 19:50:32 2022] PUT /identity/v3/projects/682345a068d445309abcfb72f27e61bb/groups/e3471e8a871441e2a26520c953e8c780/roles/2cd6d3dd613c4824a26cacf53fb6c07f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:32.683962 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ddf1c-3eae-4c9c-88f3-d69da40733e5 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:32.684393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ddf1c-3eae-4c9c-88f3-d69da40733e5 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:32.684503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ddf1c-3eae-4c9c-88f3-d69da40733e5 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:32.686213 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 206/416] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:32.691768 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99378f7a-d5d7-4171-ab8f-e0d146911782 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:32.692059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99378f7a-d5d7-4171-ab8f-e0d146911782 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:32.692341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99378f7a-d5d7-4171-ab8f-e0d146911782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:32.701971 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7b230-1109-4820-b76b-19a22e0c3438 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:32.702630 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7b230-1109-4820-b76b-19a22e0c3438 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:32.703181 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7b230-1109-4820-b76b-19a22e0c3438 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:32.705063 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 207/417] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:50:32.712220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec82381-73ab-47c3-9fa2-0bacd3474fd6 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:32.712511 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec82381-73ab-47c3-9fa2-0bacd3474fd6 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:32.712792 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec82381-73ab-47c3-9fa2-0bacd3474fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:32.716073 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-99378f7a-d5d7-4171-ab8f-e0d146911782 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:32.717290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-99378f7a-d5d7-4171-ab8f-e0d146911782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:32.745751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-2ec82381-73ab-47c3-9fa2-0bacd3474fd6 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:32.746995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-2ec82381-73ab-47c3-9fa2-0bacd3474fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:32.855257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 211/418] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:32.883121 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99378f7a-d5d7-4171-ab8f-e0d146911782 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:32.897083 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 208/419] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:32.950034 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec82381-73ab-47c3-9fa2-0bacd3474fd6 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:32.957492 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:33.064109 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99378f7a-d5d7-4171-ab8f-e0d146911782 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:33.072849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:33.112979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec82381-73ab-47c3-9fa2-0bacd3474fd6 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:33.131917 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c594eea-6188-49bb-ba68-fc0efc81d97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:33.133113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c594eea-6188-49bb-ba68-fc0efc81d97a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:33.133623 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c594eea-6188-49bb-ba68-fc0efc81d97a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:33.134128 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c594eea-6188-49bb-ba68-fc0efc81d97a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:33.156326 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c594eea-6188-49bb-ba68-fc0efc81d97a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:33.161239 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a5e0d-5ae0-42b8-9bcd-16ba5d02af3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:33.162055 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a5e0d-5ae0-42b8-9bcd-16ba5d02af3c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:33.166298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a5e0d-5ae0-42b8-9bcd-16ba5d02af3c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:33.166298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a5e0d-5ae0-42b8-9bcd-16ba5d02af3c None admin] PATH_INFO: `/v3/roles/member` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-9c594eea-6188-49bb-ba68-fc0efc81d97a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:33.170204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:33.171242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:33.172185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:33.172185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:33.172185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:33.172185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:33.172185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:33.172185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:33.172185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:33.172185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:33.172185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:33.172185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:33.172185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:33.172185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:33.174532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a5e0d-5ae0-42b8-9bcd-16ba5d02af3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:33.175249 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 212/420] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:50:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:33.184002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c594eea-6188-49bb-ba68-fc0efc81d97a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-481a5e0d-5ae0-42b8-9bcd-16ba5d02af3c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:33.195336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:33.196577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:33.197481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:33.197481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:33.197481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:33.197481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:33.197481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:33.197481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:33.197481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:33.197481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:33.197481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:33.197481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:33.197481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:33.197481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:33.197481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 209/421] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:33.204484 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a5e0d-5ae0-42b8-9bcd-16ba5d02af3c None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:33.346305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c594eea-6188-49bb-ba68-fc0efc81d97a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:33.346662 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:33.402932 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a5e0d-5ae0-42b8-9bcd-16ba5d02af3c None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:33.408302 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556f1f4-920f-4398-b7cf-3c728ef95518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:33.408990 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556f1f4-920f-4398-b7cf-3c728ef95518 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:33.409271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556f1f4-920f-4398-b7cf-3c728ef95518 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:33.409579 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556f1f4-920f-4398-b7cf-3c728ef95518 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:33.410355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556f1f4-920f-4398-b7cf-3c728ef95518 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:33.410662 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556f1f4-920f-4398-b7cf-3c728ef95518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:33.412051 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556f1f4-920f-4398-b7cf-3c728ef95518 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:33.431382 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 213/422] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:33.435433 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556f1f4-920f-4398-b7cf-3c728ef95518 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:33.471108 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f622cc-8a43-4f9e-b9d9-2fe0a0cbbefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:33.471728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f622cc-8a43-4f9e-b9d9-2fe0a0cbbefe None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:33.472020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f622cc-8a43-4f9e-b9d9-2fe0a0cbbefe None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:33.472622 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f622cc-8a43-4f9e-b9d9-2fe0a0cbbefe None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:33.473392 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f622cc-8a43-4f9e-b9d9-2fe0a0cbbefe None admin] RBAC: Adding query filter params (name=member) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:33.473714 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f622cc-8a43-4f9e-b9d9-2fe0a0cbbefe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:33.482283 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f622cc-8a43-4f9e-b9d9-2fe0a0cbbefe None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:33.484142 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 210/423] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:33.490026 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f622cc-8a43-4f9e-b9d9-2fe0a0cbbefe None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:33.491879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:33.581847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556f1f4-920f-4398-b7cf-3c728ef95518 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:33.635941 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f622cc-8a43-4f9e-b9d9-2fe0a0cbbefe None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:33.636149 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06854eb9-4f30-4c49-9a62-3edbf97c19c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:33.636722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06854eb9-4f30-4c49-9a62-3edbf97c19c3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:33.637006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06854eb9-4f30-4c49-9a62-3edbf97c19c3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:33.637283 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06854eb9-4f30-4c49-9a62-3edbf97c19c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:33.670722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:33.681032 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e15c5-f20e-40f6-aea5-392e43b95256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:33.681702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e15c5-f20e-40f6-aea5-392e43b95256 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:33.681990 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e15c5-f20e-40f6-aea5-392e43b95256 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:33.682293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e15c5-f20e-40f6-aea5-392e43b95256 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:33.738298 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06854eb9-4f30-4c49-9a62-3edbf97c19c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:33.739846 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06854eb9-4f30-4c49-9a62-3edbf97c19c3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:33.770910 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:33.856888 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 214/424] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:33.862856 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06854eb9-4f30-4c49-9a62-3edbf97c19c3 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:33.891197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e15c5-f20e-40f6-aea5-392e43b95256 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:33.892680 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e15c5-f20e-40f6-aea5-392e43b95256 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:33.970099 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06854eb9-4f30-4c49-9a62-3edbf97c19c3 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:33.998480 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:34.013082 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d468076-5bc8-40d0-ab41-1d09f3cb8342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:34.013966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d468076-5bc8-40d0-ab41-1d09f3cb8342 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:34.014250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d468076-5bc8-40d0-ab41-1d09f3cb8342 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:34.014551 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d468076-5bc8-40d0-ab41-1d09f3cb8342 None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:34.015997 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d468076-5bc8-40d0-ab41-1d09f3cb8342 None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:34.017801 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d468076-5bc8-40d0-ab41-1d09f3cb8342 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:34.024244 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 215/425] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:33 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:34.025685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d468076-5bc8-40d0-ab41-1d09f3cb8342 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:34.130770 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d468076-5bc8-40d0-ab41-1d09f3cb8342 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:34.166161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893966a4-9ab2-4c64-8a57-e7bba5e18459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:34.166811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 211/426] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:50:34.167958 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893966a4-9ab2-4c64-8a57-e7bba5e18459 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:34.168418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893966a4-9ab2-4c64-8a57-e7bba5e18459 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:34.168848 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893966a4-9ab2-4c64-8a57-e7bba5e18459 None admin] PATH_INFO: `/v3/groups/89c57e2b55954eb1aafdf8513a450951` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:34.170342 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893966a4-9ab2-4c64-8a57-e7bba5e18459 None admin] RBAC: Authorizing `identity:get_group(group_id=89c57e2b55954eb1aafdf8513a450951)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:34.172700 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e15c5-f20e-40f6-aea5-392e43b95256 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:34.172931 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893966a4-9ab2-4c64-8a57-e7bba5e18459 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:34.178364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 216/427] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:50:34 2022] GET /identity/v3/groups/89c57e2b55954eb1aafdf8513a450951 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:34.181180 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893966a4-9ab2-4c64-8a57-e7bba5e18459 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:34.252394 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:34.268098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:34.296865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893966a4-9ab2-4c64-8a57-e7bba5e18459 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:34.351861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c44ad-3bd1-4f11-884c-f08effa75ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:34.352809 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c44ad-3bd1-4f11-884c-f08effa75ec1 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:34.353283 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c44ad-3bd1-4f11-884c-f08effa75ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:34.353868 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c44ad-3bd1-4f11-884c-f08effa75ec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:34.355690 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c44ad-3bd1-4f11-884c-f08effa75ec1 None admin] RBAC: Adding query filter params (group.id=89c57e2b55954eb1aafdf8513a450951, role.id=964e02e925f147e69a9e02e02a69612b, scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:34.356407 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c44ad-3bd1-4f11-884c-f08effa75ec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:34.359006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c44ad-3bd1-4f11-884c-f08effa75ec1 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:34.374430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 217/428] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:50:34 2022] GET /identity/v3/role_assignments?group.id=89c57e2b55954eb1aafdf8513a450951&scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18&role.id=964e02e925f147e69a9e02e02a69612b => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:34.404535 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e15c5-f20e-40f6-aea5-392e43b95256 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:34.442111 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263aebc-50af-47c1-a80d-98fcd4c3baf0 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:34.442569 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263aebc-50af-47c1-a80d-98fcd4c3baf0 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:34.442942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263aebc-50af-47c1-a80d-98fcd4c3baf0 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:34.444667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 218/429] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:50:34.450112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9ee28-91ef-4c25-99f0-ed92e3d17d9d None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:34.450564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9ee28-91ef-4c25-99f0-ed92e3d17d9d None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:34.450937 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9ee28-91ef-4c25-99f0-ed92e3d17d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:34.451058 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecc5da-15c4-4110-aa3b-d953056eaf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:34.451832 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecc5da-15c4-4110-aa3b-d953056eaf2a None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:34.452113 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecc5da-15c4-4110-aa3b-d953056eaf2a None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:34.452386 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecc5da-15c4-4110-aa3b-d953056eaf2a None admin] PATH_INFO: `/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:34.453945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecc5da-15c4-4110-aa3b-d953056eaf2a None admin] RBAC: Authorizing `identity:get_project(project_id=d1c47666fb654b3e9538d26cdb9bdd18)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:34.455534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecc5da-15c4-4110-aa3b-d953056eaf2a None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:34.457520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 212/430] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:34 2022] GET /identity/v3/projects/d1c47666fb654b3e9538d26cdb9bdd18 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:34.462640 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecc5da-15c4-4110-aa3b-d953056eaf2a None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:34.473557 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-60a9ee28-91ef-4c25-99f0-ed92e3d17d9d None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:34.474852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-60a9ee28-91ef-4c25-99f0-ed92e3d17d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:34.621544 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 219/431] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:34.639660 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecc5da-15c4-4110-aa3b-d953056eaf2a None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:34.642625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:34.655397 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9ee28-91ef-4c25-99f0-ed92e3d17d9d None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:34.680956 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b244d1-79a5-4f2b-98b5-61dfe4b2f3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:34.681819 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b244d1-79a5-4f2b-98b5-61dfe4b2f3d0 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:34.682097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b244d1-79a5-4f2b-98b5-61dfe4b2f3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:34.682419 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b244d1-79a5-4f2b-98b5-61dfe4b2f3d0 None admin] PATH_INFO: `/v3/groups/89c57e2b55954eb1aafdf8513a450951` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:34.683688 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b244d1-79a5-4f2b-98b5-61dfe4b2f3d0 None admin] RBAC: Authorizing `identity:get_group(group_id=89c57e2b55954eb1aafdf8513a450951)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:34.685154 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b244d1-79a5-4f2b-98b5-61dfe4b2f3d0 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:34.687244 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 213/432] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:50:34 2022] GET /identity/v3/groups/89c57e2b55954eb1aafdf8513a450951 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:34.694892 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b244d1-79a5-4f2b-98b5-61dfe4b2f3d0 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:34.731517 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:34.779901 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9ee28-91ef-4c25-99f0-ed92e3d17d9d None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:34.810180 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b244d1-79a5-4f2b-98b5-61dfe4b2f3d0 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:34.819015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443d00a-6524-4133-9466-4f2fbc77ebf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:34.819848 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443d00a-6524-4133-9466-4f2fbc77ebf7 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:34.820133 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443d00a-6524-4133-9466-4f2fbc77ebf7 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:34.820414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443d00a-6524-4133-9466-4f2fbc77ebf7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:34.832595 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443d00a-6524-4133-9466-4f2fbc77ebf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-7443d00a-6524-4133-9466-4f2fbc77ebf7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:34.841303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:34.842419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:34.843362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:34.843362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:34.843362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:34.843362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:34.843362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:34.843362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:34.843362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:34.843362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:34.843362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:34.843362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:34.843362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:34.843362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:34.844638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 220/433] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:50:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:34.845634 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9f904-c98d-463d-8551-0f353f297d3b None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:34.845929 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9f904-c98d-463d-8551-0f353f297d3b None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:34.846214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9f904-c98d-463d-8551-0f353f297d3b None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:34.848060 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 221/434] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:34.851095 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9f904-c98d-463d-8551-0f353f297d3b None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:34.858232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0080390-6443-484b-bb32-68b4fcf3d713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:34.858987 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0080390-6443-484b-bb32-68b4fcf3d713 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:34.859341 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0080390-6443-484b-bb32-68b4fcf3d713 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:34.859624 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0080390-6443-484b-bb32-68b4fcf3d713 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:34.860483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0080390-6443-484b-bb32-68b4fcf3d713 None admin] RBAC: Adding query filter params (group.id=89c57e2b55954eb1aafdf8513a450951, role.id=2fd6a3e7df18446993c578dab05dfdce, scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:34.860805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0080390-6443-484b-bb32-68b4fcf3d713 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:34.862284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0080390-6443-484b-bb32-68b4fcf3d713 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:34.869211 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 214/435] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:50:34 2022] GET /identity/v3/role_assignments?group.id=89c57e2b55954eb1aafdf8513a450951&scope.project.id=d1c47666fb654b3e9538d26cdb9bdd18&role.id=2fd6a3e7df18446993c578dab05dfdce => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:34.870343 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec5406-c731-4a28-b5e7-b0ee5e2a14e3 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:34.870951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec5406-c731-4a28-b5e7-b0ee5e2a14e3 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:34.870951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec5406-c731-4a28-b5e7-b0ee5e2a14e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:34.888750 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-a4ec5406-c731-4a28-b5e7-b0ee5e2a14e3 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:34.889916 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-a4ec5406-c731-4a28-b5e7-b0ee5e2a14e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:34.914699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:34.974171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9f904-c98d-463d-8551-0f353f297d3b None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:34.976699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 215/436] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:34.994684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:34.998674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec5406-c731-4a28-b5e7-b0ee5e2a14e3 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:35.014118 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c956ed-153a-49eb-857b-74a205cb8f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:35.015147 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c956ed-153a-49eb-857b-74a205cb8f82 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:35.015685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c956ed-153a-49eb-857b-74a205cb8f82 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:35.016113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c956ed-153a-49eb-857b-74a205cb8f82 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:35.017055 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c956ed-153a-49eb-857b-74a205cb8f82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:35.017523 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c956ed-153a-49eb-857b-74a205cb8f82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:35.019151 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c956ed-153a-49eb-857b-74a205cb8f82 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:35.028491 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 222/437] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:35.034574 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c956ed-153a-49eb-857b-74a205cb8f82 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:35.123871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec5406-c731-4a28-b5e7-b0ee5e2a14e3 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:35.164962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c956ed-153a-49eb-857b-74a205cb8f82 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:35.171205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07da39f-ff72-4652-bc03-e88860f2b1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:35.172162 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07da39f-ff72-4652-bc03-e88860f2b1cb None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:35.172451 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07da39f-ff72-4652-bc03-e88860f2b1cb None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:35.172710 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07da39f-ff72-4652-bc03-e88860f2b1cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:35.185965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07da39f-ff72-4652-bc03-e88860f2b1cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-a07da39f-ff72-4652-bc03-e88860f2b1cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:35.194066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:35.195358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:35.196490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:35.196490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:35.196490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:35.196490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:35.196490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:35.196490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:35.196490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:35.196490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:35.196490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:35.196490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:35.196490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:50:35.196490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:35.198975 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 216/438] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:35.205153 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03ecea-df7f-4028-9859-cd70053c5825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:35.205746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07da39f-ff72-4652-bc03-e88860f2b1cb None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:35.205882 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03ecea-df7f-4028-9859-cd70053c5825 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:35.206089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03ecea-df7f-4028-9859-cd70053c5825 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:35.206405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03ecea-df7f-4028-9859-cd70053c5825 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:35.226612 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:35.266442 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:35.292375 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03ecea-df7f-4028-9859-cd70053c5825 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:35.293905 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03ecea-df7f-4028-9859-cd70053c5825 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:35.346583 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07da39f-ff72-4652-bc03-e88860f2b1cb None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:35.387666 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ced7f8-be55-4c4e-8189-e63a33ba179c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:35.388693 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ced7f8-be55-4c4e-8189-e63a33ba179c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:35.389010 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ced7f8-be55-4c4e-8189-e63a33ba179c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:35.389285 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ced7f8-be55-4c4e-8189-e63a33ba179c None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:35.389996 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ced7f8-be55-4c4e-8189-e63a33ba179c None admin] RBAC: Adding query filter params (name=member) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:35.390363 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ced7f8-be55-4c4e-8189-e63a33ba179c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:35.391814 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ced7f8-be55-4c4e-8189-e63a33ba179c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:35.400931 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 217/439] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:50:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:35.402400 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224deeff-ae5c-48eb-bb56-bbda7d39bf94 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:35.402780 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224deeff-ae5c-48eb-bb56-bbda7d39bf94 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:35.402986 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224deeff-ae5c-48eb-bb56-bbda7d39bf94 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:35.404338 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 218/440] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:35.406550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 223/441] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:50:35.409794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224deeff-ae5c-48eb-bb56-bbda7d39bf94 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:35.410393 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0593a-ef7c-4fec-ba20-c822182ed8a6 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:35.410687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0593a-ef7c-4fec-ba20-c822182ed8a6 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:35.410960 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0593a-ef7c-4fec-ba20-c822182ed8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:35.428352 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-8ab0593a-ef7c-4fec-ba20-c822182ed8a6 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:35.429477 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-8ab0593a-ef7c-4fec-ba20-c822182ed8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:35.509668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 224/442] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:35.510380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0593a-ef7c-4fec-ba20-c822182ed8a6 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:35.519074 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:35.525135 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:35.545069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224deeff-ae5c-48eb-bb56-bbda7d39bf94 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:35.582496 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7441d-5fb8-4b96-863d-a6efbd47ea78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:35.583151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7441d-5fb8-4b96-863d-a6efbd47ea78 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:35.583432 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7441d-5fb8-4b96-863d-a6efbd47ea78 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:35.583704 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7441d-5fb8-4b96-863d-a6efbd47ea78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:35.617859 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0593a-ef7c-4fec-ba20-c822182ed8a6 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:35.649931 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7441d-5fb8-4b96-863d-a6efbd47ea78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:35.651428 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7441d-5fb8-4b96-863d-a6efbd47ea78 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:35.651768 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae69f2a-14b5-477c-a872-9a347c7c3da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:35.652641 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae69f2a-14b5-477c-a872-9a347c7c3da6 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:35.652902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae69f2a-14b5-477c-a872-9a347c7c3da6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:35.653186 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae69f2a-14b5-477c-a872-9a347c7c3da6 None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:35.654502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae69f2a-14b5-477c-a872-9a347c7c3da6 None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:35.656095 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae69f2a-14b5-477c-a872-9a347c7c3da6 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:35.658016 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 225/443] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:35 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:35.658632 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae69f2a-14b5-477c-a872-9a347c7c3da6 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:35.729021 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:35.741116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:35.751674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 219/444] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:35.752507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7441d-5fb8-4b96-863d-a6efbd47ea78 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:35.767190 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae69f2a-14b5-477c-a872-9a347c7c3da6 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:35.804310 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c551939-1e9c-4346-beca-71fed987b9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:35.805437 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c551939-1e9c-4346-beca-71fed987b9a2 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:35.805839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c551939-1e9c-4346-beca-71fed987b9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:35.806325 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c551939-1e9c-4346-beca-71fed987b9a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:35.818946 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c551939-1e9c-4346-beca-71fed987b9a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-7c551939-1e9c-4346-beca-71fed987b9a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:35.826657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:35.827716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:35.828772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:35.828772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:35.828772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:35.828772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:35.828772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:35.828772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:35.828772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:35.828772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:35.828772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:35.828772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:35.828772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:35.828772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:35.828772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 226/445] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:50:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:35.829560 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c551939-1e9c-4346-beca-71fed987b9a2 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:35.855960 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7441d-5fb8-4b96-863d-a6efbd47ea78 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:35.891645 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c15091-538d-4229-9172-1576efc408ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:35.892553 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c15091-538d-4229-9172-1576efc408ff None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:35.892837 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c15091-538d-4229-9172-1576efc408ff None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:35.893107 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c15091-538d-4229-9172-1576efc408ff None admin] PATH_INFO: `/v3/groups/89c57e2b55954eb1aafdf8513a450951` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:35.894348 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c15091-538d-4229-9172-1576efc408ff None admin] RBAC: Authorizing `identity:get_group(group_id=89c57e2b55954eb1aafdf8513a450951)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:35.895807 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c15091-538d-4229-9172-1576efc408ff None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:35.897450 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 220/446] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:50:35 2022] GET /identity/v3/groups/89c57e2b55954eb1aafdf8513a450951 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:35.898208 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c15091-538d-4229-9172-1576efc408ff None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:35.931908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c551939-1e9c-4346-beca-71fed987b9a2 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:35.948011 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:35.966621 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b599f-a4af-4624-823a-fee47be92c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:35.967426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b599f-a4af-4624-823a-fee47be92c35 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:35.967708 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b599f-a4af-4624-823a-fee47be92c35 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:35.967981 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b599f-a4af-4624-823a-fee47be92c35 None admin] PATH_INFO: `/v3/projects/c06a45678b434f629eb93e70012c1738` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:35.969475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b599f-a4af-4624-823a-fee47be92c35 None admin] RBAC: Authorizing `identity:get_project(project_id=c06a45678b434f629eb93e70012c1738)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:35.970973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b599f-a4af-4624-823a-fee47be92c35 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:35.973004 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 227/447] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:35 2022] GET /identity/v3/projects/c06a45678b434f629eb93e70012c1738 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:35.973764 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b599f-a4af-4624-823a-fee47be92c35 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:35.983050 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:36.002733 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c15091-538d-4229-9172-1576efc408ff None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:36.037133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51dfbd-d06a-4afd-bfe1-9da6bedb5c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:36.037801 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51dfbd-d06a-4afd-bfe1-9da6bedb5c29 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:36.038078 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51dfbd-d06a-4afd-bfe1-9da6bedb5c29 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:36.038382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51dfbd-d06a-4afd-bfe1-9da6bedb5c29 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:36.039139 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51dfbd-d06a-4afd-bfe1-9da6bedb5c29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:36.039447 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51dfbd-d06a-4afd-bfe1-9da6bedb5c29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:36.040852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51dfbd-d06a-4afd-bfe1-9da6bedb5c29 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:36.049158 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 221/448] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:36.049939 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51dfbd-d06a-4afd-bfe1-9da6bedb5c29 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:36.079421 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b599f-a4af-4624-823a-fee47be92c35 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:36.115537 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7264e96-4971-46d2-8bdc-56ab8e4f2a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:36.116221 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7264e96-4971-46d2-8bdc-56ab8e4f2a10 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:36.116494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7264e96-4971-46d2-8bdc-56ab8e4f2a10 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:36.116765 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7264e96-4971-46d2-8bdc-56ab8e4f2a10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:36.117617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7264e96-4971-46d2-8bdc-56ab8e4f2a10 None admin] RBAC: Adding query filter params (group.id=89c57e2b55954eb1aafdf8513a450951, role.id=964e02e925f147e69a9e02e02a69612b, scope.project.id=c06a45678b434f629eb93e70012c1738) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:36.117926 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7264e96-4971-46d2-8bdc-56ab8e4f2a10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:36.119398 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7264e96-4971-46d2-8bdc-56ab8e4f2a10 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:36.125278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 228/449] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:50:35 2022] GET /identity/v3/role_assignments?group.id=89c57e2b55954eb1aafdf8513a450951&scope.project.id=c06a45678b434f629eb93e70012c1738&role.id=964e02e925f147e69a9e02e02a69612b => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:36.126040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7264e96-4971-46d2-8bdc-56ab8e4f2a10 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:36.162801 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51dfbd-d06a-4afd-bfe1-9da6bedb5c29 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:36.173623 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:36.206833 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720b580-3994-41c1-b4d4-2c6cabf52e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:36.207240 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:36.208284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720b580-3994-41c1-b4d4-2c6cabf52e69 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:36.208786 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720b580-3994-41c1-b4d4-2c6cabf52e69 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:36.209242 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720b580-3994-41c1-b4d4-2c6cabf52e69 None admin] PATH_INFO: `/v3/groups/89c57e2b55954eb1aafdf8513a450951` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:36.210616 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720b580-3994-41c1-b4d4-2c6cabf52e69 None admin] RBAC: Authorizing `identity:get_group(group_id=89c57e2b55954eb1aafdf8513a450951)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:36.212134 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720b580-3994-41c1-b4d4-2c6cabf52e69 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:36.214037 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 222/450] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:50:36 2022] GET /identity/v3/groups/89c57e2b55954eb1aafdf8513a450951 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:36.220317 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720b580-3994-41c1-b4d4-2c6cabf52e69 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:36.244358 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7264e96-4971-46d2-8bdc-56ab8e4f2a10 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:36.284841 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf22c0-9752-4dff-86b8-a9d4808f4ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:36.285625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf22c0-9752-4dff-86b8-a9d4808f4ef9 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:36.285913 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf22c0-9752-4dff-86b8-a9d4808f4ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:36.286193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf22c0-9752-4dff-86b8-a9d4808f4ef9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:36.330777 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720b580-3994-41c1-b4d4-2c6cabf52e69 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:36.367850 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf22c0-9752-4dff-86b8-a9d4808f4ef9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:36.369468 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf22c0-9752-4dff-86b8-a9d4808f4ef9 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:36.373748 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a67a1d-c333-47a1-8cc3-bb202ddc35f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:36.374460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a67a1d-c333-47a1-8cc3-bb202ddc35f2 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:36.374772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a67a1d-c333-47a1-8cc3-bb202ddc35f2 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:36.375234 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a67a1d-c333-47a1-8cc3-bb202ddc35f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:36.376089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a67a1d-c333-47a1-8cc3-bb202ddc35f2 None admin] RBAC: Adding query filter params (group.id=89c57e2b55954eb1aafdf8513a450951, role.id=2fd6a3e7df18446993c578dab05dfdce, scope.project.id=c06a45678b434f629eb93e70012c1738) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:36.376408 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a67a1d-c333-47a1-8cc3-bb202ddc35f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:36.378301 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a67a1d-c333-47a1-8cc3-bb202ddc35f2 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:36.385531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 223/451] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:50:36 2022] GET /identity/v3/role_assignments?group.id=89c57e2b55954eb1aafdf8513a450951&scope.project.id=c06a45678b434f629eb93e70012c1738&role.id=2fd6a3e7df18446993c578dab05dfdce => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:36.441580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:36.479028 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 229/452] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:36.485674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a67a1d-c333-47a1-8cc3-bb202ddc35f2 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:36.533514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:36.594644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a67a1d-c333-47a1-8cc3-bb202ddc35f2 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:36.629399 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f17225-22b1-407a-bfd4-b36378b78e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:36.630334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f17225-22b1-407a-bfd4-b36378b78e61 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:36.630594 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f17225-22b1-407a-bfd4-b36378b78e61 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:36.630869 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f17225-22b1-407a-bfd4-b36378b78e61 None admin] PATH_INFO: `/v3/projects/682345a068d445309abcfb72f27e61bb` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:36.632097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f17225-22b1-407a-bfd4-b36378b78e61 None admin] RBAC: Authorizing `identity:get_project(project_id=682345a068d445309abcfb72f27e61bb)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:36.633564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f17225-22b1-407a-bfd4-b36378b78e61 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:36.635648 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 224/453] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:50:36 2022] GET /identity/v3/projects/682345a068d445309abcfb72f27e61bb => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:36.641256 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf22c0-9752-4dff-86b8-a9d4808f4ef9 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:36.746176 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf22c0-9752-4dff-86b8-a9d4808f4ef9 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:36.780486 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f5fbb-d2db-4f7b-b245-2b4642da58e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:36.781701 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f5fbb-d2db-4f7b-b245-2b4642da58e7 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:36.781922 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f5fbb-d2db-4f7b-b245-2b4642da58e7 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:36.782253 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f5fbb-d2db-4f7b-b245-2b4642da58e7 None admin] PATH_INFO: `/v3/groups/e3471e8a871441e2a26520c953e8c780` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:36.783515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8f5fbb-d2db-4f7b-b245-2b4642da58e7 None admin] RBAC: Authorizing `identity:get_group(group_id=e3471e8a871441e2a26520c953e8c780)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:36.784954 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8f5fbb-d2db-4f7b-b245-2b4642da58e7 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:36.786909 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 230/454] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 19:50:36 2022] GET /identity/v3/groups/e3471e8a871441e2a26520c953e8c780 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:36.792866 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f17225-22b1-407a-bfd4-b36378b78e61 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:36.899024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f17225-22b1-407a-bfd4-b36378b78e61 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:36.908334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:36.933693 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df28d9-b407-4762-ab9a-247d24d0fb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:36.934389 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df28d9-b407-4762-ab9a-247d24d0fb59 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:36.934679 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df28d9-b407-4762-ab9a-247d24d0fb59 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:36.934957 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df28d9-b407-4762-ab9a-247d24d0fb59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:36.935804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df28d9-b407-4762-ab9a-247d24d0fb59 None admin] RBAC: Adding query filter params (group.id=e3471e8a871441e2a26520c953e8c780, role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=682345a068d445309abcfb72f27e61bb) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:36.936120 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df28d9-b407-4762-ab9a-247d24d0fb59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:36.938515 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df28d9-b407-4762-ab9a-247d24d0fb59 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:36.946654 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 225/455] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 19:50:36 2022] GET /identity/v3/role_assignments?group.id=e3471e8a871441e2a26520c953e8c780&scope.project.id=682345a068d445309abcfb72f27e61bb&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:39.660251 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc04ac1-519c-4cbf-a53c-3bd971ba6a33 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:39.660844 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc04ac1-519c-4cbf-a53c-3bd971ba6a33 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:39.660844 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc04ac1-519c-4cbf-a53c-3bd971ba6a33 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:39.662487 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 231/456] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:39.668542 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d5d32-0804-4ce1-a5d2-ad4ba04b6ac1 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:39.669093 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d5d32-0804-4ce1-a5d2-ad4ba04b6ac1 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:39.669196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d5d32-0804-4ce1-a5d2-ad4ba04b6ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:39.695427 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-5c7d5d32-0804-4ce1-a5d2-ad4ba04b6ac1 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:39.696653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-5c7d5d32-0804-4ce1-a5d2-ad4ba04b6ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:39.800621 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 226/457] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:39.838691 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc04ac1-519c-4cbf-a53c-3bd971ba6a33 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:39.901787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:39.963124 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0a655-4882-4c79-9ab1-42265043c272 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:39.963336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0a655-4882-4c79-9ab1-42265043c272 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:39.963591 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0a655-4882-4c79-9ab1-42265043c272 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:39.965234 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 227/458] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:39.971128 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47827e66-6e3b-436b-bdfe-6d7d0427c70c None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:39.971413 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47827e66-6e3b-436b-bdfe-6d7d0427c70c None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:39.971693 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47827e66-6e3b-436b-bdfe-6d7d0427c70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:39.993954 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc04ac1-519c-4cbf-a53c-3bd971ba6a33 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:39.998482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-47827e66-6e3b-436b-bdfe-6d7d0427c70c None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:39.999830 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-47827e66-6e3b-436b-bdfe-6d7d0427c70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:40.066998 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbca969-f761-4bad-9ff7-6f17f76794a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:40.068264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbca969-f761-4bad-9ff7-6f17f76794a6 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.068753 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbca969-f761-4bad-9ff7-6f17f76794a6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.069070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 228/459] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:40.069465 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbca969-f761-4bad-9ff7-6f17f76794a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.081128 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbca969-f761-4bad-9ff7-6f17f76794a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:40.083007 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbca969-f761-4bad-9ff7-6f17f76794a6 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:40.102499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-4fbca969-f761-4bad-9ff7-6f17f76794a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:40.102499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:40.102499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:40.102499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:40.102499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:40.102499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:40.102499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:40.102499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:40.102499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:40.102499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:40.102499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:40.102499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:40.102499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:40.102499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:40.102499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:40.103525 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:40.104492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:40.104492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:40.104492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:40.104492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:40.107169 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 232/460] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:40.113072 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47827e66-6e3b-436b-bdfe-6d7d0427c70c None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:40.113427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbca969-f761-4bad-9ff7-6f17f76794a6 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:40.166627 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:40.208649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:40.225921 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbca969-f761-4bad-9ff7-6f17f76794a6 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:40.226609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47827e66-6e3b-436b-bdfe-6d7d0427c70c None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:40.273465 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b617b-0e0c-4be6-ad07-5bb29390c44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:40.274503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b617b-0e0c-4be6-ad07-5bb29390c44d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.274879 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b617b-0e0c-4be6-ad07-5bb29390c44d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.275249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b617b-0e0c-4be6-ad07-5bb29390c44d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.277681 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b0374-1578-4a79-bee2-f3c2ac919552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:40.278807 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b0374-1578-4a79-bee2-f3c2ac919552 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.279318 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b0374-1578-4a79-bee2-f3c2ac919552 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.279823 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b0374-1578-4a79-bee2-f3c2ac919552 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.280816 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b0374-1578-4a79-bee2-f3c2ac919552 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:40.281545 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b0374-1578-4a79-bee2-f3c2ac919552 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:40.283399 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b0374-1578-4a79-bee2-f3c2ac919552 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:40.288277 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b617b-0e0c-4be6-ad07-5bb29390c44d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:40.289776 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b617b-0e0c-4be6-ad07-5bb29390c44d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:40.302636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 233/461] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:40.304632 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8b352-0346-4379-a64c-0906acadab84 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.305420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8b352-0346-4379-a64c-0906acadab84 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.305972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8b352-0346-4379-a64c-0906acadab84 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.307997 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 234/462] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:40.308886 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8b352-0346-4379-a64c-0906acadab84 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:40.311610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-b93b617b-0e0c-4be6-ad07-5bb29390c44d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:40.311610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:40.311610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:40.311610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:40.311610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:40.311610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:40.311610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:40.311610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:40.311610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:40.311610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:40.311610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:40.311610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:40.311610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:40.311610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:40.311610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:40.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:40.313705 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:40.313705 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:40.313705 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:40.313705 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:40.313705 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 229/463] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:40.314649 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf4ff9-dbaa-49fd-a5cc-cf2532ccc40e None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.314961 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf4ff9-dbaa-49fd-a5cc-cf2532ccc40e None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.315606 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf4ff9-dbaa-49fd-a5cc-cf2532ccc40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.336924 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-51bf4ff9-dbaa-49fd-a5cc-cf2532ccc40e None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:40.338276 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-51bf4ff9-dbaa-49fd-a5cc-cf2532ccc40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:40.428365 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8b352-0346-4379-a64c-0906acadab84 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:40.435216 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 230/464] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:40.436137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf4ff9-dbaa-49fd-a5cc-cf2532ccc40e None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:40.457703 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:40.469074 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311519eb-86e4-4582-800b-0fdbe89c46c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:40.469747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311519eb-86e4-4582-800b-0fdbe89c46c1 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.470034 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311519eb-86e4-4582-800b-0fdbe89c46c1 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.470328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311519eb-86e4-4582-800b-0fdbe89c46c1 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.471045 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311519eb-86e4-4582-800b-0fdbe89c46c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:40.472822 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311519eb-86e4-4582-800b-0fdbe89c46c1 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:40.496777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 235/465] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:50:40 2022] POST /identity/v3/users => generated 246 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:50:40.498818 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba265b2-a3fa-4d13-897d-3b8acc9e355b None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.499249 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba265b2-a3fa-4d13-897d-3b8acc9e355b None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.499977 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba265b2-a3fa-4d13-897d-3b8acc9e355b None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.501906 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 236/466] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:40.502683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba265b2-a3fa-4d13-897d-3b8acc9e355b None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:40.521237 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:40.560146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf4ff9-dbaa-49fd-a5cc-cf2532ccc40e None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:40.599572 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e219959-7600-4aec-a9eb-86e2a179e600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:40.600760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e219959-7600-4aec-a9eb-86e2a179e600 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.601255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e219959-7600-4aec-a9eb-86e2a179e600 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.601797 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e219959-7600-4aec-a9eb-86e2a179e600 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.602836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e219959-7600-4aec-a9eb-86e2a179e600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:40.603372 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e219959-7600-4aec-a9eb-86e2a179e600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:40.605242 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e219959-7600-4aec-a9eb-86e2a179e600 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:40.619356 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 231/467] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:40.621197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e4f70-ed89-4746-a56c-f502208421d3 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.621696 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e4f70-ed89-4746-a56c-f502208421d3 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.622184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e4f70-ed89-4746-a56c-f502208421d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.639759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba265b2-a3fa-4d13-897d-3b8acc9e355b None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:40.640623 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-391e4f70-ed89-4746-a56c-f502208421d3 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:40.642282 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-391e4f70-ed89-4746-a56c-f502208421d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:40.682856 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2da5b-eaf7-450d-a385-4d3351390dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:40.683701 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2da5b-eaf7-450d-a385-4d3351390dbf None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.683987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2da5b-eaf7-450d-a385-4d3351390dbf None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.684263 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2da5b-eaf7-450d-a385-4d3351390dbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.697108 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2da5b-eaf7-450d-a385-4d3351390dbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:40.698800 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2da5b-eaf7-450d-a385-4d3351390dbf None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:40.718234 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 232/468] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:40.719193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e4f70-ed89-4746-a56c-f502208421d3 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:40.719303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-3ac2da5b-eaf7-450d-a385-4d3351390dbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:40.719303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:40.719303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:40.719303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:40.719303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:40.719303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:40.719303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:40.719303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:40.719303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:40.719303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:40.719303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:40.719303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:40.719303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:40.719303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:40.719303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:40.720203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:40.721183 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:40.721183 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:40.721183 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:40.721183 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:40.721183 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 237/469] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:40.726646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2da5b-eaf7-450d-a385-4d3351390dbf None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:40.728684 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:40.801870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:40.851871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e4f70-ed89-4746-a56c-f502208421d3 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:40.852992 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2da5b-eaf7-450d-a385-4d3351390dbf None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:40.895160 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ccce1-9bde-4146-889e-7fd27464cd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:40.895849 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ccce1-9bde-4146-889e-7fd27464cd29 None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.896128 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ccce1-9bde-4146-889e-7fd27464cd29 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.896401 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ccce1-9bde-4146-889e-7fd27464cd29 None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.896961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4a4a9-2de8-4c76-a320-78f5a8652143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:40.897098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ccce1-9bde-4146-889e-7fd27464cd29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:40.897954 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4a4a9-2de8-4c76-a320-78f5a8652143 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.898482 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4a4a9-2de8-4c76-a320-78f5a8652143 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.898609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ccce1-9bde-4146-889e-7fd27464cd29 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:40.899081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4a4a9-2de8-4c76-a320-78f5a8652143 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.900117 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4a4a9-2de8-4c76-a320-78f5a8652143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:40.900634 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4a4a9-2de8-4c76-a320-78f5a8652143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:40.902317 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4a4a9-2de8-4c76-a320-78f5a8652143 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:40.919140 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 238/470] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:40.920349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4a4a9-2de8-4c76-a320-78f5a8652143 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:40.928158 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 233/471] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:50:40 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:50:40.929142 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ccce1-9bde-4146-889e-7fd27464cd29 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:41.021274 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:41.063326 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4a4a9-2de8-4c76-a320-78f5a8652143 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:41.073049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ccce1-9bde-4146-889e-7fd27464cd29 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:41.092389 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:41.114946 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2146f3-059b-41a3-ad08-c5c90dd88f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:41.116345 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2146f3-059b-41a3-ad08-c5c90dd88f8b None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:41.116890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a0c2e-21fc-44ad-957f-4ef65f2c9b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:41.117202 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2146f3-059b-41a3-ad08-c5c90dd88f8b None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:41.117705 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a0c2e-21fc-44ad-957f-4ef65f2c9b5b None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:41.117954 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2146f3-059b-41a3-ad08-c5c90dd88f8b None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:41.118062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a0c2e-21fc-44ad-957f-4ef65f2c9b5b None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:41.118244 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a0c2e-21fc-44ad-957f-4ef65f2c9b5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:41.119437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2146f3-059b-41a3-ad08-c5c90dd88f8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:41.121470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2146f3-059b-41a3-ad08-c5c90dd88f8b None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:41.130981 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a0c2e-21fc-44ad-957f-4ef65f2c9b5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:41.133564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a0c2e-21fc-44ad-957f-4ef65f2c9b5b None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:41.149928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 234/472] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:50:40 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:50:41.151808 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1eb28f-e566-4d7a-a670-2c52893fa749 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:41.152397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1eb28f-e566-4d7a-a670-2c52893fa749 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:41.152892 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1eb28f-e566-4d7a-a670-2c52893fa749 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:41.154931 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 235/473] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:41.160787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-c21a0c2e-21fc-44ad-957f-4ef65f2c9b5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:41.160787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:41.160787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:41.160787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:41.160787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:41.160787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:41.160787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:41.160787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:41.160787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:41.160787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:41.160787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:41.160787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:41.160787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:41.160787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:41.160787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:41.161871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:41.163030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:41.163030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:41.163030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:41.163030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:41.163030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 239/474] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:41.165282 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495524c-d660-40cb-915c-1bcee51d84ea None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:41.165757 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495524c-d660-40cb-915c-1bcee51d84ea None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:41.166195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495524c-d660-40cb-915c-1bcee51d84ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:41.168152 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a0c2e-21fc-44ad-957f-4ef65f2c9b5b None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:41.185548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-4495524c-d660-40cb-915c-1bcee51d84ea None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:41.186890 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-4495524c-d660-40cb-915c-1bcee51d84ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:41.262138 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 236/475] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:41.285795 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495524c-d660-40cb-915c-1bcee51d84ea None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:41.293850 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a0c2e-21fc-44ad-957f-4ef65f2c9b5b None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:41.344531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db8330-b868-4a4f-8fcd-28eccf214123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:41.345140 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db8330-b868-4a4f-8fcd-28eccf214123 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:41.345424 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db8330-b868-4a4f-8fcd-28eccf214123 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:41.345699 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db8330-b868-4a4f-8fcd-28eccf214123 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:41.347498 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db8330-b868-4a4f-8fcd-28eccf214123 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:41.347813 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db8330-b868-4a4f-8fcd-28eccf214123 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:41.349681 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db8330-b868-4a4f-8fcd-28eccf214123 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:41.353730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:41.363924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 240/476] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:41.369555 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db8330-b868-4a4f-8fcd-28eccf214123 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:41.403768 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:41.418678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495524c-d660-40cb-915c-1bcee51d84ea None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:41.457751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3b714-9d98-4aa5-8035-c3847accd1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:41.458774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3b714-9d98-4aa5-8035-c3847accd1f6 None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:41.459230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3b714-9d98-4aa5-8035-c3847accd1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:41.459654 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3b714-9d98-4aa5-8035-c3847accd1f6 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:41.460581 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb3b714-9d98-4aa5-8035-c3847accd1f6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:41.462143 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb3b714-9d98-4aa5-8035-c3847accd1f6 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:41.470741 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.sql.core [None req-4cb3b714-9d98-4aa5-8035-c3847accd1f6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 19:50:41.470741 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 19:50:41.470741 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [parameters: {'id': 'ab85e23d4b274e1b88e170ea72ba5ab5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 19:50:41.470741 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=150549) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 19:50:41.491116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db8330-b868-4a4f-8fcd-28eccf214123 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:41.493458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-4cb3b714-9d98-4aa5-8035-c3847accd1f6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 19:50:41.493458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:41.493458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:50:41.493458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:50:41.493458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:50:41.493458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:50:41.493458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:50:41.493458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:50:41.493458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:50:41.493458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application conn.query(q) May 19 19:50:41.493458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:50:41.493458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:50:41.493458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:50:41.493458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application result.read() May 19 19:50:41.493458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return ref.to_dict() May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:50:41.494686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application next(self.gen) May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application yield resource May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application next(self.gen) May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self.session.rollback() May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self.force_reraise() May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise self.value May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application session.commit() May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 19:50:41.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._prepare_impl() May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self.session.flush() May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._flush(objects) May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application compat.raise_( May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application flush_context.execute() May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rec.execute(self) May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 19:50:41.497187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:50:41.498458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application conn.query(q) May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application result.read() May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:50:41.499678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [parameters: {'id': 'ab85e23d4b274e1b88e170ea72ba5ab5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:41.501446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 19:50:41.504092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:41.512729 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 237/477] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 19:50:41 2022] POST /identity/v3/roles => generated 144 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:41.513325 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3b714-9d98-4aa5-8035-c3847accd1f6 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:41.549595 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e055db-7621-40cc-ab5b-6fecfc5c0907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:41.550698 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e055db-7621-40cc-ab5b-6fecfc5c0907 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:41.551194 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e055db-7621-40cc-ab5b-6fecfc5c0907 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:41.551957 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e055db-7621-40cc-ab5b-6fecfc5c0907 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:41.553784 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e055db-7621-40cc-ab5b-6fecfc5c0907 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:41.555567 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e055db-7621-40cc-ab5b-6fecfc5c0907 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:41.585170 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 241/478] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:50:41 2022] POST /identity/v3/users => generated 249 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:50:41.650429 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3b714-9d98-4aa5-8035-c3847accd1f6 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:41.666649 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:41.719120 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76664f5b-907d-467c-904e-c700fa55e184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:41.720003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76664f5b-907d-467c-904e-c700fa55e184 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:41.720286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76664f5b-907d-467c-904e-c700fa55e184 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:41.720559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76664f5b-907d-467c-904e-c700fa55e184 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:41.737058 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76664f5b-907d-467c-904e-c700fa55e184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-76664f5b-907d-467c-904e-c700fa55e184 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:41.745182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:41.746221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:41.747194 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:41.747194 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:41.747194 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:41.747194 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:41.747194 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:41.747194 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:41.747194 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:41.747194 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:41.747194 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:41.747194 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:41.747194 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:50:41.747194 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:41.750860 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 238/479] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:50:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:41.756661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e055db-7621-40cc-ab5b-6fecfc5c0907 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:41.873718 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:41.891103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e055db-7621-40cc-ab5b-6fecfc5c0907 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:41.926234 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d137c-e838-4516-a04b-eba77cd183fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:41.926857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d137c-e838-4516-a04b-eba77cd183fc None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:41.927139 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d137c-e838-4516-a04b-eba77cd183fc None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:41.927416 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d137c-e838-4516-a04b-eba77cd183fc None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:41.928178 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d137c-e838-4516-a04b-eba77cd183fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:41.928480 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d137c-e838-4516-a04b-eba77cd183fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:41.929893 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d137c-e838-4516-a04b-eba77cd183fc None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:41.939944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 242/480] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:43.144929 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462320c-9f3b-4869-9465-d666c2a39249 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:43.145508 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462320c-9f3b-4869-9465-d666c2a39249 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:43.145508 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462320c-9f3b-4869-9465-d666c2a39249 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:43.147130 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 239/481] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:43.152736 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b70326-1460-4c2a-bd40-1ae5f8905b48 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:43.153200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b70326-1460-4c2a-bd40-1ae5f8905b48 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:43.153310 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b70326-1460-4c2a-bd40-1ae5f8905b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:43.183670 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-57b70326-1460-4c2a-bd40-1ae5f8905b48 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:43.184892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-57b70326-1460-4c2a-bd40-1ae5f8905b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:43.272929 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 243/482] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:43.311293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462320c-9f3b-4869-9465-d666c2a39249 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:43.441621 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462320c-9f3b-4869-9465-d666c2a39249 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:43.485991 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0ac65-c529-42e0-92e2-afd4b3d60294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:43.486846 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0ac65-c529-42e0-92e2-afd4b3d60294 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:43.487127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0ac65-c529-42e0-92e2-afd4b3d60294 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:43.487406 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0ac65-c529-42e0-92e2-afd4b3d60294 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:43.503504 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0ac65-c529-42e0-92e2-afd4b3d60294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-4eb0ac65-c529-42e0-92e2-afd4b3d60294 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:43.511507 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:43.512570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:43.513506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:43.513506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:43.513506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:43.513506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:43.513506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:43.513506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:43.513506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:43.513506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:43.513506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:43.513506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:43.513506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:43.513506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:43.517808 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 240/483] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:43.518944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b70326-1460-4c2a-bd40-1ae5f8905b48 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:43.612364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:43.636122 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b70326-1460-4c2a-bd40-1ae5f8905b48 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:43.677785 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a7a20-0946-4d97-95e5-9315103e7b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:43.678492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7a20-0946-4d97-95e5-9315103e7b17 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:43.678776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7a20-0946-4d97-95e5-9315103e7b17 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:43.679048 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7a20-0946-4d97-95e5-9315103e7b17 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:43.679823 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a7a20-0946-4d97-95e5-9315103e7b17 None admin] RBAC: Adding query filter params (name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:43.680128 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a7a20-0946-4d97-95e5-9315103e7b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:43.681508 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a7a20-0946-4d97-95e5-9315103e7b17 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:43.691195 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 244/484] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:50:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:43.696834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0ac65-c529-42e0-92e2-afd4b3d60294 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:43.760124 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85729be-1766-4ec7-a561-3fb16343628d None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:43.760496 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85729be-1766-4ec7-a561-3fb16343628d None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:43.760680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85729be-1766-4ec7-a561-3fb16343628d None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:43.762312 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 245/485] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:43.767421 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffa98a-c3ef-47b8-bd06-3daa30f831b0 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:43.767701 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffa98a-c3ef-47b8-bd06-3daa30f831b0 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:43.767979 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffa98a-c3ef-47b8-bd06-3daa30f831b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:43.771660 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:43.789108 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-70ffa98a-c3ef-47b8-bd06-3daa30f831b0 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:43.790357 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-70ffa98a-c3ef-47b8-bd06-3daa30f831b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:43.866625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 246/486] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:43.892691 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffa98a-c3ef-47b8-bd06-3daa30f831b0 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:43.935372 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0ac65-c529-42e0-92e2-afd4b3d60294 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:44.008248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d6a67-6c99-466d-841c-95ccb22e69bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:44.008980 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d6a67-6c99-466d-841c-95ccb22e69bc None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.009258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d6a67-6c99-466d-841c-95ccb22e69bc None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.009529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d6a67-6c99-466d-841c-95ccb22e69bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.014693 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:44.038308 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d6a67-6c99-466d-841c-95ccb22e69bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:44.039786 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d6a67-6c99-466d-841c-95ccb22e69bc None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:44.082288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-2e0d6a67-6c99-466d-841c-95ccb22e69bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:44.082288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:44.082288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:44.082288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:44.082288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:44.082288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:44.082288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:44.082288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:44.082288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:44.082288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:44.082288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:44.082288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:44.082288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:44.082288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:44.082288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:44.083362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:44.084376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:44.084376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:44.084376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:44.084376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:44.084623 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffa98a-c3ef-47b8-bd06-3daa30f831b0 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:44.087182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 241/487] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:44.093630 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d6a67-6c99-466d-841c-95ccb22e69bc None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:44.120408 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e186f89-4351-4502-bd2f-0ba27163db0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:44.121353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e186f89-4351-4502-bd2f-0ba27163db0a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.121633 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e186f89-4351-4502-bd2f-0ba27163db0a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.121905 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e186f89-4351-4502-bd2f-0ba27163db0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.133991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e186f89-4351-4502-bd2f-0ba27163db0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-4e186f89-4351-4502-bd2f-0ba27163db0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:44.142697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:44.143751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:44.144677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:44.144677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:44.144677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:44.144677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:44.144677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:44.144677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:44.144677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:44.144677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:44.144677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:44.144677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:44.144677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:44.144677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:44.144677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 247/488] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:44.150356 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e186f89-4351-4502-bd2f-0ba27163db0a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:44.267714 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e186f89-4351-4502-bd2f-0ba27163db0a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:44.275224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:44.294933 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d6a67-6c99-466d-841c-95ccb22e69bc None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:44.311388 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:44.318500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58058f-6bf8-4e61-84d1-48fc5f465540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:44.319306 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58058f-6bf8-4e61-84d1-48fc5f465540 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.319681 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58058f-6bf8-4e61-84d1-48fc5f465540 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.320057 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58058f-6bf8-4e61-84d1-48fc5f465540 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.320905 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58058f-6bf8-4e61-84d1-48fc5f465540 None admin] RBAC: Adding query filter params (name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:44.321368 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58058f-6bf8-4e61-84d1-48fc5f465540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:44.322844 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58058f-6bf8-4e61-84d1-48fc5f465540 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:44.331854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 248/489] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:50:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:44.336648 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955ed63-9a15-49ee-88c2-fac831af4ef4 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.336648 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955ed63-9a15-49ee-88c2-fac831af4ef4 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.336648 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955ed63-9a15-49ee-88c2-fac831af4ef4 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.338612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 249/490] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:44.339281 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955ed63-9a15-49ee-88c2-fac831af4ef4 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:44.347851 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c460cf-60a9-40e1-b9d6-96283567d72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:44.348831 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c460cf-60a9-40e1-b9d6-96283567d72e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.349497 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c460cf-60a9-40e1-b9d6-96283567d72e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.349981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c460cf-60a9-40e1-b9d6-96283567d72e None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.351524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c460cf-60a9-40e1-b9d6-96283567d72e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:44.352041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c460cf-60a9-40e1-b9d6-96283567d72e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:44.354542 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c460cf-60a9-40e1-b9d6-96283567d72e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:44.372438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 242/491] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:44.379222 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee8572-eb4c-4d90-8264-b715b5735997 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.379767 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee8572-eb4c-4d90-8264-b715b5735997 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.380248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee8572-eb4c-4d90-8264-b715b5735997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.397726 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-d7ee8572-eb4c-4d90-8264-b715b5735997 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:44.399365 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-d7ee8572-eb4c-4d90-8264-b715b5735997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:44.455205 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955ed63-9a15-49ee-88c2-fac831af4ef4 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:44.475171 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 243/492] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:44.476369 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee8572-eb4c-4d90-8264-b715b5735997 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:44.506976 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ee3bb-510c-4aaf-a539-513a7770fbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:44.507825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ee3bb-510c-4aaf-a539-513a7770fbe2 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.508105 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ee3bb-510c-4aaf-a539-513a7770fbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.508427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ee3bb-510c-4aaf-a539-513a7770fbe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.528617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ee3bb-510c-4aaf-a539-513a7770fbe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:44.529995 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ee3bb-510c-4aaf-a539-513a7770fbe2 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:44.548839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-4e1ee3bb-510c-4aaf-a539-513a7770fbe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:44.548839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:44.548839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:44.548839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:44.548839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:44.548839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:44.548839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:44.548839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:44.548839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:44.548839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:44.548839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:44.548839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:44.548839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:44.548839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:44.548839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:44.550039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:44.551058 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:44.551058 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:44.551058 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:44.551058 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:44.554821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 250/493] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:44.555671 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ee3bb-510c-4aaf-a539-513a7770fbe2 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:44.556220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:44.588604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee8572-eb4c-4d90-8264-b715b5735997 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:44.605755 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:44.627691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c13278-9b2c-4d0a-8304-2259766f9693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:44.628502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c13278-9b2c-4d0a-8304-2259766f9693 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.628782 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c13278-9b2c-4d0a-8304-2259766f9693 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.629072 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c13278-9b2c-4d0a-8304-2259766f9693 None admin] PATH_INFO: `/v3/users/nova` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.636379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c13278-9b2c-4d0a-8304-2259766f9693 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:44.637852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c13278-9b2c-4d0a-8304-2259766f9693 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-e8c13278-9b2c-4d0a-8304-2259766f9693 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:44.653824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:44.654917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:44.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:44.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:44.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:44.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:44.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:44.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:44.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:44.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:44.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:50:44.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:44.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 244/494] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:50:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:44.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c13278-9b2c-4d0a-8304-2259766f9693 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:44.666344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ee3bb-510c-4aaf-a539-513a7770fbe2 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:44.707107 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70df3b0-ed35-4273-82b6-bd673ed8a287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:44.708285 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70df3b0-ed35-4273-82b6-bd673ed8a287 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.708581 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70df3b0-ed35-4273-82b6-bd673ed8a287 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.708857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70df3b0-ed35-4273-82b6-bd673ed8a287 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.722541 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70df3b0-ed35-4273-82b6-bd673ed8a287 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-b70df3b0-ed35-4273-82b6-bd673ed8a287 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:44.730282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:44.731305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:44.732210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:44.732210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:44.732210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:44.732210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:44.732210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:44.732210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:44.732210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:44.732210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:44.732210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:44.732210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:44.732210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:44.732210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:44.732210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 251/495] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:44.733462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c9f14-f7f4-4ec2-9986-20c5686a4d9c None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.733648 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c9f14-f7f4-4ec2-9986-20c5686a4d9c None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.733956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c9f14-f7f4-4ec2-9986-20c5686a4d9c None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.735908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 252/496] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:44.739022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c9f14-f7f4-4ec2-9986-20c5686a4d9c None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:44.772725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c13278-9b2c-4d0a-8304-2259766f9693 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:44.806281 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:44.812015 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63d365-d38f-4fad-853e-364b6431da91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:44.812693 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63d365-d38f-4fad-853e-364b6431da91 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.812976 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63d365-d38f-4fad-853e-364b6431da91 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.813237 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63d365-d38f-4fad-853e-364b6431da91 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.813962 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63d365-d38f-4fad-853e-364b6431da91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:44.814286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63d365-d38f-4fad-853e-364b6431da91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:44.815691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63d365-d38f-4fad-853e-364b6431da91 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:44.831898 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 245/497] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:44.831898 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63d365-d38f-4fad-853e-364b6431da91 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:44.855216 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c9f14-f7f4-4ec2-9986-20c5686a4d9c None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:44.858601 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:44.891158 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3ad3a-b94a-4753-99b0-a4736f5ef7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:44.891828 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3ad3a-b94a-4753-99b0-a4736f5ef7a0 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.892106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3ad3a-b94a-4753-99b0-a4736f5ef7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.892379 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3ad3a-b94a-4753-99b0-a4736f5ef7a0 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.893223 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3ad3a-b94a-4753-99b0-a4736f5ef7a0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:44.893533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3ad3a-b94a-4753-99b0-a4736f5ef7a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:44.895019 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3ad3a-b94a-4753-99b0-a4736f5ef7a0 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:44.930350 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 253/498] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:50:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:44.931580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58122cfb-9b88-47c3-8749-427287de2aab None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.931863 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58122cfb-9b88-47c3-8749-427287de2aab None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.932137 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58122cfb-9b88-47c3-8749-427287de2aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.942379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63d365-d38f-4fad-853e-364b6431da91 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:44.948055 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-58122cfb-9b88-47c3-8749-427287de2aab None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:44.949244 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-58122cfb-9b88-47c3-8749-427287de2aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:44.982930 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a66c6a-dcbd-453e-8a4f-6f0a13d6e486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:44.983928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a66c6a-dcbd-453e-8a4f-6f0a13d6e486 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.984481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a66c6a-dcbd-453e-8a4f-6f0a13d6e486 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.984836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a66c6a-dcbd-453e-8a4f-6f0a13d6e486 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.985608 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a66c6a-dcbd-453e-8a4f-6f0a13d6e486 None admin] RBAC: Adding query filter params (name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:44.985922 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a66c6a-dcbd-453e-8a4f-6f0a13d6e486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:44.987322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a66c6a-dcbd-453e-8a4f-6f0a13d6e486 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:44.995699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 246/499] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:50:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:44.996500 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a66c6a-dcbd-453e-8a4f-6f0a13d6e486 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:45.024685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 254/500] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:45.026308 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a335121-69fc-4364-b8c2-1fa264f471be None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:45.026630 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a335121-69fc-4364-b8c2-1fa264f471be None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:45.026906 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a335121-69fc-4364-b8c2-1fa264f471be None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:45.028151 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 255/501] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:45.028833 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a335121-69fc-4364-b8c2-1fa264f471be None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:45.065361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:45.110785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a66c6a-dcbd-453e-8a4f-6f0a13d6e486 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:45.124816 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:45.135234 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a335121-69fc-4364-b8c2-1fa264f471be None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:45.144163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886a639-8d22-497a-bc3d-54773b8ea2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:45.144987 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886a639-8d22-497a-bc3d-54773b8ea2cf None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:45.145275 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886a639-8d22-497a-bc3d-54773b8ea2cf None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:45.145549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886a639-8d22-497a-bc3d-54773b8ea2cf None admin] PATH_INFO: `/v3/users/glance` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:45.153252 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886a639-8d22-497a-bc3d-54773b8ea2cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:45.154761 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886a639-8d22-497a-bc3d-54773b8ea2cf None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-2886a639-8d22-497a-bc3d-54773b8ea2cf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:45.169217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:45.170524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:45.171678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:45.171678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:45.171678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:45.171678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:45.171678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:45.171678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:45.171678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:45.171678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:45.171678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 19:50:45.171678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:45.172549 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f73de-a6ee-494b-b667-72354c909973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:45.172549 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f73de-a6ee-494b-b667-72354c909973 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:45.172549 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f73de-a6ee-494b-b667-72354c909973 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:45.172786 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 247/502] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:45.172870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f73de-a6ee-494b-b667-72354c909973 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:45.172955 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886a639-8d22-497a-bc3d-54773b8ea2cf None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:45.241953 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f73de-a6ee-494b-b667-72354c909973 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:45.243490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f73de-a6ee-494b-b667-72354c909973 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:45.271714 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886a639-8d22-497a-bc3d-54773b8ea2cf None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:45.285579 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:45.308201 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2b5e9-4dcd-4a6b-bd14-b2074537df4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:45.309021 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2b5e9-4dcd-4a6b-bd14-b2074537df4d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:45.309305 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2b5e9-4dcd-4a6b-bd14-b2074537df4d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:45.309590 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2b5e9-4dcd-4a6b-bd14-b2074537df4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:45.320440 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2b5e9-4dcd-4a6b-bd14-b2074537df4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:45.321842 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2b5e9-4dcd-4a6b-bd14-b2074537df4d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:45.339284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-0de2b5e9-4dcd-4a6b-bd14-b2074537df4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:45.339284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:45.339284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:45.339284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:45.339284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:45.339284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:45.339284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:45.339284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:45.339284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:45.339284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:45.339284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:45.339284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:45.339284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:45.339284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:45.339284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:45.340265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:45.341611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:45.341611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:45.341611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:45.341611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:45.342411 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:45.342858 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 248/503] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:45.342858 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c00c59-a6f0-43e7-9253-2c6df30fd928 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:45.342858 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c00c59-a6f0-43e7-9253-2c6df30fd928 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:45.342858 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c00c59-a6f0-43e7-9253-2c6df30fd928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:45.353268 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 256/504] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:45.354432 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f73de-a6ee-494b-b667-72354c909973 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:45.360765 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-21c00c59-a6f0-43e7-9253-2c6df30fd928 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:45.361978 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-21c00c59-a6f0-43e7-9253-2c6df30fd928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:45.428954 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 249/505] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:45.429800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c00c59-a6f0-43e7-9253-2c6df30fd928 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:45.458432 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f73de-a6ee-494b-b667-72354c909973 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:45.493116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e39ec-5dfd-4ef7-b9cf-52e08b674592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:45.493920 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e39ec-5dfd-4ef7-b9cf-52e08b674592 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:45.494199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e39ec-5dfd-4ef7-b9cf-52e08b674592 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:45.494479 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e39ec-5dfd-4ef7-b9cf-52e08b674592 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:45.505148 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e39ec-5dfd-4ef7-b9cf-52e08b674592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-2a1e39ec-5dfd-4ef7-b9cf-52e08b674592 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:45.512849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:45.514014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:45.515009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:45.515009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:45.515009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:45.515009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:45.515009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:45.515009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:45.515009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:45.515009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:45.515009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:45.515009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:45.515009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:45.515009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:45.515009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 257/506] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:45.515009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e39ec-5dfd-4ef7-b9cf-52e08b674592 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:45.537059 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:45.543061 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c00c59-a6f0-43e7-9253-2c6df30fd928 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:45.574685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:45.578449 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51d853-e6bc-44c2-b3ed-c397645d5d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:45.579133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51d853-e6bc-44c2-b3ed-c397645d5d58 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:45.579411 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51d853-e6bc-44c2-b3ed-c397645d5d58 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:45.579683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51d853-e6bc-44c2-b3ed-c397645d5d58 None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:45.580524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51d853-e6bc-44c2-b3ed-c397645d5d58 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:45.580832 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51d853-e6bc-44c2-b3ed-c397645d5d58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:45.582304 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51d853-e6bc-44c2-b3ed-c397645d5d58 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:45.619424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 250/507] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 19:50:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:45.620267 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51d853-e6bc-44c2-b3ed-c397645d5d58 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:45.622919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e39ec-5dfd-4ef7-b9cf-52e08b674592 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:45.658508 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c0fd4-d8f9-4a28-8b2a-5c1177387593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:45.659155 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c0fd4-d8f9-4a28-8b2a-5c1177387593 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:45.659408 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c0fd4-d8f9-4a28-8b2a-5c1177387593 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:45.659705 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c0fd4-d8f9-4a28-8b2a-5c1177387593 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:45.660422 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c0fd4-d8f9-4a28-8b2a-5c1177387593 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:45.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c0fd4-d8f9-4a28-8b2a-5c1177387593 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:45.662121 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c0fd4-d8f9-4a28-8b2a-5c1177387593 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:45.675065 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 258/508] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:45.675844 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c0fd4-d8f9-4a28-8b2a-5c1177387593 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:45.734329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51d853-e6bc-44c2-b3ed-c397645d5d58 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:45.770481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a798a5-f1b9-4f85-8813-1e2d3296757c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:45.771323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a798a5-f1b9-4f85-8813-1e2d3296757c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:45.771609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a798a5-f1b9-4f85-8813-1e2d3296757c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:45.771887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a798a5-f1b9-4f85-8813-1e2d3296757c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:45.781696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c0fd4-d8f9-4a28-8b2a-5c1177387593 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:45.782310 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a798a5-f1b9-4f85-8813-1e2d3296757c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:45.783784 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a798a5-f1b9-4f85-8813-1e2d3296757c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:45.794909 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:45.802053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-c9a798a5-f1b9-4f85-8813-1e2d3296757c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:45.802053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:45.802053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:45.802053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:45.802053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:45.802053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:45.802053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:45.802053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:45.802053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:45.802053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:45.802053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:45.802053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:45.802053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:45.802053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:45.802053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:45.803248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:45.804671 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:45.804671 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:45.804671 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:45.804671 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:45.804671 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 251/509] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:45.804671 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a798a5-f1b9-4f85-8813-1e2d3296757c None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:45.816529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:45.819864 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c551a-9f6a-4f05-bfcc-e4ba89e38e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:45.820752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c551a-9f6a-4f05-bfcc-e4ba89e38e55 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:45.821054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c551a-9f6a-4f05-bfcc-e4ba89e38e55 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:45.821343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c551a-9f6a-4f05-bfcc-e4ba89e38e55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:45.832072 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c551a-9f6a-4f05-bfcc-e4ba89e38e55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:45.833480 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c551a-9f6a-4f05-bfcc-e4ba89e38e55 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:45.852218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-6e1c551a-9f6a-4f05-bfcc-e4ba89e38e55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:45.852218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:45.852218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:45.852218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:45.852218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:45.852218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:45.852218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:45.852218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:45.852218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:45.852218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:45.852218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:45.852218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:45.852218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:45.852218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:45.852218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:45.853406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:45.854462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:45.854462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:45.854462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:45.854462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:45.854462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 259/510] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:45.854462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c551a-9f6a-4f05-bfcc-e4ba89e38e55 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:45.914781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a798a5-f1b9-4f85-8813-1e2d3296757c None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:45.948832 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84798aeb-1e75-4d80-9f63-27fca07a09d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:45.949505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84798aeb-1e75-4d80-9f63-27fca07a09d9 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:45.949786 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84798aeb-1e75-4d80-9f63-27fca07a09d9 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:45.950104 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84798aeb-1e75-4d80-9f63-27fca07a09d9 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:45.950866 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84798aeb-1e75-4d80-9f63-27fca07a09d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:45.951208 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84798aeb-1e75-4d80-9f63-27fca07a09d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:45.952991 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84798aeb-1e75-4d80-9f63-27fca07a09d9 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:45.953614 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c551a-9f6a-4f05-bfcc-e4ba89e38e55 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:45.960621 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 252/511] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:50:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:45.961375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84798aeb-1e75-4d80-9f63-27fca07a09d9 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:45.989842 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7f878-db05-4cfd-958a-63c0175c9c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:45.990527 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7f878-db05-4cfd-958a-63c0175c9c9e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:45.990806 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7f878-db05-4cfd-958a-63c0175c9c9e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:45.991075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7f878-db05-4cfd-958a-63c0175c9c9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:46.022106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:46.030225 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:46.063233 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a7f878-db05-4cfd-958a-63c0175c9c9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:46.064771 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a7f878-db05-4cfd-958a-63c0175c9c9e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:46.069942 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84798aeb-1e75-4d80-9f63-27fca07a09d9 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:46.104538 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fa7f3-e781-4372-a963-af87c6d18c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:46.105275 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fa7f3-e781-4372-a963-af87c6d18c5e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:46.105585 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fa7f3-e781-4372-a963-af87c6d18c5e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:46.105878 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fa7f3-e781-4372-a963-af87c6d18c5e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:46.113746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fa7f3-e781-4372-a963-af87c6d18c5e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:46.115242 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fa7f3-e781-4372-a963-af87c6d18c5e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-778fa7f3-e781-4372-a963-af87c6d18c5e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:46.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:46.135483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:46.136791 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:46.136791 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:46.136791 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:46.136791 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:46.136791 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:46.136791 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:46.136791 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:46.136791 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:46.136791 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:50:46.136791 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:46.136791 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 253/512] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:46.136791 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fa7f3-e781-4372-a963-af87c6d18c5e None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:46.171567 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 260/513] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:46.172708 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7f878-db05-4cfd-958a-63c0175c9c9e None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:46.239674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:46.246687 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fa7f3-e781-4372-a963-af87c6d18c5e None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:46.273751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:46.281205 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7f878-db05-4cfd-958a-63c0175c9c9e None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:46.285029 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c91e05-3109-4b84-919d-56db00f31360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:46.285944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c91e05-3109-4b84-919d-56db00f31360 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:46.286567 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c91e05-3109-4b84-919d-56db00f31360 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:46.286567 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c91e05-3109-4b84-919d-56db00f31360 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:46.287580 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c91e05-3109-4b84-919d-56db00f31360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:46.288010 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c91e05-3109-4b84-919d-56db00f31360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:46.290201 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c91e05-3109-4b84-919d-56db00f31360 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:46.304851 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 254/514] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:46.305684 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c91e05-3109-4b84-919d-56db00f31360 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:46.323925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19510a6c-769f-4aee-81b6-a72005a2f211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:46.324599 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19510a6c-769f-4aee-81b6-a72005a2f211 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:46.324880 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19510a6c-769f-4aee-81b6-a72005a2f211 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:46.325137 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19510a6c-769f-4aee-81b6-a72005a2f211 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:46.325871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19510a6c-769f-4aee-81b6-a72005a2f211 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:46.326169 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19510a6c-769f-4aee-81b6-a72005a2f211 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:46.327654 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19510a6c-769f-4aee-81b6-a72005a2f211 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:46.338743 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 261/515] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:46.339533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19510a6c-769f-4aee-81b6-a72005a2f211 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:46.414162 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c91e05-3109-4b84-919d-56db00f31360 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:46.441341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19510a6c-769f-4aee-81b6-a72005a2f211 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:46.448898 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6504c-acc8-4aa2-ac06-626ce4450dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:46.449705 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6504c-acc8-4aa2-ac06-626ce4450dfe None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:46.449985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6504c-acc8-4aa2-ac06-626ce4450dfe None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:46.450249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6504c-acc8-4aa2-ac06-626ce4450dfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:46.461153 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6504c-acc8-4aa2-ac06-626ce4450dfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:46.462656 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6504c-acc8-4aa2-ac06-626ce4450dfe None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:46.465566 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:46.476798 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc17c1-bca8-42b2-9c9a-e12b5599c5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:46.477481 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc17c1-bca8-42b2-9c9a-e12b5599c5aa None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:46.477767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc17c1-bca8-42b2-9c9a-e12b5599c5aa None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:46.478047 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc17c1-bca8-42b2-9c9a-e12b5599c5aa None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:46.478941 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc17c1-bca8-42b2-9c9a-e12b5599c5aa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:46.479257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc17c1-bca8-42b2-9c9a-e12b5599c5aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:46.480696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc17c1-bca8-42b2-9c9a-e12b5599c5aa None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:46.481089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-32b6504c-acc8-4aa2-ac06-626ce4450dfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:46.481089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:46.481089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:46.481089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:46.481089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:46.481089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:46.481089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:46.481089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:46.481089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:46.481089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:46.481089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:46.481089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:46.481089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:46.481089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:46.481089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:46.482033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:46.483045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:46.483045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:46.483045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:46.483045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:46.483045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 255/516] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:46.483364 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6504c-acc8-4aa2-ac06-626ce4450dfe None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:46.501025 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 262/517] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 19:50:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:46.501863 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc17c1-bca8-42b2-9c9a-e12b5599c5aa None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:46.533928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:46.592744 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6504c-acc8-4aa2-ac06-626ce4450dfe None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:46.600150 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc17c1-bca8-42b2-9c9a-e12b5599c5aa None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:46.629635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b738d0-2f9e-4457-ac0a-2c17d76e97da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:46.630473 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b738d0-2f9e-4457-ac0a-2c17d76e97da None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:46.630755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b738d0-2f9e-4457-ac0a-2c17d76e97da None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:46.631028 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b738d0-2f9e-4457-ac0a-2c17d76e97da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:46.634973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95769d36-b93a-4202-8d1e-fbaf3fe656b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:46.635778 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95769d36-b93a-4202-8d1e-fbaf3fe656b1 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:46.636062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95769d36-b93a-4202-8d1e-fbaf3fe656b1 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:46.636335 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95769d36-b93a-4202-8d1e-fbaf3fe656b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:46.641518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b738d0-2f9e-4457-ac0a-2c17d76e97da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:46.643012 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b738d0-2f9e-4457-ac0a-2c17d76e97da None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:46.644195 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95769d36-b93a-4202-8d1e-fbaf3fe656b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:46.645847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95769d36-b93a-4202-8d1e-fbaf3fe656b1 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:46.660661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-95769d36-b93a-4202-8d1e-fbaf3fe656b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:46.660661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:46.660661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:46.660661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:46.660661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:46.660661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:46.660661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:46.660661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:46.660661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:46.660661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:46.660661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:46.660661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:46.660661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:46.660661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:46.660661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:46.661847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:46.663037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:46.663037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:46.663037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:46.663037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:46.663037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:46.663037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:46.663037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:46.663037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:46.663615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-e8b738d0-2f9e-4457-ac0a-2c17d76e97da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:46.663615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:46.663615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:46.663615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:46.663615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:46.663615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:46.663615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:46.663615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:46.663615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:46.663615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:46.663615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:46.663615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:46.663615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:46.663615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:46.663615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:46.664543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:46.665539 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:46.665539 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:46.665539 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:46.665539 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:46.665539 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 256/518] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:46.665539 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b738d0-2f9e-4457-ac0a-2c17d76e97da None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:46.666557 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 263/519] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:46.667526 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95769d36-b93a-4202-8d1e-fbaf3fe656b1 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:46.714712 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:46.766286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:46.769514 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95769d36-b93a-4202-8d1e-fbaf3fe656b1 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:46.775775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b738d0-2f9e-4457-ac0a-2c17d76e97da None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:46.804020 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0edd8-c89a-4964-bf0f-d3b2989edfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:46.804689 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0edd8-c89a-4964-bf0f-d3b2989edfbe None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:46.804971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0edd8-c89a-4964-bf0f-d3b2989edfbe None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:46.805245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0edd8-c89a-4964-bf0f-d3b2989edfbe None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:46.805968 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0edd8-c89a-4964-bf0f-d3b2989edfbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:46.806282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0edd8-c89a-4964-bf0f-d3b2989edfbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:46.807671 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0edd8-c89a-4964-bf0f-d3b2989edfbe None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:46.812065 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca08077-dcbc-4683-a1b1-a245f3713dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:46.813270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca08077-dcbc-4683-a1b1-a245f3713dd6 None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:46.813553 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca08077-dcbc-4683-a1b1-a245f3713dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:46.813826 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca08077-dcbc-4683-a1b1-a245f3713dd6 None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:46.814590 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca08077-dcbc-4683-a1b1-a245f3713dd6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:46.816042 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca08077-dcbc-4683-a1b1-a245f3713dd6 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:46.819536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 264/520] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:46.820388 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0edd8-c89a-4964-bf0f-d3b2989edfbe None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:46.840600 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 257/521] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:50:46 2022] POST /identity/v3/users => generated 247 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:50:46.841424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca08077-dcbc-4683-a1b1-a245f3713dd6 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:46.960490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca08077-dcbc-4683-a1b1-a245f3713dd6 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:46.962799 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0edd8-c89a-4964-bf0f-d3b2989edfbe None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:46.964585 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:46.997225 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc733b2-0fc7-46d4-978d-204c63e135df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:46.998005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc733b2-0fc7-46d4-978d-204c63e135df None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:46.998385 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc733b2-0fc7-46d4-978d-204c63e135df None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:46.998729 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc733b2-0fc7-46d4-978d-204c63e135df None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:46.999798 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc733b2-0fc7-46d4-978d-204c63e135df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:47.000300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc733b2-0fc7-46d4-978d-204c63e135df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:47.001989 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc733b2-0fc7-46d4-978d-204c63e135df None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:47.005304 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37707c02-d4b1-4cc1-b260-04a49e650da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:47.005977 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37707c02-d4b1-4cc1-b260-04a49e650da3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:47.006292 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37707c02-d4b1-4cc1-b260-04a49e650da3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:47.006573 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37707c02-d4b1-4cc1-b260-04a49e650da3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:47.026791 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 258/522] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:50:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:47.026791 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc733b2-0fc7-46d4-978d-204c63e135df None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:47.036597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:47.094952 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37707c02-d4b1-4cc1-b260-04a49e650da3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:47.096456 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37707c02-d4b1-4cc1-b260-04a49e650da3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:47.140889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc733b2-0fc7-46d4-978d-204c63e135df None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:47.177224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62330ed-c822-4887-ae88-ddf5c569056c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:47.178184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62330ed-c822-4887-ae88-ddf5c569056c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:47.178707 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62330ed-c822-4887-ae88-ddf5c569056c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:47.179273 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62330ed-c822-4887-ae88-ddf5c569056c None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:47.180043 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62330ed-c822-4887-ae88-ddf5c569056c None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:47.180324 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62330ed-c822-4887-ae88-ddf5c569056c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:47.181730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62330ed-c822-4887-ae88-ddf5c569056c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:47.199034 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 259/523] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:47.199034 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62330ed-c822-4887-ae88-ddf5c569056c None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:47.203591 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 265/524] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:47.204478 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37707c02-d4b1-4cc1-b260-04a49e650da3 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:47.206888 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:47.272593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:47.310764 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62330ed-c822-4887-ae88-ddf5c569056c None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:47.312374 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37707c02-d4b1-4cc1-b260-04a49e650da3 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:47.346095 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff268f43-e9ff-41fe-9b71-32e32869a544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:47.346920 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff268f43-e9ff-41fe-9b71-32e32869a544 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:47.347566 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff268f43-e9ff-41fe-9b71-32e32869a544 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:47.347839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff268f43-e9ff-41fe-9b71-32e32869a544 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:47.348914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2f3ab-8e3c-48bb-9af8-5f7b1559b9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:47.349574 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2f3ab-8e3c-48bb-9af8-5f7b1559b9c1 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:47.349886 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2f3ab-8e3c-48bb-9af8-5f7b1559b9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:47.350168 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2f3ab-8e3c-48bb-9af8-5f7b1559b9c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:47.351318 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2f3ab-8e3c-48bb-9af8-5f7b1559b9c1 None admin] RBAC: Adding query filter params (role.id=3387bd4782b047ae8d880e386b0351f0, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=0da7f490d9604107b974e9e461ebc38b) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:47.351318 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2f3ab-8e3c-48bb-9af8-5f7b1559b9c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:47.352776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2f3ab-8e3c-48bb-9af8-5f7b1559b9c1 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:47.356471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff268f43-e9ff-41fe-9b71-32e32869a544 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:47.357918 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff268f43-e9ff-41fe-9b71-32e32869a544 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:47.370627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 266/525] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:47 2022] GET /identity/v3/role_assignments?user.id=0da7f490d9604107b974e9e461ebc38b&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=3387bd4782b047ae8d880e386b0351f0 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:47.370627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2f3ab-8e3c-48bb-9af8-5f7b1559b9c1 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-ff268f43-e9ff-41fe-9b71-32e32869a544 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:47.373044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:47.374052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:47.375211 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:47.375211 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:47.375211 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:47.375211 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:47.375211 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:47.375211 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:47.375211 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:47.375211 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:47.375211 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:50:47.375211 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:47.378919 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 260/526] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:47.379822 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff268f43-e9ff-41fe-9b71-32e32869a544 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:47.468149 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:47.502956 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff268f43-e9ff-41fe-9b71-32e32869a544 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:47.521895 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2f3ab-8e3c-48bb-9af8-5f7b1559b9c1 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:47.540565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:47.553304 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06788e-0382-48e8-9dfb-ad7448df45ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:47.554635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06788e-0382-48e8-9dfb-ad7448df45ed None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:47.554920 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06788e-0382-48e8-9dfb-ad7448df45ed None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:47.555195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06788e-0382-48e8-9dfb-ad7448df45ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:47.565644 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29158509-12c7-4f78-a645-24b51e07e2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:47.566843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158509-12c7-4f78-a645-24b51e07e2a6 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:47.566843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158509-12c7-4f78-a645-24b51e07e2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:47.567088 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158509-12c7-4f78-a645-24b51e07e2a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:47.571034 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06788e-0382-48e8-9dfb-ad7448df45ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:47.573479 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06788e-0382-48e8-9dfb-ad7448df45ed None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:47.578806 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29158509-12c7-4f78-a645-24b51e07e2a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:47.581601 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29158509-12c7-4f78-a645-24b51e07e2a6 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:47.593256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-9c06788e-0382-48e8-9dfb-ad7448df45ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:47.593256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:47.593256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:47.593256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:47.593256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:47.593256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:47.593256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:47.593256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:47.593256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:47.593256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:47.593256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:47.593256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:47.593256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:47.593256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:47.593256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:47.594544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:47.595541 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:47.595541 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:47.595541 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:47.595541 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:47.597625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-29158509-12c7-4f78-a645-24b51e07e2a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:47.597625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:47.597625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:47.597625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:47.597625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:47.597625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:47.597625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:47.597625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:47.597625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:47.597625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:47.597625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:47.597625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:47.597625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:47.597625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:47.597625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:47.598762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:47.599729 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:47.599729 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:47.599729 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:47.599729 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:47.599729 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:47.599729 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:47.599729 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:47.599729 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:47.605153 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 267/528] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:47.605153 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29158509-12c7-4f78-a645-24b51e07e2a6 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:47.605494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 261/528] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:47.606674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06788e-0382-48e8-9dfb-ad7448df45ed None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:47.731824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29158509-12c7-4f78-a645-24b51e07e2a6 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:47.741359 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06788e-0382-48e8-9dfb-ad7448df45ed None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:47.761088 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:47.772185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d4f9e-e39f-4ca9-bf03-b97657fd390c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:47.773309 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d4f9e-e39f-4ca9-bf03-b97657fd390c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:47.773815 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d4f9e-e39f-4ca9-bf03-b97657fd390c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:47.774328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d4f9e-e39f-4ca9-bf03-b97657fd390c None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:47.775581 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d4f9e-e39f-4ca9-bf03-b97657fd390c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:47.776072 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac01aa5-4533-4a6c-90ac-479eab99f810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:47.776348 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d4f9e-e39f-4ca9-bf03-b97657fd390c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:47.776744 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac01aa5-4533-4a6c-90ac-479eab99f810 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:47.777025 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac01aa5-4533-4a6c-90ac-479eab99f810 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:47.777302 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac01aa5-4533-4a6c-90ac-479eab99f810 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:47.778032 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac01aa5-4533-4a6c-90ac-479eab99f810 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:47.778367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac01aa5-4533-4a6c-90ac-479eab99f810 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:47.779210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d4f9e-e39f-4ca9-bf03-b97657fd390c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:47.779741 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac01aa5-4533-4a6c-90ac-479eab99f810 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:47.796896 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 262/529] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:47.796896 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac01aa5-4533-4a6c-90ac-479eab99f810 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:47.804910 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 268/530] 10.222.0.84 () {56 vars in 1139 bytes} [Thu May 19 19:50:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:47.805927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d4f9e-e39f-4ca9-bf03-b97657fd390c None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:47.846248 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:47.906959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac01aa5-4533-4a6c-90ac-479eab99f810 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:47.915702 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d4f9e-e39f-4ca9-bf03-b97657fd390c None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:47.942510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a390e4-ffc1-4190-90df-8c8a54f775ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:47.943187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a390e4-ffc1-4190-90df-8c8a54f775ee None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:47.943467 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a390e4-ffc1-4190-90df-8c8a54f775ee None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:47.943741 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a390e4-ffc1-4190-90df-8c8a54f775ee None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:47.944540 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a390e4-ffc1-4190-90df-8c8a54f775ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:47.944848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a390e4-ffc1-4190-90df-8c8a54f775ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:47.946341 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a390e4-ffc1-4190-90df-8c8a54f775ee None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:47.954189 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8387d3f-5a36-4846-a76e-9ad5a2ca6951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:47.955805 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8387d3f-5a36-4846-a76e-9ad5a2ca6951 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:47.956501 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8387d3f-5a36-4846-a76e-9ad5a2ca6951 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:47.957106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8387d3f-5a36-4846-a76e-9ad5a2ca6951 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:47.962694 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 263/531] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:50:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:47.963613 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a390e4-ffc1-4190-90df-8c8a54f775ee None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:47.965323 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8387d3f-5a36-4846-a76e-9ad5a2ca6951 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:47.967052 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8387d3f-5a36-4846-a76e-9ad5a2ca6951 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:47.986966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-b8387d3f-5a36-4846-a76e-9ad5a2ca6951 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:47.986966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:47.986966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:47.986966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:47.986966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:47.986966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:47.986966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:47.986966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:47.986966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:47.986966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:47.986966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:47.986966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:47.986966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:47.986966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:47.986966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:47.988002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:47.988981 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:47.988981 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:47.988981 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:47.988981 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:47.988981 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:47.988981 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:47.988981 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:47.988981 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:47.996415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 269/532] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:47.996415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8387d3f-5a36-4846-a76e-9ad5a2ca6951 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:48.046286 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:48.097903 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:48.101535 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a390e4-ffc1-4190-90df-8c8a54f775ee None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:48.121113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8387d3f-5a36-4846-a76e-9ad5a2ca6951 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:48.138753 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c18e10-32bf-4b24-981e-51afeab3c8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:48.139460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c18e10-32bf-4b24-981e-51afeab3c8e2 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:48.139685 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c18e10-32bf-4b24-981e-51afeab3c8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:48.139956 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c18e10-32bf-4b24-981e-51afeab3c8e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:48.167763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72333759-b1cb-4dbe-8f66-71fe5bfe9c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:48.169070 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72333759-b1cb-4dbe-8f66-71fe5bfe9c76 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:48.169873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72333759-b1cb-4dbe-8f66-71fe5bfe9c76 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:48.170653 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72333759-b1cb-4dbe-8f66-71fe5bfe9c76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:48.172138 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72333759-b1cb-4dbe-8f66-71fe5bfe9c76 None admin] RBAC: Adding query filter params (role.id=3387bd4782b047ae8d880e386b0351f0, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=64296c932944405abbf8e767570c84f1) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:48.172679 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72333759-b1cb-4dbe-8f66-71fe5bfe9c76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:48.174997 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72333759-b1cb-4dbe-8f66-71fe5bfe9c76 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:48.183359 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 270/533] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:47 2022] GET /identity/v3/role_assignments?user.id=64296c932944405abbf8e767570c84f1&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=3387bd4782b047ae8d880e386b0351f0 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:48.184462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72333759-b1cb-4dbe-8f66-71fe5bfe9c76 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:48.264612 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c18e10-32bf-4b24-981e-51afeab3c8e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:48.266614 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c18e10-32bf-4b24-981e-51afeab3c8e2 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:48.307221 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72333759-b1cb-4dbe-8f66-71fe5bfe9c76 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:48.318962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:48.348522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121de0e-7808-4225-a442-894187e8be52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:48.349529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121de0e-7808-4225-a442-894187e8be52 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:48.350590 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121de0e-7808-4225-a442-894187e8be52 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:48.351190 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121de0e-7808-4225-a442-894187e8be52 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:48.352307 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4121de0e-7808-4225-a442-894187e8be52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:48.352804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4121de0e-7808-4225-a442-894187e8be52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:48.354497 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4121de0e-7808-4225-a442-894187e8be52 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:48.376718 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 271/534] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:50:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:48.383507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121de0e-7808-4225-a442-894187e8be52 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:48.501623 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121de0e-7808-4225-a442-894187e8be52 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:48.523745 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:48.533564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 264/535] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:48.534710 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c18e10-32bf-4b24-981e-51afeab3c8e2 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:48.550713 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aaceae-e648-4e8b-b1e7-feb0c031bab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:48.551846 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaceae-e648-4e8b-b1e7-feb0c031bab3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:48.552353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaceae-e648-4e8b-b1e7-feb0c031bab3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:48.552936 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaceae-e648-4e8b-b1e7-feb0c031bab3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:48.554163 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aaceae-e648-4e8b-b1e7-feb0c031bab3 None admin] RBAC: Adding query filter params (role.id=3387bd4782b047ae8d880e386b0351f0, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=6b89f5a099d24437a25d5d7f21e10e2e) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:48.555003 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aaceae-e648-4e8b-b1e7-feb0c031bab3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:48.556799 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aaceae-e648-4e8b-b1e7-feb0c031bab3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:48.563327 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 272/536] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:48 2022] GET /identity/v3/role_assignments?user.id=6b89f5a099d24437a25d5d7f21e10e2e&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=3387bd4782b047ae8d880e386b0351f0 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:48.720420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c18e10-32bf-4b24-981e-51afeab3c8e2 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:48.781423 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2005ea4-3485-469e-a88f-dc63acbca68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:48.782442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2005ea4-3485-469e-a88f-dc63acbca68e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:48.782726 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2005ea4-3485-469e-a88f-dc63acbca68e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:48.783001 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2005ea4-3485-469e-a88f-dc63acbca68e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:48.800947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2005ea4-3485-469e-a88f-dc63acbca68e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:48.802562 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2005ea4-3485-469e-a88f-dc63acbca68e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:48.824322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-c2005ea4-3485-469e-a88f-dc63acbca68e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:48.824322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:48.824322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:48.824322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:48.824322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:48.824322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:48.824322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:48.824322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:48.824322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:48.824322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:48.824322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:48.824322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:48.824322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:48.824322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:48.824322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:48.825355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:48.826381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:48.826381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:48.826381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:48.826381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:48.826381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 265/537] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:48.832418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aaceae-e648-4e8b-b1e7-feb0c031bab3 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:48.869165 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:48.964505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aaceae-e648-4e8b-b1e7-feb0c031bab3 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:49.008844 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64925d42-f5cc-46ee-912e-c291720da56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:49.009825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64925d42-f5cc-46ee-912e-c291720da56d None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:49.010351 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64925d42-f5cc-46ee-912e-c291720da56d None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:49.010818 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64925d42-f5cc-46ee-912e-c291720da56d None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:49.011802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64925d42-f5cc-46ee-912e-c291720da56d None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:49.012310 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64925d42-f5cc-46ee-912e-c291720da56d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:49.013889 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64925d42-f5cc-46ee-912e-c291720da56d None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:49.034701 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 273/538] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:49.040619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2005ea4-3485-469e-a88f-dc63acbca68e None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:49.132631 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:49.224974 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2005ea4-3485-469e-a88f-dc63acbca68e None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:49.284130 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245d573-1a1e-4e2f-802f-d04e7c4379a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:49.285921 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245d573-1a1e-4e2f-802f-d04e7c4379a5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:49.286680 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245d573-1a1e-4e2f-802f-d04e7c4379a5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:49.287381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245d573-1a1e-4e2f-802f-d04e7c4379a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:49.297512 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800bd5b-7b22-4870-9bcd-3273f6bef487 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:49.298466 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800bd5b-7b22-4870-9bcd-3273f6bef487 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:49.299282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800bd5b-7b22-4870-9bcd-3273f6bef487 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:49.300891 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245d573-1a1e-4e2f-802f-d04e7c4379a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:49.302555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245d573-1a1e-4e2f-802f-d04e7c4379a5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:49.303260 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 274/539] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:49.313081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04e6f9-bf5f-439b-aa03-433b764d7988 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:49.313960 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04e6f9-bf5f-439b-aa03-433b764d7988 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:49.314793 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04e6f9-bf5f-439b-aa03-433b764d7988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:49.329573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-1245d573-1a1e-4e2f-802f-d04e7c4379a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:49.329573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:49.329573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:49.329573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:49.329573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:49.329573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:49.329573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:49.329573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:49.329573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:49.329573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:49.329573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:49.329573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:49.329573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:49.329573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:49.329573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:49.330674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:49.331746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:49.331746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:49.331746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:49.331746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:49.331746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:49.331746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:49.331746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:49.331746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:49.348397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 266/540] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:49.352076 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245d573-1a1e-4e2f-802f-d04e7c4379a5 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:49.356760 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-2a04e6f9-bf5f-439b-aa03-433b764d7988 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:49.359770 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-2a04e6f9-bf5f-439b-aa03-433b764d7988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:49.488457 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245d573-1a1e-4e2f-802f-d04e7c4379a5 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:49.500307 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 275/541] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:49.514283 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:49.542626 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153ed38-c36c-4556-9b80-328a6fa3bad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:49.544128 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153ed38-c36c-4556-9b80-328a6fa3bad0 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:49.544638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153ed38-c36c-4556-9b80-328a6fa3bad0 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:49.545149 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153ed38-c36c-4556-9b80-328a6fa3bad0 None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:49.546303 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153ed38-c36c-4556-9b80-328a6fa3bad0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:49.546833 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153ed38-c36c-4556-9b80-328a6fa3bad0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:49.548526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153ed38-c36c-4556-9b80-328a6fa3bad0 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:49.561635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 267/542] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:50:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:49.563813 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04e6f9-bf5f-439b-aa03-433b764d7988 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:49.567493 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153ed38-c36c-4556-9b80-328a6fa3bad0 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:49.617907 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:49.725280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153ed38-c36c-4556-9b80-328a6fa3bad0 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:49.761120 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a6d35-5f38-4597-acb2-747a4162ef84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:49.761797 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a6d35-5f38-4597-acb2-747a4162ef84 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:49.762088 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a6d35-5f38-4597-acb2-747a4162ef84 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:49.762395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a6d35-5f38-4597-acb2-747a4162ef84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:49.763252 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a6d35-5f38-4597-acb2-747a4162ef84 None admin] RBAC: Adding query filter params (role.id=3387bd4782b047ae8d880e386b0351f0, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=24d916d5c44c4078958734bd59a220b3) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:49.763700 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a6d35-5f38-4597-acb2-747a4162ef84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:49.765480 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a6d35-5f38-4597-acb2-747a4162ef84 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:49.772068 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 268/543] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:49 2022] GET /identity/v3/role_assignments?user.id=24d916d5c44c4078958734bd59a220b3&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=3387bd4782b047ae8d880e386b0351f0 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:49.799208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04e6f9-bf5f-439b-aa03-433b764d7988 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:49.864770 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76f82b-836d-4dae-b54a-4483f08bcd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:49.865607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76f82b-836d-4dae-b54a-4483f08bcd13 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:49.865895 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76f82b-836d-4dae-b54a-4483f08bcd13 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:49.866173 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76f82b-836d-4dae-b54a-4483f08bcd13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:49.888445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76f82b-836d-4dae-b54a-4483f08bcd13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-9e76f82b-836d-4dae-b54a-4483f08bcd13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:49.896887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:49.897973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:49.898945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:49.898945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:49.898945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:49.898945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:49.898945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:49.898945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:49.898945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:49.898945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:49.898945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:49.898945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:49.898945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:49.898945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:49.898945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 276/544] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:49.904176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a6d35-5f38-4597-acb2-747a4162ef84 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:49.917716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cacac2-641b-45f7-872d-e4ac36aa4e9e None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:49.918071 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cacac2-641b-45f7-872d-e4ac36aa4e9e None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:49.918313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cacac2-641b-45f7-872d-e4ac36aa4e9e None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:49.920041 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 277/545] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:50:49.928902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e086f0-ec9e-49a9-956f-db919fc53025 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:49.929339 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e086f0-ec9e-49a9-956f-db919fc53025 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:49.929723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e086f0-ec9e-49a9-956f-db919fc53025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:49.960769 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-b5e086f0-ec9e-49a9-956f-db919fc53025 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:49.962011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-b5e086f0-ec9e-49a9-956f-db919fc53025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:49.972710 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:50.048579 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a6d35-5f38-4597-acb2-747a4162ef84 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:50.085216 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72fd4a-aca7-4388-8348-b7ef1416cff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:50.085818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72fd4a-aca7-4388-8348-b7ef1416cff7 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:50.086103 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72fd4a-aca7-4388-8348-b7ef1416cff7 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:50.086407 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72fd4a-aca7-4388-8348-b7ef1416cff7 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:50.087161 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72fd4a-aca7-4388-8348-b7ef1416cff7 None admin] RBAC: Adding query filter params (name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:50.087633 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72fd4a-aca7-4388-8348-b7ef1416cff7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:50.089502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72fd4a-aca7-4388-8348-b7ef1416cff7 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:50.102486 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 269/546] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:50:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:50.105178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72fd4a-aca7-4388-8348-b7ef1416cff7 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:50.116801 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 278/547] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:50.155103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e086f0-ec9e-49a9-956f-db919fc53025 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:50.186635 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:50.213128 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72fd4a-aca7-4388-8348-b7ef1416cff7 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:50.226714 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:50.250595 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f20f8f-a990-46a9-af5e-221df48ee92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:50.251412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f20f8f-a990-46a9-af5e-221df48ee92c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:50.251702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f20f8f-a990-46a9-af5e-221df48ee92c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:50.251986 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f20f8f-a990-46a9-af5e-221df48ee92c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:50.263827 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f20f8f-a990-46a9-af5e-221df48ee92c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:50.265444 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f20f8f-a990-46a9-af5e-221df48ee92c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:50.283151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-28f20f8f-a990-46a9-af5e-221df48ee92c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:50.283151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:50.283151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:50.283151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:50.283151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:50.283151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:50.283151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:50.283151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:50.283151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:50.283151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:50.283151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:50.283151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:50.283151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:50.283151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:50.283151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:50.284204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:50.285196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:50.285196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:50.285196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:50.285196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:50.285196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 270/548] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:50.290161 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f20f8f-a990-46a9-af5e-221df48ee92c None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:50.393926 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f20f8f-a990-46a9-af5e-221df48ee92c None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:50.407296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e086f0-ec9e-49a9-956f-db919fc53025 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:50.429187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaa634-601f-493e-8115-32d81536f47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:50.429870 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaa634-601f-493e-8115-32d81536f47e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:50.430159 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaa634-601f-493e-8115-32d81536f47e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:50.430460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaa634-601f-493e-8115-32d81536f47e None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:50.431204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaa634-601f-493e-8115-32d81536f47e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:50.431521 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaa634-601f-493e-8115-32d81536f47e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:50.432914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaa634-601f-493e-8115-32d81536f47e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:50.445926 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef1e9a-daa7-496c-80aa-95af11235899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:50.446343 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 271/549] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:50.447449 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef1e9a-daa7-496c-80aa-95af11235899 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:50.447954 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef1e9a-daa7-496c-80aa-95af11235899 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:50.448441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef1e9a-daa7-496c-80aa-95af11235899 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:50.458339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaa634-601f-493e-8115-32d81536f47e None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:50.464558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:50.479985 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef1e9a-daa7-496c-80aa-95af11235899 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-c7ef1e9a-daa7-496c-80aa-95af11235899 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:50.494284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:50.495349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:50.496278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:50.496278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:50.496278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:50.496278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:50.496278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:50.496278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:50.496278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:50.496278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:50.496278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:50.496278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:50.496278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:50.496278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:50.499598 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 279/550] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:50.504722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef1e9a-daa7-496c-80aa-95af11235899 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:50.555139 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaa634-601f-493e-8115-32d81536f47e None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:50.590425 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c5fbd-4b06-4434-8222-56000088b653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:50.591238 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c5fbd-4b06-4434-8222-56000088b653 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:50.591521 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c5fbd-4b06-4434-8222-56000088b653 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:50.591855 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c5fbd-4b06-4434-8222-56000088b653 None admin] PATH_INFO: `/v3/users/swift` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:50.600209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c5fbd-4b06-4434-8222-56000088b653 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:50.601671 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c5fbd-4b06-4434-8222-56000088b653 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-f45c5fbd-4b06-4434-8222-56000088b653 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:50.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:50.619381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:50.620420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:50.620420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:50.620420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:50.620420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:50.620420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:50.620420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:50.620420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:50.620420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:50.620420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:50:50.620420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:50.620420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 272/551] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:50:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:50.626284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c5fbd-4b06-4434-8222-56000088b653 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:50.661395 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:50.696243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef1e9a-daa7-496c-80aa-95af11235899 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:50.727509 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:50.750989 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c5fbd-4b06-4434-8222-56000088b653 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:50.786072 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1beb03-3f8b-42e5-b82c-f8be14c49e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:50.786889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1beb03-3f8b-42e5-b82c-f8be14c49e1d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:50.787177 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1beb03-3f8b-42e5-b82c-f8be14c49e1d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:50.787455 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1beb03-3f8b-42e5-b82c-f8be14c49e1d None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:50.788414 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1beb03-3f8b-42e5-b82c-f8be14c49e1d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:50.788731 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1beb03-3f8b-42e5-b82c-f8be14c49e1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:50.789373 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02ab3a-d995-4cb0-85f3-f3d1d19a2dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:50.790178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1beb03-3f8b-42e5-b82c-f8be14c49e1d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:50.790439 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02ab3a-d995-4cb0-85f3-f3d1d19a2dae None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:50.790934 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02ab3a-d995-4cb0-85f3-f3d1d19a2dae None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:50.791427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02ab3a-d995-4cb0-85f3-f3d1d19a2dae None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:50.792412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02ab3a-d995-4cb0-85f3-f3d1d19a2dae None admin] RBAC: Adding query filter params (name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:50.792943 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02ab3a-d995-4cb0-85f3-f3d1d19a2dae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:50.794565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02ab3a-d995-4cb0-85f3-f3d1d19a2dae None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:50.804437 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 280/552] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:50:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:50.815459 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02ab3a-d995-4cb0-85f3-f3d1d19a2dae None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:50.817388 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 273/553] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 19:50:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:50.822767 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1beb03-3f8b-42e5-b82c-f8be14c49e1d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:50.926002 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1beb03-3f8b-42e5-b82c-f8be14c49e1d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:50.957784 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:50.966963 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076de73-9ca5-4084-a163-652c18d7a250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:50.967725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076de73-9ca5-4084-a163-652c18d7a250 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:50.968037 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076de73-9ca5-4084-a163-652c18d7a250 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:50.968454 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076de73-9ca5-4084-a163-652c18d7a250 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:51.037168 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02ab3a-d995-4cb0-85f3-f3d1d19a2dae None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:51.040489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076de73-9ca5-4084-a163-652c18d7a250 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:51.042090 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076de73-9ca5-4084-a163-652c18d7a250 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:51.105100 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89aa276-2fbf-4cbe-b430-a59551d9a1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:51.107437 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89aa276-2fbf-4cbe-b430-a59551d9a1ff None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:51.107941 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89aa276-2fbf-4cbe-b430-a59551d9a1ff None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:51.108427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89aa276-2fbf-4cbe-b430-a59551d9a1ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:51.112187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:51.124058 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89aa276-2fbf-4cbe-b430-a59551d9a1ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:51.126476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89aa276-2fbf-4cbe-b430-a59551d9a1ff None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:51.151208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-c89aa276-2fbf-4cbe-b430-a59551d9a1ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:51.151208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:51.151208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:51.151208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:51.151208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:51.151208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:51.151208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:51.151208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:51.151208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:51.151208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:51.151208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:51.151208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:51.151208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:51.151208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:51.151208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:51.152419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:51.153603 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:51.153603 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:51.153603 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:51.153603 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:51.153603 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 281/554] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:51.158970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd865a4-8d88-48ee-b097-ce14ee6f389b None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:51.159268 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd865a4-8d88-48ee-b097-ce14ee6f389b None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:51.159544 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd865a4-8d88-48ee-b097-ce14ee6f389b None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:51.161048 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 282/555] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:51.164811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd865a4-8d88-48ee-b097-ce14ee6f389b None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:51.172821 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 274/556] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:50:51.174406 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e1c67-dd63-41ce-996e-ae73f44a0d00 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:51.174698 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e1c67-dd63-41ce-996e-ae73f44a0d00 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:51.174979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e1c67-dd63-41ce-996e-ae73f44a0d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:51.193645 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-fd5e1c67-dd63-41ce-996e-ae73f44a0d00 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:51.194804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-fd5e1c67-dd63-41ce-996e-ae73f44a0d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:51.220344 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:51.274296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 275/557] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:51.274296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e1c67-dd63-41ce-996e-ae73f44a0d00 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:51.315862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd865a4-8d88-48ee-b097-ce14ee6f389b None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:51.351928 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0648d7e-565b-44de-8225-80ac2bb83db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:51.353129 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0648d7e-565b-44de-8225-80ac2bb83db5 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:51.353419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0648d7e-565b-44de-8225-80ac2bb83db5 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:51.353698 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0648d7e-565b-44de-8225-80ac2bb83db5 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:51.354483 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0648d7e-565b-44de-8225-80ac2bb83db5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:51.354799 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0648d7e-565b-44de-8225-80ac2bb83db5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:51.356242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0648d7e-565b-44de-8225-80ac2bb83db5 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:51.371385 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 283/558] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:51.376381 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0648d7e-565b-44de-8225-80ac2bb83db5 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:51.415942 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e1c67-dd63-41ce-996e-ae73f44a0d00 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:51.427706 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:51.464908 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db55fb-27dd-4ebd-b3ab-5a01c69ea276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:51.466779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db55fb-27dd-4ebd-b3ab-5a01c69ea276 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:51.467431 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db55fb-27dd-4ebd-b3ab-5a01c69ea276 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:51.468056 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db55fb-27dd-4ebd-b3ab-5a01c69ea276 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:51.482505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db55fb-27dd-4ebd-b3ab-5a01c69ea276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:51.484030 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db55fb-27dd-4ebd-b3ab-5a01c69ea276 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:51.489467 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0648d7e-565b-44de-8225-80ac2bb83db5 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:51.501951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-88db55fb-27dd-4ebd-b3ab-5a01c69ea276 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:51.501951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:51.501951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:51.501951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:51.501951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:51.501951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:51.501951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:51.501951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:51.501951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:51.501951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:51.501951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:51.501951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:51.501951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:51.501951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:51.501951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:51.503097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:51.504079 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:51.504079 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:51.504079 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:51.504079 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:51.504079 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 276/559] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:51.504830 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db55fb-27dd-4ebd-b3ab-5a01c69ea276 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:51.517682 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:51.540720 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50883be-6be0-47b2-afd5-43b5ea4b8e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:51.542767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50883be-6be0-47b2-afd5-43b5ea4b8e0c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:51.543054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50883be-6be0-47b2-afd5-43b5ea4b8e0c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:51.543330 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50883be-6be0-47b2-afd5-43b5ea4b8e0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:51.557035 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50883be-6be0-47b2-afd5-43b5ea4b8e0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-f50883be-6be0-47b2-afd5-43b5ea4b8e0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:51.566683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:51.567761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:51.568821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:51.568821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:51.568821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:51.568821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:51.568821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:51.568821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:51.568821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:51.568821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:51.568821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:51.568821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:51.568821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:51.568821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:51.568821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 284/560] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:51.568821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50883be-6be0-47b2-afd5-43b5ea4b8e0c None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:51.617847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db55fb-27dd-4ebd-b3ab-5a01c69ea276 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:51.656714 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d137ff-7a24-4a3c-aae8-a90e4b3bc95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:51.657800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d137ff-7a24-4a3c-aae8-a90e4b3bc95c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:51.658084 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d137ff-7a24-4a3c-aae8-a90e4b3bc95c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:51.658386 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d137ff-7a24-4a3c-aae8-a90e4b3bc95c None admin] PATH_INFO: `/v3/users/nova` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:51.666373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d137ff-7a24-4a3c-aae8-a90e4b3bc95c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:51.667730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d137ff-7a24-4a3c-aae8-a90e4b3bc95c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:51.672265 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50883be-6be0-47b2-afd5-43b5ea4b8e0c None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:51.679468 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-30d137ff-7a24-4a3c-aae8-a90e4b3bc95c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:51.684548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:51.685639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:51.687133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:51.687133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:51.687133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:51.687133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:51.687133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:51.687133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:51.687133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:51.687133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:51.687133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:50:51.687133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:51.687133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 277/561] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:50:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:51.687133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed73aa-8256-4346-af23-7c641fb560a0 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:51.687987 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed73aa-8256-4346-af23-7c641fb560a0 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:51.687987 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed73aa-8256-4346-af23-7c641fb560a0 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:51.693135 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 278/562] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:51.693135 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed73aa-8256-4346-af23-7c641fb560a0 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:51.719701 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a17e90-a2f8-466e-aebe-cf8c44a8ca93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:51.720926 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a17e90-a2f8-466e-aebe-cf8c44a8ca93 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:51.721303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a17e90-a2f8-466e-aebe-cf8c44a8ca93 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:51.721769 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a17e90-a2f8-466e-aebe-cf8c44a8ca93 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:51.722939 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a17e90-a2f8-466e-aebe-cf8c44a8ca93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:51.723366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a17e90-a2f8-466e-aebe-cf8c44a8ca93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:51.724927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a17e90-a2f8-466e-aebe-cf8c44a8ca93 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:51.737602 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 285/563] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:51.742539 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a17e90-a2f8-466e-aebe-cf8c44a8ca93 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:51.774977 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:51.810009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed73aa-8256-4346-af23-7c641fb560a0 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:51.845446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef3229-baa8-4fe4-8d59-38ecc2cf86df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:51.846285 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef3229-baa8-4fe4-8d59-38ecc2cf86df None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:51.846504 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a17e90-a2f8-466e-aebe-cf8c44a8ca93 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:51.846796 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef3229-baa8-4fe4-8d59-38ecc2cf86df None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:51.847475 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef3229-baa8-4fe4-8d59-38ecc2cf86df None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:51.848328 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef3229-baa8-4fe4-8d59-38ecc2cf86df None admin] RBAC: Adding query filter params (name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:51.848732 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef3229-baa8-4fe4-8d59-38ecc2cf86df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:51.850229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef3229-baa8-4fe4-8d59-38ecc2cf86df None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:51.863861 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 279/564] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:50:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:51.865273 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181a31f-9d21-4404-b365-6c426df0fd57 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:51.865532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181a31f-9d21-4404-b365-6c426df0fd57 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:51.865838 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181a31f-9d21-4404-b365-6c426df0fd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:51.883808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777cbb9-0ac9-4232-a266-53b519ced644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:51.884276 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-2181a31f-9d21-4404-b365-6c426df0fd57 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:51.884463 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777cbb9-0ac9-4232-a266-53b519ced644 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:51.884760 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777cbb9-0ac9-4232-a266-53b519ced644 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:51.885034 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777cbb9-0ac9-4232-a266-53b519ced644 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:51.885474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-2181a31f-9d21-4404-b365-6c426df0fd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:51.885882 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777cbb9-0ac9-4232-a266-53b519ced644 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:51.886190 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777cbb9-0ac9-4232-a266-53b519ced644 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:51.887647 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777cbb9-0ac9-4232-a266-53b519ced644 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:51.910415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 286/565] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:50:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:51.910415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777cbb9-0ac9-4232-a266-53b519ced644 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:51.934863 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:51.966672 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 280/566] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:51.966672 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181a31f-9d21-4404-b365-6c426df0fd57 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:52.024534 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777cbb9-0ac9-4232-a266-53b519ced644 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:52.050365 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:52.061512 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14c5ec-e7c6-4ce1-b4eb-96f62aafb7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:52.062416 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14c5ec-e7c6-4ce1-b4eb-96f62aafb7e3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:52.062700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14c5ec-e7c6-4ce1-b4eb-96f62aafb7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:52.062972 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14c5ec-e7c6-4ce1-b4eb-96f62aafb7e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:52.070303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14c5ec-e7c6-4ce1-b4eb-96f62aafb7e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:52.071814 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14c5ec-e7c6-4ce1-b4eb-96f62aafb7e3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:52.086636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-eb14c5ec-e7c6-4ce1-b4eb-96f62aafb7e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:52.086636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:52.086636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:52.086636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:52.086636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:52.086636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:52.086636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:52.086636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:52.086636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:52.086636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:52.086636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:52.086636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:52.086636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:52.086636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:52.086636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:52.087908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:52.089026 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:52.089026 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:52.089026 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:52.089026 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:52.089026 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:52.089026 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:52.089026 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:52.089026 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:52.089026 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 287/567] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:52.089026 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14c5ec-e7c6-4ce1-b4eb-96f62aafb7e3 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:52.104474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181a31f-9d21-4404-b365-6c426df0fd57 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:52.141139 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1ac53-0a42-43cf-bc6f-8e0bc21f7b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:52.141952 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1ac53-0a42-43cf-bc6f-8e0bc21f7b11 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:52.142231 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1ac53-0a42-43cf-bc6f-8e0bc21f7b11 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:52.142536 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1ac53-0a42-43cf-bc6f-8e0bc21f7b11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:52.152944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1ac53-0a42-43cf-bc6f-8e0bc21f7b11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:52.154452 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1ac53-0a42-43cf-bc6f-8e0bc21f7b11 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:52.172871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-d3a1ac53-0a42-43cf-bc6f-8e0bc21f7b11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:52.172871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:52.172871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:52.172871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:52.172871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:52.172871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:52.172871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:52.172871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:52.172871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:52.172871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:52.172871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:52.172871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:52.172871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:52.172871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:52.172871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:52.173927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:52.175333 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:52.175333 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:52.175333 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:52.175333 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:52.178797 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 281/568] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:52.179870 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1ac53-0a42-43cf-bc6f-8e0bc21f7b11 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:52.186652 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:52.203840 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14c5ec-e7c6-4ce1-b4eb-96f62aafb7e3 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:52.238699 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45272f98-1c53-4e02-a02e-b90a87118753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:52.239422 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45272f98-1c53-4e02-a02e-b90a87118753 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:52.239704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45272f98-1c53-4e02-a02e-b90a87118753 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:52.239979 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45272f98-1c53-4e02-a02e-b90a87118753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:52.252757 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45272f98-1c53-4e02-a02e-b90a87118753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:52.254241 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45272f98-1c53-4e02-a02e-b90a87118753 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:52.271640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-45272f98-1c53-4e02-a02e-b90a87118753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:52.271640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:52.271640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:52.271640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:52.271640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:52.271640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:52.271640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:52.271640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:52.271640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:52.271640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:52.271640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:52.271640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:52.271640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:52.271640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:52.271640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:52.272843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:52.274164 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:52.274164 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:52.274164 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:52.274164 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:52.274164 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 288/569] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:52.274164 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45272f98-1c53-4e02-a02e-b90a87118753 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:52.297274 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1ac53-0a42-43cf-bc6f-8e0bc21f7b11 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:52.311855 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:52.332250 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63face-7b8e-4e61-99a6-2d114334e1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:52.333062 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63face-7b8e-4e61-99a6-2d114334e1f9 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:52.333345 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63face-7b8e-4e61-99a6-2d114334e1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:52.333632 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63face-7b8e-4e61-99a6-2d114334e1f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:52.344870 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63face-7b8e-4e61-99a6-2d114334e1f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-be63face-7b8e-4e61-99a6-2d114334e1f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:52.352945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:52.354170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:52.355184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:52.355184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:52.355184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:52.355184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:52.355184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:52.355184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:52.355184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:52.355184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:52.355184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:52.355184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:52.355184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:52.355184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:52.355184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 282/570] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:52.355184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63face-7b8e-4e61-99a6-2d114334e1f9 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:52.380710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45272f98-1c53-4e02-a02e-b90a87118753 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:52.413628 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cccb1e-3027-48e1-809b-5a7608f23107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:52.414312 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cccb1e-3027-48e1-809b-5a7608f23107 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:52.414595 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cccb1e-3027-48e1-809b-5a7608f23107 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:52.414867 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cccb1e-3027-48e1-809b-5a7608f23107 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:52.415660 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cccb1e-3027-48e1-809b-5a7608f23107 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:52.415969 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cccb1e-3027-48e1-809b-5a7608f23107 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:52.417415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cccb1e-3027-48e1-809b-5a7608f23107 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:52.420051 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:52.431442 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 289/571] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:50:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:52.432729 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003d3a7-83b1-4801-b35c-e879fe340aea None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:52.433015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003d3a7-83b1-4801-b35c-e879fe340aea None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:52.433293 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003d3a7-83b1-4801-b35c-e879fe340aea None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:52.434570 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 290/572] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:52.435254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003d3a7-83b1-4801-b35c-e879fe340aea None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:52.460025 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63face-7b8e-4e61-99a6-2d114334e1f9 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:52.493540 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effe90e-db90-4d2e-89c7-e42129b29b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:52.494144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effe90e-db90-4d2e-89c7-e42129b29b57 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:52.494456 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effe90e-db90-4d2e-89c7-e42129b29b57 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:52.494723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effe90e-db90-4d2e-89c7-e42129b29b57 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:52.495463 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effe90e-db90-4d2e-89c7-e42129b29b57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:52.495770 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effe90e-db90-4d2e-89c7-e42129b29b57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:52.497127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effe90e-db90-4d2e-89c7-e42129b29b57 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:52.508927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 283/573] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:52.509521 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effe90e-db90-4d2e-89c7-e42129b29b57 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:52.536591 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003d3a7-83b1-4801-b35c-e879fe340aea None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:52.543439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:52.570788 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb0d18-e0a5-4186-b0a8-04c28f050a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:52.571455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb0d18-e0a5-4186-b0a8-04c28f050a48 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:52.571735 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb0d18-e0a5-4186-b0a8-04c28f050a48 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:52.572007 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb0d18-e0a5-4186-b0a8-04c28f050a48 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:52.572737 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb0d18-e0a5-4186-b0a8-04c28f050a48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:52.573047 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb0d18-e0a5-4186-b0a8-04c28f050a48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:52.574451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb0d18-e0a5-4186-b0a8-04c28f050a48 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:52.585575 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 291/574] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:52.586389 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb0d18-e0a5-4186-b0a8-04c28f050a48 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:52.613737 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effe90e-db90-4d2e-89c7-e42129b29b57 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:52.642897 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:52.648112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508d988-4405-4951-905a-c494d66c7f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:52.648789 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508d988-4405-4951-905a-c494d66c7f2d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:52.649082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508d988-4405-4951-905a-c494d66c7f2d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:52.649349 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508d988-4405-4951-905a-c494d66c7f2d None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:52.650108 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508d988-4405-4951-905a-c494d66c7f2d None admin] RBAC: Adding query filter params (name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:52.650453 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508d988-4405-4951-905a-c494d66c7f2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:52.651828 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508d988-4405-4951-905a-c494d66c7f2d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:52.660489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 284/575] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:50:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:52.661630 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67819e62-4726-404d-8539-97c94eaeccb0 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:52.662032 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67819e62-4726-404d-8539-97c94eaeccb0 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:52.662441 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67819e62-4726-404d-8539-97c94eaeccb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:52.679695 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-67819e62-4726-404d-8539-97c94eaeccb0 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:52.680894 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-67819e62-4726-404d-8539-97c94eaeccb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:52.688467 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb0d18-e0a5-4186-b0a8-04c28f050a48 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:52.726811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67746e-830a-43e5-b0bc-53ce2013a0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:52.727487 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67746e-830a-43e5-b0bc-53ce2013a0a6 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:52.727769 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67746e-830a-43e5-b0bc-53ce2013a0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:52.728052 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67746e-830a-43e5-b0bc-53ce2013a0a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:52.729142 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67746e-830a-43e5-b0bc-53ce2013a0a6 None admin] RBAC: Adding query filter params (role.id=3387bd4782b047ae8d880e386b0351f0, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=acebbcf1f90d487aa21a8eb99102ce09) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:52.729267 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67746e-830a-43e5-b0bc-53ce2013a0a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:52.730767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67746e-830a-43e5-b0bc-53ce2013a0a6 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:52.736457 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 292/576] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:50:52 2022] GET /identity/v3/role_assignments?user.id=acebbcf1f90d487aa21a8eb99102ce09&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=3387bd4782b047ae8d880e386b0351f0 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:52.737240 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67746e-830a-43e5-b0bc-53ce2013a0a6 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:52.746637 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 285/577] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:52.747455 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67819e62-4726-404d-8539-97c94eaeccb0 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:52.782380 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:52.863489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67746e-830a-43e5-b0bc-53ce2013a0a6 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:52.864514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67819e62-4726-404d-8539-97c94eaeccb0 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:52.889122 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:52.904891 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d1fbd-d82a-4abc-975d-ec7213a49f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:52.905171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b32ed7-1568-4d12-9ed7-7e11657a9c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:52.905755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d1fbd-d82a-4abc-975d-ec7213a49f54 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:52.906034 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d1fbd-d82a-4abc-975d-ec7213a49f54 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:52.906159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b32ed7-1568-4d12-9ed7-7e11657a9c17 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:52.906325 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d1fbd-d82a-4abc-975d-ec7213a49f54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:52.906646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b32ed7-1568-4d12-9ed7-7e11657a9c17 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:52.907018 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b32ed7-1568-4d12-9ed7-7e11657a9c17 None admin] PATH_INFO: `/v3/users/glance` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:52.914393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d1fbd-d82a-4abc-975d-ec7213a49f54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:52.915785 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b32ed7-1568-4d12-9ed7-7e11657a9c17 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:52.915935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d1fbd-d82a-4abc-975d-ec7213a49f54 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:52.917364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b32ed7-1568-4d12-9ed7-7e11657a9c17 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:52.930405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-0b9d1fbd-d82a-4abc-975d-ec7213a49f54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:52.930405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:52.930405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:52.930405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:52.930405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:52.930405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:52.930405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:52.930405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:52.930405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:52.930405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:52.930405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:52.930405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:52.930405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:52.930405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:52.930405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:52.931437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:52.933564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:52.933564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:52.933564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:52.933564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:52.933564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:52.933564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:52.933564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:52.933564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:52.933564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 286/578] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:52.933564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d1fbd-d82a-4abc-975d-ec7213a49f54 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-20b32ed7-1568-4d12-9ed7-7e11657a9c17 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:52.934187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:52.935492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:52.936851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:52.936851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:52.936851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:52.936851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:52.936851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:52.936851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:52.936851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:52.936851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:52.936851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 19:50:52.936851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:52.936851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 293/579] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:52.936851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b32ed7-1568-4d12-9ed7-7e11657a9c17 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:53.031394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:53.047067 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d1fbd-d82a-4abc-975d-ec7213a49f54 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:53.052863 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b32ed7-1568-4d12-9ed7-7e11657a9c17 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:53.084808 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d1dd2-efc9-4188-bd2b-83282b2999b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:53.085773 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d1dd2-efc9-4188-bd2b-83282b2999b5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:53.086061 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d1dd2-efc9-4188-bd2b-83282b2999b5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:53.086361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d1dd2-efc9-4188-bd2b-83282b2999b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:53.092017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc8444-351f-45ac-9259-320007536222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:53.094376 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc8444-351f-45ac-9259-320007536222 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:53.094739 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc8444-351f-45ac-9259-320007536222 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:53.095021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc8444-351f-45ac-9259-320007536222 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:53.099855 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d1dd2-efc9-4188-bd2b-83282b2999b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:53.101398 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d1dd2-efc9-4188-bd2b-83282b2999b5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:53.107772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc8444-351f-45ac-9259-320007536222 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-5bfc8444-351f-45ac-9259-320007536222 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:53.116042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:53.122159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:53.123254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:53.123254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:53.123254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:53.123254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:53.123254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:53.123254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:53.123254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:53.123254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:53.123254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:53.123254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:53.123254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:53.123254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:53.123254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 294/580] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:53.123254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc8444-351f-45ac-9259-320007536222 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-443d1dd2-efc9-4188-bd2b-83282b2999b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:53.125197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:53.128258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:53.128258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:53.128258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:53.128258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:53.128258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 287/581] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:53.128258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d1dd2-efc9-4188-bd2b-83282b2999b5 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:53.164126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:53.234534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d1dd2-efc9-4188-bd2b-83282b2999b5 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:53.235277 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc8444-351f-45ac-9259-320007536222 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:53.270593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41bb34-1c57-48a5-a2f9-37f8da901aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:53.271381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41bb34-1c57-48a5-a2f9-37f8da901aa6 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:53.271744 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41bb34-1c57-48a5-a2f9-37f8da901aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:53.272023 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41bb34-1c57-48a5-a2f9-37f8da901aa6 None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:53.272878 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41bb34-1c57-48a5-a2f9-37f8da901aa6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:53.273191 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41bb34-1c57-48a5-a2f9-37f8da901aa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:53.273528 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c6ccf-94c6-44e1-a855-5e83bd4cab5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:53.274391 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c6ccf-94c6-44e1-a855-5e83bd4cab5a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:53.274671 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41bb34-1c57-48a5-a2f9-37f8da901aa6 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:53.274982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c6ccf-94c6-44e1-a855-5e83bd4cab5a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:53.275398 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c6ccf-94c6-44e1-a855-5e83bd4cab5a None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:53.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c6ccf-94c6-44e1-a855-5e83bd4cab5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:53.276969 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c6ccf-94c6-44e1-a855-5e83bd4cab5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:53.279381 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c6ccf-94c6-44e1-a855-5e83bd4cab5a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:53.284684 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:53.293791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 295/582] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:50:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:53.294485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c6ccf-94c6-44e1-a855-5e83bd4cab5a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:53.295727 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 288/583] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 19:50:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:53.296836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41bb34-1c57-48a5-a2f9-37f8da901aa6 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:53.400127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:53.403812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c6ccf-94c6-44e1-a855-5e83bd4cab5a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:53.403940 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41bb34-1c57-48a5-a2f9-37f8da901aa6 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:53.441511 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8ae1a-d64e-4352-870e-4f65229e6a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:53.442252 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8ae1a-d64e-4352-870e-4f65229e6a96 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:53.442502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8ae1a-d64e-4352-870e-4f65229e6a96 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:53.442777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8ae1a-d64e-4352-870e-4f65229e6a96 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:53.443543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8ae1a-d64e-4352-870e-4f65229e6a96 None admin] RBAC: Adding query filter params (name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:53.443848 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8ae1a-d64e-4352-870e-4f65229e6a96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:53.443963 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfad0ab-4bd3-4ae8-a5d2-b572a80a3579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:53.444519 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfad0ab-4bd3-4ae8-a5d2-b572a80a3579 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:53.444800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfad0ab-4bd3-4ae8-a5d2-b572a80a3579 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:53.445068 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfad0ab-4bd3-4ae8-a5d2-b572a80a3579 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:53.445225 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8ae1a-d64e-4352-870e-4f65229e6a96 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:53.445789 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfad0ab-4bd3-4ae8-a5d2-b572a80a3579 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:53.446094 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfad0ab-4bd3-4ae8-a5d2-b572a80a3579 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:53.447709 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfad0ab-4bd3-4ae8-a5d2-b572a80a3579 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:53.458481 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 296/584] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:50:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:53.458481 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8ae1a-d64e-4352-870e-4f65229e6a96 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:53.468965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 289/585] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:53.469684 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfad0ab-4bd3-4ae8-a5d2-b572a80a3579 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:53.531254 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:53.565526 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8ae1a-d64e-4352-870e-4f65229e6a96 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:53.582146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfad0ab-4bd3-4ae8-a5d2-b572a80a3579 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:53.604798 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b4ded-6785-4acc-ad70-5778910c7ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:53.605709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b4ded-6785-4acc-ad70-5778910c7ddd None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:53.605994 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b4ded-6785-4acc-ad70-5778910c7ddd None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:53.606280 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b4ded-6785-4acc-ad70-5778910c7ddd None admin] PATH_INFO: `/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/0da7f490d9604107b974e9e461ebc38b/roles/3387bd4782b047ae8d880e386b0351f0` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:53.617590 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42f08e-85a5-4552-9ea1-d58bb0140a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:53.618630 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42f08e-85a5-4552-9ea1-d58bb0140a81 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:53.619081 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42f08e-85a5-4552-9ea1-d58bb0140a81 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:53.619353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42f08e-85a5-4552-9ea1-d58bb0140a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:53.622874 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b4ded-6785-4acc-ad70-5778910c7ddd None admin] RBAC: Authorizing `identity:create_grant(project_id=d638a29a793d41e7bfa686c0fcc29149, user_id=0da7f490d9604107b974e9e461ebc38b, role_id=3387bd4782b047ae8d880e386b0351f0)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:53.624395 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b4ded-6785-4acc-ad70-5778910c7ddd None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:53.630094 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42f08e-85a5-4552-9ea1-d58bb0140a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:53.631580 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42f08e-85a5-4552-9ea1-d58bb0140a81 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:53.646298 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 297/586] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:50:53 2022] PUT /identity/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/0da7f490d9604107b974e9e461ebc38b/roles/3387bd4782b047ae8d880e386b0351f0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:53.646298 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b4ded-6785-4acc-ad70-5778910c7ddd None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:53.649391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-8d42f08e-85a5-4552-9ea1-d58bb0140a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:53.649391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:53.649391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:53.649391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:53.649391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:53.649391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:53.649391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:53.649391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:53.649391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:53.649391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:53.649391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:53.649391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:53.649391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:53.649391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:53.649391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:53.650751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:53.651768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:53.651768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:53.651768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:53.651768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:53.651768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 290/587] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:53.651768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42f08e-85a5-4552-9ea1-d58bb0140a81 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:53.672896 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:53.776399 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42f08e-85a5-4552-9ea1-d58bb0140a81 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:53.787543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:53.798553 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b4ded-6785-4acc-ad70-5778910c7ddd None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:53.819049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4634b1-226c-4d76-9c7b-113d990b3181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:53.820271 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4634b1-226c-4d76-9c7b-113d990b3181 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:53.820769 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4634b1-226c-4d76-9c7b-113d990b3181 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:53.821262 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4634b1-226c-4d76-9c7b-113d990b3181 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:53.830650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4634b1-226c-4d76-9c7b-113d990b3181 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:53.832529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4634b1-226c-4d76-9c7b-113d990b3181 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:53.838904 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381acd6b-152f-45c0-b68a-393cd924a202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:53.839924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381acd6b-152f-45c0-b68a-393cd924a202 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:53.840207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381acd6b-152f-45c0-b68a-393cd924a202 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:53.840482 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381acd6b-152f-45c0-b68a-393cd924a202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-ea4634b1-226c-4d76-9c7b-113d990b3181 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:53.851170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:53.852290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:53.853334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:53.853334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:53.853334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:53.853334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:53.853334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:53.853334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:53.853334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:53.853334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:53.853334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:50:53.853334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:53.853940 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381acd6b-152f-45c0-b68a-393cd924a202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:53.856174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381acd6b-152f-45c0-b68a-393cd924a202 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:53.856878 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 291/588] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:53.862774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4634b1-226c-4d76-9c7b-113d990b3181 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:53.877808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-381acd6b-152f-45c0-b68a-393cd924a202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:53.877808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:53.877808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:53.877808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:53.877808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:53.877808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:53.877808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:53.877808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:53.877808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:53.877808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:53.877808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:53.877808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:53.877808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:53.877808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:53.877808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:53.879006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:53.880017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:53.880017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:53.880017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:53.880017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:53.880017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 298/589] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:53.880017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381acd6b-152f-45c0-b68a-393cd924a202 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:53.956497 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:53.991101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4634b1-226c-4d76-9c7b-113d990b3181 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:54.006671 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381acd6b-152f-45c0-b68a-393cd924a202 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:54.026702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6cbd8-4573-49d3-812d-1b6a6911fc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:54.027377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6cbd8-4573-49d3-812d-1b6a6911fc41 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:54.027661 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6cbd8-4573-49d3-812d-1b6a6911fc41 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:54.027935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6cbd8-4573-49d3-812d-1b6a6911fc41 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:54.028727 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6cbd8-4573-49d3-812d-1b6a6911fc41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:54.029041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6cbd8-4573-49d3-812d-1b6a6911fc41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:54.030866 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6cbd8-4573-49d3-812d-1b6a6911fc41 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:54.043333 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 292/590] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:54.044194 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6cbd8-4573-49d3-812d-1b6a6911fc41 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:54.048436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e557f-dbc1-4e75-8a19-7847cfd3b823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:54.049186 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e557f-dbc1-4e75-8a19-7847cfd3b823 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:54.049471 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e557f-dbc1-4e75-8a19-7847cfd3b823 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:54.049748 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e557f-dbc1-4e75-8a19-7847cfd3b823 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:54.050677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e557f-dbc1-4e75-8a19-7847cfd3b823 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:54.050992 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e557f-dbc1-4e75-8a19-7847cfd3b823 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:54.052487 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e557f-dbc1-4e75-8a19-7847cfd3b823 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:54.072645 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 299/591] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 19:50:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:54.073533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e557f-dbc1-4e75-8a19-7847cfd3b823 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:54.076396 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:54.171445 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6cbd8-4573-49d3-812d-1b6a6911fc41 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:54.186613 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e557f-dbc1-4e75-8a19-7847cfd3b823 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:54.212940 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:54.220438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab28cb6-4b25-4996-be66-c848284c0a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:54.221127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab28cb6-4b25-4996-be66-c848284c0a66 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:54.221826 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab28cb6-4b25-4996-be66-c848284c0a66 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:54.222486 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab28cb6-4b25-4996-be66-c848284c0a66 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:54.223517 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab28cb6-4b25-4996-be66-c848284c0a66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:54.223829 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab28cb6-4b25-4996-be66-c848284c0a66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:54.225224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab28cb6-4b25-4996-be66-c848284c0a66 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:54.228087 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29619b-6c7d-44bc-ac9a-6c7a13b028bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:54.229171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29619b-6c7d-44bc-ac9a-6c7a13b028bf None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:54.229450 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29619b-6c7d-44bc-ac9a-6c7a13b028bf None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:54.229723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29619b-6c7d-44bc-ac9a-6c7a13b028bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:54.237813 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29619b-6c7d-44bc-ac9a-6c7a13b028bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:54.239414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29619b-6c7d-44bc-ac9a-6c7a13b028bf None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:54.239742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 293/592] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:54.240694 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab28cb6-4b25-4996-be66-c848284c0a66 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:54.255516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-dd29619b-6c7d-44bc-ac9a-6c7a13b028bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:54.255516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:54.255516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:54.255516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:54.255516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:54.255516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:54.255516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:54.255516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:54.255516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:54.255516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:54.255516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:54.255516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:54.255516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:54.255516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:54.255516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:54.256759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:54.257892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:54.257892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:54.257892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:54.257892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:54.257892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:54.257892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:54.257892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:54.257892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:54.257892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 300/593] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:54.257892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29619b-6c7d-44bc-ac9a-6c7a13b028bf None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:54.359212 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:54.371647 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29619b-6c7d-44bc-ac9a-6c7a13b028bf None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:54.376826 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab28cb6-4b25-4996-be66-c848284c0a66 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:54.410009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c4311-8776-4e9d-8f21-da16556694cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:54.410916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c4311-8776-4e9d-8f21-da16556694cf None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:54.411334 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c4311-8776-4e9d-8f21-da16556694cf None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:54.411708 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c4311-8776-4e9d-8f21-da16556694cf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:54.420333 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c4311-8776-4e9d-8f21-da16556694cf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:54.421963 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c4311-8776-4e9d-8f21-da16556694cf None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:54.422617 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68323993-3368-46a8-869d-04bf9e771e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:54.423605 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68323993-3368-46a8-869d-04bf9e771e34 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:54.424309 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68323993-3368-46a8-869d-04bf9e771e34 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:54.424746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68323993-3368-46a8-869d-04bf9e771e34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:54.437072 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68323993-3368-46a8-869d-04bf9e771e34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-4f7c4311-8776-4e9d-8f21-da16556694cf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:54.437916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:54.439160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:54.440432 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:54.440432 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:54.440432 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:54.440432 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:54.440432 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:54.440432 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:54.440432 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:54.440432 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:54.440432 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:50:54.440432 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:54.442148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68323993-3368-46a8-869d-04bf9e771e34 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:54.442520 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 301/594] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:54.443467 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c4311-8776-4e9d-8f21-da16556694cf None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:54.465563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-68323993-3368-46a8-869d-04bf9e771e34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:54.465563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:54.465563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:54.465563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:54.465563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:54.465563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:54.465563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:54.465563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:54.465563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:54.465563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:54.465563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:54.465563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:54.465563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:54.465563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:54.465563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:54.466857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:54.468095 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:54.468095 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:54.468095 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:54.468095 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:54.470851 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 294/595] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:54.472510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68323993-3368-46a8-869d-04bf9e771e34 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:54.498032 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:54.559808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c4311-8776-4e9d-8f21-da16556694cf None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:54.596762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4532f7b-586d-43b2-b0c5-63048db95267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:54.597931 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4532f7b-586d-43b2-b0c5-63048db95267 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:54.598344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4532f7b-586d-43b2-b0c5-63048db95267 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:54.598739 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4532f7b-586d-43b2-b0c5-63048db95267 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:54.600027 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4532f7b-586d-43b2-b0c5-63048db95267 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:54.600215 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4532f7b-586d-43b2-b0c5-63048db95267 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:54.601778 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4532f7b-586d-43b2-b0c5-63048db95267 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:54.610593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68323993-3368-46a8-869d-04bf9e771e34 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:54.617901 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 302/596] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:50:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:54.618926 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4532f7b-586d-43b2-b0c5-63048db95267 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:54.655883 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304bf80-2676-4bcb-8c8d-08b2b315f86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:54.658123 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304bf80-2676-4bcb-8c8d-08b2b315f86d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:54.658676 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304bf80-2676-4bcb-8c8d-08b2b315f86d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:54.659180 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304bf80-2676-4bcb-8c8d-08b2b315f86d None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:54.660259 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304bf80-2676-4bcb-8c8d-08b2b315f86d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:54.660785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304bf80-2676-4bcb-8c8d-08b2b315f86d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:54.662975 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304bf80-2676-4bcb-8c8d-08b2b315f86d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:54.675437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:54.687271 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 295/597] 10.222.0.84 () {56 vars in 1139 bytes} [Thu May 19 19:50:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:54.688400 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304bf80-2676-4bcb-8c8d-08b2b315f86d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:54.744818 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4532f7b-586d-43b2-b0c5-63048db95267 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:54.746356 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:54.792421 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cacf8-8af1-4343-a82c-a91a3c7bbdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:54.793097 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cacf8-8af1-4343-a82c-a91a3c7bbdbf None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:54.793380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cacf8-8af1-4343-a82c-a91a3c7bbdbf None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:54.793658 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cacf8-8af1-4343-a82c-a91a3c7bbdbf None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:54.794861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cacf8-8af1-4343-a82c-a91a3c7bbdbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:54.795325 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cacf8-8af1-4343-a82c-a91a3c7bbdbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:54.796857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cacf8-8af1-4343-a82c-a91a3c7bbdbf None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:54.797680 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304bf80-2676-4bcb-8c8d-08b2b315f86d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:54.814655 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 303/598] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:54.816297 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cacf8-8af1-4343-a82c-a91a3c7bbdbf None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:54.855132 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30e972-f4c7-424a-96bc-f71b412995ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:54.856047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30e972-f4c7-424a-96bc-f71b412995ed None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:54.856319 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30e972-f4c7-424a-96bc-f71b412995ed None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:54.856595 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30e972-f4c7-424a-96bc-f71b412995ed None admin] PATH_INFO: `/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/64296c932944405abbf8e767570c84f1/roles/3387bd4782b047ae8d880e386b0351f0` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:54.877421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30e972-f4c7-424a-96bc-f71b412995ed None admin] RBAC: Authorizing `identity:create_grant(project_id=d638a29a793d41e7bfa686c0fcc29149, user_id=64296c932944405abbf8e767570c84f1, role_id=3387bd4782b047ae8d880e386b0351f0)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:54.879623 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30e972-f4c7-424a-96bc-f71b412995ed None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:54.901895 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 296/599] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:50:54 2022] PUT /identity/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/64296c932944405abbf8e767570c84f1/roles/3387bd4782b047ae8d880e386b0351f0 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:54.903296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30e972-f4c7-424a-96bc-f71b412995ed None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:54.979809 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cacf8-8af1-4343-a82c-a91a3c7bbdbf None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:54.990745 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:55.045826 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30e972-f4c7-424a-96bc-f71b412995ed None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:55.046075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1df81-9565-496d-b757-d69d04e7aa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:55.046830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1df81-9565-496d-b757-d69d04e7aa49 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:55.047310 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1df81-9565-496d-b757-d69d04e7aa49 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:55.047589 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1df81-9565-496d-b757-d69d04e7aa49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:55.063628 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b1df81-9565-496d-b757-d69d04e7aa49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:55.065823 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b1df81-9565-496d-b757-d69d04e7aa49 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:55.085159 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49d82a-b491-4a09-95f8-5de84b62e366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:55.085975 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49d82a-b491-4a09-95f8-5de84b62e366 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:55.086285 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49d82a-b491-4a09-95f8-5de84b62e366 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:55.086578 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49d82a-b491-4a09-95f8-5de84b62e366 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:55.089401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-c4b1df81-9565-496d-b757-d69d04e7aa49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:55.089401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:55.089401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:55.089401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:55.089401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:55.089401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:55.089401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:55.089401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:55.089401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:55.089401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:55.089401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:55.089401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:55.089401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:55.089401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:55.089401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:55.090434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:55.091458 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:55.091458 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:55.091458 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:55.091458 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:55.098304 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 304/600] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:55.098304 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995092a-247a-4665-886c-f4d5a8fd7528 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:55.098304 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995092a-247a-4665-886c-f4d5a8fd7528 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:55.098304 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995092a-247a-4665-886c-f4d5a8fd7528 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:55.098965 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 305/601] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:55.099206 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a49d82a-b491-4a09-95f8-5de84b62e366 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:55.099674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995092a-247a-4665-886c-f4d5a8fd7528 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:55.100925 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a49d82a-b491-4a09-95f8-5de84b62e366 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:55.122756 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:55.124650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-5a49d82a-b491-4a09-95f8-5de84b62e366 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:55.124650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:55.124650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:55.124650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:55.124650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:55.124650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:55.124650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:55.124650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:55.124650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:55.124650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:55.124650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:55.124650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:55.124650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:55.124650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:55.124650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:55.125638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:55.127710 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:55.127710 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:55.127710 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:55.127710 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:55.127710 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:55.127710 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:55.127710 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:55.127710 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:55.127710 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 297/602] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:55.129176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec146e31-4fe0-4dd9-abb0-f394d4a5645b None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:55.129176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec146e31-4fe0-4dd9-abb0-f394d4a5645b None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:55.129176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec146e31-4fe0-4dd9-abb0-f394d4a5645b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:55.151959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-ec146e31-4fe0-4dd9-abb0-f394d4a5645b None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:55.153291 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-ec146e31-4fe0-4dd9-abb0-f394d4a5645b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:55.237340 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 298/603] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:55.238337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec146e31-4fe0-4dd9-abb0-f394d4a5645b None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:55.240039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995092a-247a-4665-886c-f4d5a8fd7528 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:55.291203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d564607-2a23-402c-8069-80dabbf6854e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:55.292332 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d564607-2a23-402c-8069-80dabbf6854e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:55.292875 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d564607-2a23-402c-8069-80dabbf6854e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:55.293361 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d564607-2a23-402c-8069-80dabbf6854e None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:55.294378 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d564607-2a23-402c-8069-80dabbf6854e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:55.294916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d564607-2a23-402c-8069-80dabbf6854e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:55.296674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d564607-2a23-402c-8069-80dabbf6854e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:55.302025 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:55.311524 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 306/604] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:55.312728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d564607-2a23-402c-8069-80dabbf6854e None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:55.379548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec146e31-4fe0-4dd9-abb0-f394d4a5645b None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:55.404019 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:55.416562 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae96dbd-0ebf-4011-a6d9-86c5e1912b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:55.417566 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae96dbd-0ebf-4011-a6d9-86c5e1912b9d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:55.418085 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae96dbd-0ebf-4011-a6d9-86c5e1912b9d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:55.418622 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae96dbd-0ebf-4011-a6d9-86c5e1912b9d None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:55.419689 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae96dbd-0ebf-4011-a6d9-86c5e1912b9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:55.420239 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae96dbd-0ebf-4011-a6d9-86c5e1912b9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:55.421985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae96dbd-0ebf-4011-a6d9-86c5e1912b9d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:55.431606 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d564607-2a23-402c-8069-80dabbf6854e None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:55.435258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 299/605] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:50:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:55.436481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae96dbd-0ebf-4011-a6d9-86c5e1912b9d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:55.467691 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf9566-74f3-4c52-86f4-c26fcfca0ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:55.468660 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf9566-74f3-4c52-86f4-c26fcfca0ed1 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:55.468940 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf9566-74f3-4c52-86f4-c26fcfca0ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:55.469265 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf9566-74f3-4c52-86f4-c26fcfca0ed1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:55.480516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf9566-74f3-4c52-86f4-c26fcfca0ed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-5edf9566-74f3-4c52-86f4-c26fcfca0ed1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:55.488303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:55.489455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:55.491493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:55.491493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:55.491493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:55.491493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:55.491493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:55.491493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:55.491493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:55.491493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:55.491493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:55.491493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:55.491493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:55.491493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:55.494568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 307/606] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:55.495460 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf9566-74f3-4c52-86f4-c26fcfca0ed1 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:55.557623 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:55.564937 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae96dbd-0ebf-4011-a6d9-86c5e1912b9d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:55.604280 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf9566-74f3-4c52-86f4-c26fcfca0ed1 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:55.612526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5fef4-018e-4ec4-a0b7-bc61ffb1b040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:55.613637 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5fef4-018e-4ec4-a0b7-bc61ffb1b040 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:55.614106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5fef4-018e-4ec4-a0b7-bc61ffb1b040 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:55.614637 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5fef4-018e-4ec4-a0b7-bc61ffb1b040 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:55.622087 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5fef4-018e-4ec4-a0b7-bc61ffb1b040 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:55.623841 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5fef4-018e-4ec4-a0b7-bc61ffb1b040 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:55.639306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-e9f5fef4-018e-4ec4-a0b7-bc61ffb1b040 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:55.639306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:55.639306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:55.639306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:55.639306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:55.639306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:55.639306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:55.639306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:55.639306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:55.639306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:55.639306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:55.639306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:55.639306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:55.639306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:55.639306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:55.640379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:55.641383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:55.641383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:55.641383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:55.641383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:55.641383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:55.641383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:55.641383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:55.641383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:55.643795 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 300/607] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:55.644857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5fef4-018e-4ec4-a0b7-bc61ffb1b040 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:55.648192 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89664da4-5511-4001-8a6f-f6797b2c9fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:55.649798 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89664da4-5511-4001-8a6f-f6797b2c9fc5 None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:55.649798 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89664da4-5511-4001-8a6f-f6797b2c9fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:55.649798 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89664da4-5511-4001-8a6f-f6797b2c9fc5 None admin] PATH_INFO: `/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/6b89f5a099d24437a25d5d7f21e10e2e/roles/3387bd4782b047ae8d880e386b0351f0` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:55.652848 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:55.668773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89664da4-5511-4001-8a6f-f6797b2c9fc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d638a29a793d41e7bfa686c0fcc29149, user_id=6b89f5a099d24437a25d5d7f21e10e2e, role_id=3387bd4782b047ae8d880e386b0351f0)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:55.670427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89664da4-5511-4001-8a6f-f6797b2c9fc5 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:55.688510 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 308/608] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:50:55 2022] PUT /identity/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/6b89f5a099d24437a25d5d7f21e10e2e/roles/3387bd4782b047ae8d880e386b0351f0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:55.689551 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89664da4-5511-4001-8a6f-f6797b2c9fc5 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:55.768784 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5fef4-018e-4ec4-a0b7-bc61ffb1b040 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:55.808818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a2b2f-72f8-4180-a7cc-dbc99bb80db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:55.809851 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a2b2f-72f8-4180-a7cc-dbc99bb80db9 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:55.810329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a2b2f-72f8-4180-a7cc-dbc99bb80db9 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:55.810755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a2b2f-72f8-4180-a7cc-dbc99bb80db9 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:55.811931 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a2b2f-72f8-4180-a7cc-dbc99bb80db9 None admin] RBAC: Adding query filter params (name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:55.812550 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a2b2f-72f8-4180-a7cc-dbc99bb80db9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:55.814219 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a2b2f-72f8-4180-a7cc-dbc99bb80db9 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:55.824527 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 301/609] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:50:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:55.834793 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a2b2f-72f8-4180-a7cc-dbc99bb80db9 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:55.841321 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89664da4-5511-4001-8a6f-f6797b2c9fc5 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:55.882762 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:55.890675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028a6e1-1c44-4079-b319-a500fc2f59c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:55.891809 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028a6e1-1c44-4079-b319-a500fc2f59c9 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:55.892317 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028a6e1-1c44-4079-b319-a500fc2f59c9 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:55.892819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028a6e1-1c44-4079-b319-a500fc2f59c9 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:55.894820 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028a6e1-1c44-4079-b319-a500fc2f59c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:55.895559 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028a6e1-1c44-4079-b319-a500fc2f59c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:55.898918 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028a6e1-1c44-4079-b319-a500fc2f59c9 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:55.916624 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 309/610] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:50:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:55.923470 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028a6e1-1c44-4079-b319-a500fc2f59c9 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:55.981259 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:56.050337 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028a6e1-1c44-4079-b319-a500fc2f59c9 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:56.071287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a2b2f-72f8-4180-a7cc-dbc99bb80db9 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:56.094683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2857822-347f-4290-9a98-6594216e88a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:56.095677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2857822-347f-4290-9a98-6594216e88a0 None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:56.095965 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2857822-347f-4290-9a98-6594216e88a0 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:56.096243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2857822-347f-4290-9a98-6594216e88a0 None admin] PATH_INFO: `/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/24d916d5c44c4078958734bd59a220b3/roles/3387bd4782b047ae8d880e386b0351f0` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:56.113709 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0a9bd-6860-458a-b73b-a44599699f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:56.114919 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0a9bd-6860-458a-b73b-a44599699f10 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:56.115449 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0a9bd-6860-458a-b73b-a44599699f10 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:56.115815 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0a9bd-6860-458a-b73b-a44599699f10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:56.116606 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2857822-347f-4290-9a98-6594216e88a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d638a29a793d41e7bfa686c0fcc29149, user_id=24d916d5c44c4078958734bd59a220b3, role_id=3387bd4782b047ae8d880e386b0351f0)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:56.119030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2857822-347f-4290-9a98-6594216e88a0 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:56.127530 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0a9bd-6860-458a-b73b-a44599699f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:56.129440 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0a9bd-6860-458a-b73b-a44599699f10 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:56.136090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 310/611] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:50:55 2022] PUT /identity/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/24d916d5c44c4078958734bd59a220b3/roles/3387bd4782b047ae8d880e386b0351f0 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:56.138202 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14393fee-3669-4f9b-b9a7-9625a82019a3 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:56.138686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14393fee-3669-4f9b-b9a7-9625a82019a3 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:56.139143 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14393fee-3669-4f9b-b9a7-9625a82019a3 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:56.141689 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 311/612] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:56.151327 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1e9cf-3a6f-4563-ab92-b8e46fce2a9b None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:56.151895 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1e9cf-3a6f-4563-ab92-b8e46fce2a9b None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:56.152334 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1e9cf-3a6f-4563-ab92-b8e46fce2a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:56.159249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-51f0a9bd-6860-458a-b73b-a44599699f10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:56.159249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:56.159249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:56.159249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:56.159249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:56.159249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:56.159249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:56.159249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:56.159249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:56.159249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:56.159249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:56.159249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:56.159249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:56.159249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:56.159249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:56.160494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:56.161664 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:56.161664 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:56.161664 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:56.161664 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:56.166311 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 302/613] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:56.170224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0a9bd-6860-458a-b73b-a44599699f10 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:56.187593 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-46b1e9cf-3a6f-4563-ab92-b8e46fce2a9b None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:56.189792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-46b1e9cf-3a6f-4563-ab92-b8e46fce2a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:56.291753 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 312/614] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:50:56.328541 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1e9cf-3a6f-4563-ab92-b8e46fce2a9b None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:56.354144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:56.380590 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:56.420059 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0a9bd-6860-458a-b73b-a44599699f10 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:56.467070 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1e9cf-3a6f-4563-ab92-b8e46fce2a9b None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:56.468598 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1300e-46ce-4699-9b0c-9bd3080798e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:56.469398 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1300e-46ce-4699-9b0c-9bd3080798e5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:56.469684 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1300e-46ce-4699-9b0c-9bd3080798e5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:56.469970 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1300e-46ce-4699-9b0c-9bd3080798e5 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:56.470732 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1300e-46ce-4699-9b0c-9bd3080798e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:56.471041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1300e-46ce-4699-9b0c-9bd3080798e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:56.472807 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1300e-46ce-4699-9b0c-9bd3080798e5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:56.487182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 303/615] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:56.492424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1300e-46ce-4699-9b0c-9bd3080798e5 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:56.511910 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a774ce6-b1bb-4ed3-a440-87f2476a8b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:56.512738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774ce6-b1bb-4ed3-a440-87f2476a8b86 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:56.513093 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774ce6-b1bb-4ed3-a440-87f2476a8b86 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:56.513695 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774ce6-b1bb-4ed3-a440-87f2476a8b86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:56.525476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a774ce6-b1bb-4ed3-a440-87f2476a8b86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-4a774ce6-b1bb-4ed3-a440-87f2476a8b86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:56.536413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:56.537686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:56.538962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:56.538962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:56.538962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:56.538962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:56.538962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:56.538962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:56.538962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:56.538962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:56.538962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:56.538962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:56.538962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:56.538962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:56.538962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 313/616] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:56.542985 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a774ce6-b1bb-4ed3-a440-87f2476a8b86 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:56.620221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1300e-46ce-4699-9b0c-9bd3080798e5 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:56.638199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:56.657620 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a774ce6-b1bb-4ed3-a440-87f2476a8b86 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:56.662669 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:56.666881 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fd19a-d4e8-4a0a-8edc-75a9691b023d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:56.668409 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fd19a-d4e8-4a0a-8edc-75a9691b023d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:56.668871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fd19a-d4e8-4a0a-8edc-75a9691b023d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:56.669317 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fd19a-d4e8-4a0a-8edc-75a9691b023d None admin] PATH_INFO: `/v3/users/swift` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:56.679771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fd19a-d4e8-4a0a-8edc-75a9691b023d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:56.681982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fd19a-d4e8-4a0a-8edc-75a9691b023d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-8a6fd19a-d4e8-4a0a-8edc-75a9691b023d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:56.701148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:56.702686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:56.703874 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:56.703874 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:56.703874 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:56.703874 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:56.703874 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:56.703874 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:56.703874 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:56.703874 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:56.703874 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:50:56.703874 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:56.706309 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 304/617] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:50:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:56.710091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94010c-d8bf-40fa-897b-38845a31f59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:56.711467 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fd19a-d4e8-4a0a-8edc-75a9691b023d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:56.716252 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94010c-d8bf-40fa-897b-38845a31f59d None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:56.716252 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94010c-d8bf-40fa-897b-38845a31f59d None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:56.716662 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94010c-d8bf-40fa-897b-38845a31f59d None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:56.717565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94010c-d8bf-40fa-897b-38845a31f59d None admin] RBAC: Adding query filter params (name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:56.718402 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94010c-d8bf-40fa-897b-38845a31f59d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:56.720008 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94010c-d8bf-40fa-897b-38845a31f59d None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:56.730200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 314/618] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:50:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:56.739594 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94010c-d8bf-40fa-897b-38845a31f59d None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:56.860526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fd19a-d4e8-4a0a-8edc-75a9691b023d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:56.934430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1f447-6ea4-43fa-98c7-342acb9f709a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:56.935038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1f447-6ea4-43fa-98c7-342acb9f709a None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:56.935325 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1f447-6ea4-43fa-98c7-342acb9f709a None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:56.935603 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1f447-6ea4-43fa-98c7-342acb9f709a None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:56.936466 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1f447-6ea4-43fa-98c7-342acb9f709a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:56.936852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1f447-6ea4-43fa-98c7-342acb9f709a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:56.938512 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1f447-6ea4-43fa-98c7-342acb9f709a None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:56.977136 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 305/619] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 19:50:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:56.979933 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1f447-6ea4-43fa-98c7-342acb9f709a None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:56.982146 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94010c-d8bf-40fa-897b-38845a31f59d None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:57.052829 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac52f4a-5db0-4373-a5dd-3834fea07736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:57.054063 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac52f4a-5db0-4373-a5dd-3834fea07736 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:57.054612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac52f4a-5db0-4373-a5dd-3834fea07736 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:57.055106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac52f4a-5db0-4373-a5dd-3834fea07736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:57.069191 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac52f4a-5db0-4373-a5dd-3834fea07736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:57.071357 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac52f4a-5db0-4373-a5dd-3834fea07736 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:57.078492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:57.092418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-5ac52f4a-5db0-4373-a5dd-3834fea07736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:57.092418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:57.092418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:57.092418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:57.092418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:57.092418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:57.092418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:57.092418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:57.092418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:57.092418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:57.092418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:57.092418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:57.092418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:57.092418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:57.092418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:57.093505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:57.094670 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:57.094670 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:57.094670 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:57.094670 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:57.098488 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 315/620] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:57.098623 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:57.107711 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac52f4a-5db0-4373-a5dd-3834fea07736 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:57.225767 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1f447-6ea4-43fa-98c7-342acb9f709a None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:57.290394 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac52f4a-5db0-4373-a5dd-3834fea07736 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:57.300997 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b7f87-6c1b-425c-97d0-476c7e511cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:57.302152 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b7f87-6c1b-425c-97d0-476c7e511cb1 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:57.302716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b7f87-6c1b-425c-97d0-476c7e511cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:57.303231 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b7f87-6c1b-425c-97d0-476c7e511cb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:57.320486 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b7f87-6c1b-425c-97d0-476c7e511cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:57.325053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b7f87-6c1b-425c-97d0-476c7e511cb1 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:57.357702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-bb8b7f87-6c1b-425c-97d0-476c7e511cb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:57.357702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:57.357702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:57.357702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:57.357702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:57.357702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:57.357702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:57.357702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:57.357702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:57.357702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:57.357702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:57.357702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:57.357702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:57.357702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:57.357702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:57.358877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:57.360117 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:57.360117 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:57.360117 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:57.360117 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:57.370874 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 306/621] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:57.373936 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447329ab-a4e3-4068-9eed-971bbadc2ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:57.376288 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b7f87-6c1b-425c-97d0-476c7e511cb1 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:57.379485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447329ab-a4e3-4068-9eed-971bbadc2ffa None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:57.380189 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447329ab-a4e3-4068-9eed-971bbadc2ffa None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:57.380687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447329ab-a4e3-4068-9eed-971bbadc2ffa None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:57.382618 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447329ab-a4e3-4068-9eed-971bbadc2ffa None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:57.383958 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447329ab-a4e3-4068-9eed-971bbadc2ffa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:57.388686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447329ab-a4e3-4068-9eed-971bbadc2ffa None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:57.411183 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 316/622] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:57.417182 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447329ab-a4e3-4068-9eed-971bbadc2ffa None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:57.509245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:57.528002 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b7f87-6c1b-425c-97d0-476c7e511cb1 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:57.540662 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:57.583767 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b8e83-2df3-4cf2-bdea-3e55170f2dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:57.584836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b8e83-2df3-4cf2-bdea-3e55170f2dcc None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:57.586879 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b8e83-2df3-4cf2-bdea-3e55170f2dcc None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:57.587432 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b8e83-2df3-4cf2-bdea-3e55170f2dcc None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:57.592141 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b8e83-2df3-4cf2-bdea-3e55170f2dcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:57.592804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b8e83-2df3-4cf2-bdea-3e55170f2dcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:57.594645 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b8e83-2df3-4cf2-bdea-3e55170f2dcc None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:57.610243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447329ab-a4e3-4068-9eed-971bbadc2ffa None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:57.611594 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 307/623] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:57.617513 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c7f5a-6d03-4608-8a76-61db5c6ff5a0 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:57.618444 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c7f5a-6d03-4608-8a76-61db5c6ff5a0 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:57.619179 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c7f5a-6d03-4608-8a76-61db5c6ff5a0 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:57.621001 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 308/624] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:57.621953 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c7f5a-6d03-4608-8a76-61db5c6ff5a0 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:57.652826 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d729b74-6c40-4493-aef2-8ab4ec3815c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:57.653638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d729b74-6c40-4493-aef2-8ab4ec3815c4 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:57.653921 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d729b74-6c40-4493-aef2-8ab4ec3815c4 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:57.654198 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d729b74-6c40-4493-aef2-8ab4ec3815c4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:57.662035 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d729b74-6c40-4493-aef2-8ab4ec3815c4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:57.663617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d729b74-6c40-4493-aef2-8ab4ec3815c4 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-0d729b74-6c40-4493-aef2-8ab4ec3815c4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:57.678687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:57.679792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:57.680870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:57.680870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:57.680870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:57.680870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:57.680870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:57.680870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:57.680870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:57.680870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:57.680870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:50:57.680870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:57.684512 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 317/625] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:50:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:57.685850 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bfa10-85dd-4154-baae-23e58693b132 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:57.686138 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bfa10-85dd-4154-baae-23e58693b132 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:57.686561 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bfa10-85dd-4154-baae-23e58693b132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:57.710974 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-9a1bfa10-85dd-4154-baae-23e58693b132 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:57.710974 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-9a1bfa10-85dd-4154-baae-23e58693b132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:57.754770 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c7f5a-6d03-4608-8a76-61db5c6ff5a0 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:57.792696 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8c940-c9cf-4feb-83c5-17f6c63c3ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:57.793852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8c940-c9cf-4feb-83c5-17f6c63c3ca3 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:57.794358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8c940-c9cf-4feb-83c5-17f6c63c3ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:57.794893 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8c940-c9cf-4feb-83c5-17f6c63c3ca3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:57.798362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 318/626] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:57.799635 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1bfa10-85dd-4154-baae-23e58693b132 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:57.814018 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d8c940-c9cf-4feb-83c5-17f6c63c3ca3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:57.815798 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d8c940-c9cf-4feb-83c5-17f6c63c3ca3 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:57.836412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-41d8c940-c9cf-4feb-83c5-17f6c63c3ca3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:57.836412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:57.836412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:57.836412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:57.836412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:57.836412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:57.836412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:57.836412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:57.836412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:57.836412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:57.836412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:57.836412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:57.836412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:57.836412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:57.836412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:57.837474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:57.838528 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:57.838528 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:57.838528 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:57.838528 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:57.838528 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:57.838528 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:57.838528 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:57.838528 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:57.841177 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 309/627] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:57.842192 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8c940-c9cf-4feb-83c5-17f6c63c3ca3 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:57.868634 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:57.871040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:57.937995 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1bfa10-85dd-4154-baae-23e58693b132 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:57.958400 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8c940-c9cf-4feb-83c5-17f6c63c3ca3 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:57.986166 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61f596-4d6a-4925-8a0e-1b3f8c1e7707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:57.986876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61f596-4d6a-4925-8a0e-1b3f8c1e7707 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:57.987177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61f596-4d6a-4925-8a0e-1b3f8c1e7707 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:57.987453 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61f596-4d6a-4925-8a0e-1b3f8c1e7707 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:57.988572 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61f596-4d6a-4925-8a0e-1b3f8c1e7707 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:57.988882 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61f596-4d6a-4925-8a0e-1b3f8c1e7707 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:57.991109 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61f596-4d6a-4925-8a0e-1b3f8c1e7707 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:57.998440 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efe357-a000-4f55-9e55-0739192d358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:57.999889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efe357-a000-4f55-9e55-0739192d358b None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:58.000449 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efe357-a000-4f55-9e55-0739192d358b None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:58.001013 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efe357-a000-4f55-9e55-0739192d358b None admin] PATH_INFO: `/v3/roles/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:58.011308 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 319/628] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:50:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:58.012440 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61f596-4d6a-4925-8a0e-1b3f8c1e7707 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:58.025957 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efe357-a000-4f55-9e55-0739192d358b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-40efe357-a000-4f55-9e55-0739192d358b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:58.036760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:58.037805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:58.038818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:58.038818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:58.038818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:58.038818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:58.038818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:58.038818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:58.038818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:58.038818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:58.038818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:58.038818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:58.038818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:58.038818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:58.042361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 310/629] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:58.043488 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efe357-a000-4f55-9e55-0739192d358b None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:58.140977 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:58.143578 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61f596-4d6a-4925-8a0e-1b3f8c1e7707 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:58.154656 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:58.168593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efe357-a000-4f55-9e55-0739192d358b None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:58.196722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d38876-683d-42d7-86d2-2cc57c090d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:58.197400 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d38876-683d-42d7-86d2-2cc57c090d5a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:58.197685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d38876-683d-42d7-86d2-2cc57c090d5a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:58.197963 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d38876-683d-42d7-86d2-2cc57c090d5a None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:58.199063 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d38876-683d-42d7-86d2-2cc57c090d5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:58.199375 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d38876-683d-42d7-86d2-2cc57c090d5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:58.201488 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d38876-683d-42d7-86d2-2cc57c090d5a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:58.210146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660cd9d-f1f8-489c-9e9f-fc4fc315e1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:58.211169 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660cd9d-f1f8-489c-9e9f-fc4fc315e1c4 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:58.211663 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660cd9d-f1f8-489c-9e9f-fc4fc315e1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:58.212147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660cd9d-f1f8-489c-9e9f-fc4fc315e1c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:58.217248 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 320/630] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:50:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:58.218157 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d38876-683d-42d7-86d2-2cc57c090d5a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:58.299625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1660cd9d-f1f8-489c-9e9f-fc4fc315e1c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:58.300384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1660cd9d-f1f8-489c-9e9f-fc4fc315e1c4 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:58.344852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d38876-683d-42d7-86d2-2cc57c090d5a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:58.381536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3c816-43ee-4bba-b336-ed1a9da19207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:58.382785 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3c816-43ee-4bba-b336-ed1a9da19207 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:58.383314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3c816-43ee-4bba-b336-ed1a9da19207 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:58.383782 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3c816-43ee-4bba-b336-ed1a9da19207 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:58.385008 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3c816-43ee-4bba-b336-ed1a9da19207 None admin] RBAC: Adding query filter params (name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:58.385628 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3c816-43ee-4bba-b336-ed1a9da19207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:58.387342 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3c816-43ee-4bba-b336-ed1a9da19207 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:58.396374 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 321/631] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:50:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:58.397488 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3c816-43ee-4bba-b336-ed1a9da19207 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:58.401701 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:58.427023 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 311/632] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:50:58.428342 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c46fc2-0b7f-470f-af91-3a0387e66c14 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:58.428634 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c46fc2-0b7f-470f-af91-3a0387e66c14 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:58.428939 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c46fc2-0b7f-470f-af91-3a0387e66c14 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:58.430391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 312/633] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:58.431083 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c46fc2-0b7f-470f-af91-3a0387e66c14 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:58.434732 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:58.510129 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3c816-43ee-4bba-b336-ed1a9da19207 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:58.544393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c46fc2-0b7f-470f-af91-3a0387e66c14 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:58.549166 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7837f-a51e-4c66-8c34-77ee0bcdfc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:58.550138 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7837f-a51e-4c66-8c34-77ee0bcdfc47 None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:58.550390 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7837f-a51e-4c66-8c34-77ee0bcdfc47 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:58.550663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7837f-a51e-4c66-8c34-77ee0bcdfc47 None admin] PATH_INFO: `/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/acebbcf1f90d487aa21a8eb99102ce09/roles/3387bd4782b047ae8d880e386b0351f0` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:58.566148 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7837f-a51e-4c66-8c34-77ee0bcdfc47 None admin] RBAC: Authorizing `identity:create_grant(project_id=d638a29a793d41e7bfa686c0fcc29149, user_id=acebbcf1f90d487aa21a8eb99102ce09, role_id=3387bd4782b047ae8d880e386b0351f0)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:58.567632 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7837f-a51e-4c66-8c34-77ee0bcdfc47 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:58.583851 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2e1a6-ae2a-40f9-bae8-9ecca2c30042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:58.584468 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 322/634] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:50:58 2022] PUT /identity/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/acebbcf1f90d487aa21a8eb99102ce09/roles/3387bd4782b047ae8d880e386b0351f0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:58.584697 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2e1a6-ae2a-40f9-bae8-9ecca2c30042 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:58.584981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2e1a6-ae2a-40f9-bae8-9ecca2c30042 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:58.585325 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2e1a6-ae2a-40f9-bae8-9ecca2c30042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:58.586381 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7837f-a51e-4c66-8c34-77ee0bcdfc47 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:58.597195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2e1a6-ae2a-40f9-bae8-9ecca2c30042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:58.598731 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2e1a6-ae2a-40f9-bae8-9ecca2c30042 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:58.618619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-45a2e1a6-ae2a-40f9-bae8-9ecca2c30042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:58.618619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:58.618619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:58.618619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:58.618619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:58.618619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:58.618619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:58.618619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:58.618619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:58.618619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:58.618619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:58.618619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:58.618619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:58.618619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:58.618619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:58.619746 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:58.620784 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:58.620784 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:58.620784 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:58.620784 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:58.620784 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 313/635] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:58.621638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6b51d-515d-4456-acb8-9657661a9562 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:58.621927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6b51d-515d-4456-acb8-9657661a9562 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:58.622201 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6b51d-515d-4456-acb8-9657661a9562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:58.640677 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-d1e6b51d-515d-4456-acb8-9657661a9562 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:58.641906 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-d1e6b51d-515d-4456-acb8-9657661a9562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:58.684713 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:58.705593 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:58.735807 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7837f-a51e-4c66-8c34-77ee0bcdfc47 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:58.748251 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 314/636] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:58.749184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6b51d-515d-4456-acb8-9657661a9562 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:58.786314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947b7c6-9f8d-4cfb-aa2e-9d7aa66fcfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:58.787136 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947b7c6-9f8d-4cfb-aa2e-9d7aa66fcfa3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:58.788954 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947b7c6-9f8d-4cfb-aa2e-9d7aa66fcfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:58.788954 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947b7c6-9f8d-4cfb-aa2e-9d7aa66fcfa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:58.805166 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947b7c6-9f8d-4cfb-aa2e-9d7aa66fcfa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:58.806639 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947b7c6-9f8d-4cfb-aa2e-9d7aa66fcfa3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:58.825427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-f947b7c6-9f8d-4cfb-aa2e-9d7aa66fcfa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:58.825427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:58.825427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:58.825427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:58.825427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:58.825427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:58.825427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:58.825427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:58.825427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:58.825427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:58.825427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:58.825427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:58.825427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:58.825427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:58.825427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:58.826775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:58.828011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:58.828011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:58.828011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:58.828011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:58.828604 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 323/637] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:58.829882 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddf77c-09f2-4383-b3d7-39b19a5b3c45 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:58.830165 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddf77c-09f2-4383-b3d7-39b19a5b3c45 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:58.830466 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddf77c-09f2-4383-b3d7-39b19a5b3c45 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:58.835948 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 324/638] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:58.836731 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddf77c-09f2-4383-b3d7-39b19a5b3c45 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:58.865014 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6b51d-515d-4456-acb8-9657661a9562 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:58.900349 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebb509-96ec-4b35-b2db-748a54f42a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:58.901032 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebb509-96ec-4b35-b2db-748a54f42a3c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:58.901268 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebb509-96ec-4b35-b2db-748a54f42a3c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:58.901542 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebb509-96ec-4b35-b2db-748a54f42a3c None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:58.902292 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebb509-96ec-4b35-b2db-748a54f42a3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:58.902627 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebb509-96ec-4b35-b2db-748a54f42a3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:58.904091 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebb509-96ec-4b35-b2db-748a54f42a3c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:58.915844 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 315/639] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:58.916653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebb509-96ec-4b35-b2db-748a54f42a3c None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:58.945971 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:58.946592 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddf77c-09f2-4383-b3d7-39b19a5b3c45 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:58.984675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:58.991722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b17b6-5d26-4bea-a29f-bac62df6abc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:58.992538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b17b6-5d26-4bea-a29f-bac62df6abc0 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:58.992827 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b17b6-5d26-4bea-a29f-bac62df6abc0 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:58.993103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b17b6-5d26-4bea-a29f-bac62df6abc0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:59.004681 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b17b6-5d26-4bea-a29f-bac62df6abc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-2c1b17b6-5d26-4bea-a29f-bac62df6abc0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:59.012955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:59.014081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:59.015338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:59.015338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:59.015338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:59.015338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:59.015338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:59.015338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:59.015338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:59.015338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:59.015338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:59.015338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:59.015338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:59.015338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:59.015338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 325/640] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:59.016120 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76b37a-5c77-4e15-8f9e-710adcba906e None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:59.016120 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76b37a-5c77-4e15-8f9e-710adcba906e None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:59.016256 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76b37a-5c77-4e15-8f9e-710adcba906e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:59.032664 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebb509-96ec-4b35-b2db-748a54f42a3c None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:59.033288 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-9c76b37a-5c77-4e15-8f9e-710adcba906e None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:59.034527 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-9c76b37a-5c77-4e15-8f9e-710adcba906e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:59.070588 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5987807-bf1e-4a40-8148-6a02dacd75d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:59.071187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5987807-bf1e-4a40-8148-6a02dacd75d9 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:59.071473 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5987807-bf1e-4a40-8148-6a02dacd75d9 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:59.071751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5987807-bf1e-4a40-8148-6a02dacd75d9 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:59.072488 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5987807-bf1e-4a40-8148-6a02dacd75d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:59.072801 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5987807-bf1e-4a40-8148-6a02dacd75d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:59.074185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5987807-bf1e-4a40-8148-6a02dacd75d9 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:59.091339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 316/641] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:59.092252 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5987807-bf1e-4a40-8148-6a02dacd75d9 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:59.114773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 326/642] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:50:59.114773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76b37a-5c77-4e15-8f9e-710adcba906e None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:59.198646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:59.215653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5987807-bf1e-4a40-8148-6a02dacd75d9 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:59.229846 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76b37a-5c77-4e15-8f9e-710adcba906e None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:59.251561 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cab1f-0c12-428f-a23f-5d98af5aa232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:59.252373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cab1f-0c12-428f-a23f-5d98af5aa232 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:59.252650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cab1f-0c12-428f-a23f-5d98af5aa232 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:59.252921 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cab1f-0c12-428f-a23f-5d98af5aa232 None admin] PATH_INFO: `/v3/users/glance` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:59.253704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:59.261701 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cab1f-0c12-428f-a23f-5d98af5aa232 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:59.263199 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cab1f-0c12-428f-a23f-5d98af5aa232 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:59.269877 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120dc0e-911b-4c09-a5d6-1bc6917e185d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:59.270594 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120dc0e-911b-4c09-a5d6-1bc6917e185d None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:59.270882 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120dc0e-911b-4c09-a5d6-1bc6917e185d None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:59.271159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120dc0e-911b-4c09-a5d6-1bc6917e185d None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:59.271920 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120dc0e-911b-4c09-a5d6-1bc6917e185d None admin] RBAC: Adding query filter params (name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:59.272235 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120dc0e-911b-4c09-a5d6-1bc6917e185d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:59.273614 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120dc0e-911b-4c09-a5d6-1bc6917e185d None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-c02cab1f-0c12-428f-a23f-5d98af5aa232 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:59.279003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:59.280287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:59.282006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:59.282006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:59.282006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:59.282006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:59.282006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:59.282006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:59.282006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:59.282006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:59.282006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 19:50:59.282006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:59.284978 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 317/643] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:50:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:59.289989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 327/644] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:50:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:59.290139 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cab1f-0c12-428f-a23f-5d98af5aa232 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:59.290229 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120dc0e-911b-4c09-a5d6-1bc6917e185d None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:59.393790 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cab1f-0c12-428f-a23f-5d98af5aa232 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:59.395989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120dc0e-911b-4c09-a5d6-1bc6917e185d None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:59.432500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff616e6f-018b-4a95-a03b-acb2ac2e77fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:59.433470 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff616e6f-018b-4a95-a03b-acb2ac2e77fe None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:59.433857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff616e6f-018b-4a95-a03b-acb2ac2e77fe None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:59.434238 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff616e6f-018b-4a95-a03b-acb2ac2e77fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:59.435360 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80d08-0cd3-40fe-a67d-cbcb62c8cf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:59.436233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80d08-0cd3-40fe-a67d-cbcb62c8cf6a None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:59.436680 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80d08-0cd3-40fe-a67d-cbcb62c8cf6a None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:59.437083 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80d08-0cd3-40fe-a67d-cbcb62c8cf6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:59.445230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f80d08-0cd3-40fe-a67d-cbcb62c8cf6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:59.446048 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff616e6f-018b-4a95-a03b-acb2ac2e77fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:59.446723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f80d08-0cd3-40fe-a67d-cbcb62c8cf6a None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-ff616e6f-018b-4a95-a03b-acb2ac2e77fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:59.454414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:59.455724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:59.456894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:59.456894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:59.456894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:59.456894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:59.456894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:59.456894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:59.456894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:59.456894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:59.456894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:59.456894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:59.456894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:59.456894 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:50:59.463521 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 328/645] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:50:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:59.463521 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff616e6f-018b-4a95-a03b-acb2ac2e77fe None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:59.463774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-00f80d08-0cd3-40fe-a67d-cbcb62c8cf6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:59.463774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:59.463774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:59.463774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:59.463774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:59.463774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:59.463774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:59.463774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:59.463774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:59.463774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:59.463774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:59.463774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:59.463774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:59.463774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:59.463774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:59.464886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:59.466097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:59.466097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:59.466097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:59.466097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:59.466097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:59.466097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:59.466097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:59.466097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:59.469010 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 318/646] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:50:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:59.469754 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80d08-0cd3-40fe-a67d-cbcb62c8cf6a None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:59.474672 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:59.508666 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:59.567017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff616e6f-018b-4a95-a03b-acb2ac2e77fe None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:59.586325 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80d08-0cd3-40fe-a67d-cbcb62c8cf6a None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:59.601963 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ed425-de9c-469e-b79d-dcee143551bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:59.602685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ed425-de9c-469e-b79d-dcee143551bb None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:59.602968 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ed425-de9c-469e-b79d-dcee143551bb None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:59.603244 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ed425-de9c-469e-b79d-dcee143551bb None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:59.604099 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ed425-de9c-469e-b79d-dcee143551bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:59.604407 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ed425-de9c-469e-b79d-dcee143551bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:59.605876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ed425-de9c-469e-b79d-dcee143551bb None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:59.623106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 329/647] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 19:50:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:59.623916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ed425-de9c-469e-b79d-dcee143551bb None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:59.629317 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac90899-c070-428d-9645-4785b3d0ae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:59.630289 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac90899-c070-428d-9645-4785b3d0ae60 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:59.630568 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac90899-c070-428d-9645-4785b3d0ae60 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:59.630844 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac90899-c070-428d-9645-4785b3d0ae60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:59.643597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac90899-c070-428d-9645-4785b3d0ae60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:59.645298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac90899-c070-428d-9645-4785b3d0ae60 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:59.664016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-fac90899-c070-428d-9645-4785b3d0ae60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:59.664016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:59.664016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:59.664016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:59.664016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:59.664016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:59.664016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:59.664016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:59.664016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:59.664016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:59.664016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:59.664016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:59.664016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:59.664016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:59.664016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:59.665163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:59.666155 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:59.666155 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:59.666155 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:59.666155 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:50:59.666155 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 319/648] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:59.666155 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac90899-c070-428d-9645-4785b3d0ae60 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:59.733043 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:59.738226 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:59.741830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ed425-de9c-469e-b79d-dcee143551bb None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:59.779192 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac90899-c070-428d-9645-4785b3d0ae60 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:59.783870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ff700-2157-4450-87c7-40fa8afdfaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:59.784656 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ff700-2157-4450-87c7-40fa8afdfaf5 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:59.785037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ff700-2157-4450-87c7-40fa8afdfaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:59.785408 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ff700-2157-4450-87c7-40fa8afdfaf5 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:59.786745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ff700-2157-4450-87c7-40fa8afdfaf5 None admin] RBAC: Adding query filter params (name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:59.787157 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ff700-2157-4450-87c7-40fa8afdfaf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:59.789004 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ff700-2157-4450-87c7-40fa8afdfaf5 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:59.804711 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 330/649] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:50:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:59.804711 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ff700-2157-4450-87c7-40fa8afdfaf5 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:59.814011 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05604fe7-2937-4e89-ae68-c13028d75646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:59.814837 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05604fe7-2937-4e89-ae68-c13028d75646 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:59.815214 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05604fe7-2937-4e89-ae68-c13028d75646 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:59.815600 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05604fe7-2937-4e89-ae68-c13028d75646 None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:59.816492 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05604fe7-2937-4e89-ae68-c13028d75646 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:59.816895 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05604fe7-2937-4e89-ae68-c13028d75646 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:59.818450 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05604fe7-2937-4e89-ae68-c13028d75646 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:59.832032 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 320/650] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:50:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:59.832928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05604fe7-2937-4e89-ae68-c13028d75646 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:59.916510 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ff700-2157-4450-87c7-40fa8afdfaf5 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:59.942609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05604fe7-2937-4e89-ae68-c13028d75646 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:59.953099 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698f2d8-2743-47a7-ba05-ae966b69f41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:59.953947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698f2d8-2743-47a7-ba05-ae966b69f41f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:59.954430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698f2d8-2743-47a7-ba05-ae966b69f41f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:59.954863 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698f2d8-2743-47a7-ba05-ae966b69f41f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:59.960408 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:59.982228 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904917a2-7508-4347-8d55-6e37ba6f8fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:59.982929 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904917a2-7508-4347-8d55-6e37ba6f8fb3 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:59.983214 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904917a2-7508-4347-8d55-6e37ba6f8fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:59.983656 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904917a2-7508-4347-8d55-6e37ba6f8fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:59.984460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904917a2-7508-4347-8d55-6e37ba6f8fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:59.984819 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904917a2-7508-4347-8d55-6e37ba6f8fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:59.986230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904917a2-7508-4347-8d55-6e37ba6f8fb3 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:59.987807 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:00.003017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 321/651] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:00.003017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904917a2-7508-4347-8d55-6e37ba6f8fb3 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:00.030919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698f2d8-2743-47a7-ba05-ae966b69f41f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:00.032643 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698f2d8-2743-47a7-ba05-ae966b69f41f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:00.108153 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904917a2-7508-4347-8d55-6e37ba6f8fb3 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:00.142782 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 331/652] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:50:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:51:00.143655 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698f2d8-2743-47a7-ba05-ae966b69f41f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:00.149061 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86ad0b-9431-4b88-9dff-9f927c56b255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:00.149878 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86ad0b-9431-4b88-9dff-9f927c56b255 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:00.150166 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86ad0b-9431-4b88-9dff-9f927c56b255 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:00.150483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86ad0b-9431-4b88-9dff-9f927c56b255 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:00.163068 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86ad0b-9431-4b88-9dff-9f927c56b255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:00.164548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86ad0b-9431-4b88-9dff-9f927c56b255 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:00.183197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-bc86ad0b-9431-4b88-9dff-9f927c56b255 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:00.183197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:00.183197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:00.183197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:00.183197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:00.183197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:00.183197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:00.183197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:00.183197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:00.183197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:00.183197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:00.183197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:00.183197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:00.183197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:00.183197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:00.184377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:00.185379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:00.185379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:00.185379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:00.185379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:00.185379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 322/653] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:00.185379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86ad0b-9431-4b88-9dff-9f927c56b255 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:00.214127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:00.216321 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:00.258999 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698f2d8-2743-47a7-ba05-ae966b69f41f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:00.297297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86ad0b-9431-4b88-9dff-9f927c56b255 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:00.298929 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1bb518-ce07-4cf4-a10f-e47da2a9c03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:00.299796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bb518-ce07-4cf4-a10f-e47da2a9c03a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:00.300242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bb518-ce07-4cf4-a10f-e47da2a9c03a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:00.300628 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bb518-ce07-4cf4-a10f-e47da2a9c03a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:00.301565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1bb518-ce07-4cf4-a10f-e47da2a9c03a None admin] RBAC: Adding query filter params (role.id=3387bd4782b047ae8d880e386b0351f0, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=0da7f490d9604107b974e9e461ebc38b) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:00.301882 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1bb518-ce07-4cf4-a10f-e47da2a9c03a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:00.303465 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1bb518-ce07-4cf4-a10f-e47da2a9c03a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:00.309963 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 332/654] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:00 2022] GET /identity/v3/role_assignments?user.id=0da7f490d9604107b974e9e461ebc38b&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=3387bd4782b047ae8d880e386b0351f0 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:00.315182 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1bb518-ce07-4cf4-a10f-e47da2a9c03a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:00.347425 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2775aff-6a4d-4038-b9af-ae227ab992fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:00.349407 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2775aff-6a4d-4038-b9af-ae227ab992fe None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:00.349937 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2775aff-6a4d-4038-b9af-ae227ab992fe None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:00.350721 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2775aff-6a4d-4038-b9af-ae227ab992fe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:00.363145 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2775aff-6a4d-4038-b9af-ae227ab992fe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:00.366236 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2775aff-6a4d-4038-b9af-ae227ab992fe None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-b2775aff-6a4d-4038-b9af-ae227ab992fe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:00.387293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:00.388424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:00.389477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:00.389477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:00.389477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:00.389477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:00.389477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:00.389477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:00.389477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:00.389477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:00.389477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:51:00.389477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:00.392869 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 323/655] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:51:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:00.394060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2775aff-6a4d-4038-b9af-ae227ab992fe None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:00.448697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1bb518-ce07-4cf4-a10f-e47da2a9c03a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:00.479752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:00.490637 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a9875-0151-4f55-a2f0-9eb75b2c93e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:00.491617 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a9875-0151-4f55-a2f0-9eb75b2c93e4 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:00.491998 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a9875-0151-4f55-a2f0-9eb75b2c93e4 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:00.492371 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a9875-0151-4f55-a2f0-9eb75b2c93e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:00.505920 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a9875-0151-4f55-a2f0-9eb75b2c93e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:00.510115 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a9875-0151-4f55-a2f0-9eb75b2c93e4 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:00.528328 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:00.533436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-f50a9875-0151-4f55-a2f0-9eb75b2c93e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:00.533436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:00.533436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:00.533436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:00.533436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:00.533436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:00.533436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:00.533436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:00.533436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:00.533436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:00.533436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:00.533436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:00.533436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:00.533436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:00.533436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:00.534675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:00.535849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:00.535849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:00.535849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:00.535849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:00.540057 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 333/656] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:00.540390 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a9875-0151-4f55-a2f0-9eb75b2c93e4 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:00.551134 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2775aff-6a4d-4038-b9af-ae227ab992fe None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:00.593230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca34fc-77ff-436a-9d11-43369d6ab301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:00.594058 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca34fc-77ff-436a-9d11-43369d6ab301 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:00.594479 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca34fc-77ff-436a-9d11-43369d6ab301 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:00.594794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca34fc-77ff-436a-9d11-43369d6ab301 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:00.595526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca34fc-77ff-436a-9d11-43369d6ab301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:00.595834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca34fc-77ff-436a-9d11-43369d6ab301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:00.597543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca34fc-77ff-436a-9d11-43369d6ab301 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:00.615973 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 324/657] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:00.617807 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca34fc-77ff-436a-9d11-43369d6ab301 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:00.664082 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a9875-0151-4f55-a2f0-9eb75b2c93e4 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:00.721202 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705cc96-24d2-4529-80a6-159d22847b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:00.722559 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705cc96-24d2-4529-80a6-159d22847b9d None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:00.723070 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705cc96-24d2-4529-80a6-159d22847b9d None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:00.723551 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705cc96-24d2-4529-80a6-159d22847b9d None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:00.724772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705cc96-24d2-4529-80a6-159d22847b9d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:00.725298 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705cc96-24d2-4529-80a6-159d22847b9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:00.727009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705cc96-24d2-4529-80a6-159d22847b9d None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:00.746414 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca34fc-77ff-436a-9d11-43369d6ab301 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:00.750453 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 334/658] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 19:51:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:00.751589 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705cc96-24d2-4529-80a6-159d22847b9d None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:00.787501 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860f06b-83fe-4fae-b473-aae09061fed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:00.788176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860f06b-83fe-4fae-b473-aae09061fed2 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:00.788455 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860f06b-83fe-4fae-b473-aae09061fed2 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:00.788730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860f06b-83fe-4fae-b473-aae09061fed2 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:00.789464 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e860f06b-83fe-4fae-b473-aae09061fed2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:00.789770 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e860f06b-83fe-4fae-b473-aae09061fed2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:00.791353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e860f06b-83fe-4fae-b473-aae09061fed2 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:00.791746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:00.809832 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 325/659] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:00.809832 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860f06b-83fe-4fae-b473-aae09061fed2 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:00.824005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:00.880171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705cc96-24d2-4529-80a6-159d22847b9d None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:00.916994 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860f06b-83fe-4fae-b473-aae09061fed2 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:00.917439 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231b3d6-6215-45f4-a1ea-5e772c68cba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:00.918291 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231b3d6-6215-45f4-a1ea-5e772c68cba1 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:00.918592 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231b3d6-6215-45f4-a1ea-5e772c68cba1 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:00.918873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231b3d6-6215-45f4-a1ea-5e772c68cba1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:00.926987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231b3d6-6215-45f4-a1ea-5e772c68cba1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:00.928458 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231b3d6-6215-45f4-a1ea-5e772c68cba1 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-a231b3d6-6215-45f4-a1ea-5e772c68cba1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:00.944607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:00.945745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:00.946853 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:00.946853 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:00.946853 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:00.946853 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:00.946853 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:00.946853 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:00.946853 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:00.946853 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:00.946853 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:51:00.946853 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:00.950499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 335/660] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:51:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:00.951200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231b3d6-6215-45f4-a1ea-5e772c68cba1 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:00.955219 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10f522-c25a-4a8d-92ca-9c2ebf4b7593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:00.955884 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10f522-c25a-4a8d-92ca-9c2ebf4b7593 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:00.956165 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10f522-c25a-4a8d-92ca-9c2ebf4b7593 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:00.956416 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10f522-c25a-4a8d-92ca-9c2ebf4b7593 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:01.033262 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10f522-c25a-4a8d-92ca-9c2ebf4b7593 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:01.034775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10f522-c25a-4a8d-92ca-9c2ebf4b7593 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:01.045191 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:01.057673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:01.064841 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231b3d6-6215-45f4-a1ea-5e772c68cba1 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:01.101851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f9484-a334-432c-a43c-efd50663b3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:01.102684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f9484-a334-432c-a43c-efd50663b3c3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:01.102966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f9484-a334-432c-a43c-efd50663b3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:01.103243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f9484-a334-432c-a43c-efd50663b3c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:01.110400 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f9484-a334-432c-a43c-efd50663b3c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:01.111862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f9484-a334-432c-a43c-efd50663b3c3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:01.126037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-ea1f9484-a334-432c-a43c-efd50663b3c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:01.126037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:01.126037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:01.126037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:01.126037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:01.126037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:01.126037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:01.126037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:01.126037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:01.126037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:01.126037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:01.126037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:01.126037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:01.126037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:01.126037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:01.127106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:01.128112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:01.128112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:01.128112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:01.128112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:01.128112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:01.128112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:01.128112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:01.128112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:01.128112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 336/661] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:01.128648 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da01d4e-17e0-48ec-b518-072f1dc4dcd1 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:01.128723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da01d4e-17e0-48ec-b518-072f1dc4dcd1 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:01.129122 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da01d4e-17e0-48ec-b518-072f1dc4dcd1 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:01.132463 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 337/662] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:01.132463 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da01d4e-17e0-48ec-b518-072f1dc4dcd1 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:01.149792 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 326/663] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:51:01.151540 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10f522-c25a-4a8d-92ca-9c2ebf4b7593 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:01.242579 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da01d4e-17e0-48ec-b518-072f1dc4dcd1 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:01.261959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10f522-c25a-4a8d-92ca-9c2ebf4b7593 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:01.271970 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:01.281191 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a92cb1-dc94-406d-960b-7e3341b11efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:01.281875 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a92cb1-dc94-406d-960b-7e3341b11efb None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:01.282166 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a92cb1-dc94-406d-960b-7e3341b11efb None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:01.282467 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a92cb1-dc94-406d-960b-7e3341b11efb None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:01.283378 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a92cb1-dc94-406d-960b-7e3341b11efb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:01.283616 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a92cb1-dc94-406d-960b-7e3341b11efb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:01.285049 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a92cb1-dc94-406d-960b-7e3341b11efb None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:01.304820 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 338/664] 10.222.0.84 () {56 vars in 1139 bytes} [Thu May 19 19:51:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:01.306093 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462db969-decf-4371-93b3-8b73656d6088 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:01.306463 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462db969-decf-4371-93b3-8b73656d6088 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:01.306687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462db969-decf-4371-93b3-8b73656d6088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:01.310895 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a481497-b247-4f1f-a5a1-9f27a323f0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:01.312271 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a481497-b247-4f1f-a5a1-9f27a323f0e3 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:01.312977 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a481497-b247-4f1f-a5a1-9f27a323f0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:01.313639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a481497-b247-4f1f-a5a1-9f27a323f0e3 None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:01.314940 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a481497-b247-4f1f-a5a1-9f27a323f0e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:01.315439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a481497-b247-4f1f-a5a1-9f27a323f0e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:01.317723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a481497-b247-4f1f-a5a1-9f27a323f0e3 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:01.324828 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-462db969-decf-4371-93b3-8b73656d6088 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:01.326038 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-462db969-decf-4371-93b3-8b73656d6088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:01.336183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 327/665] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:01.336183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a481497-b247-4f1f-a5a1-9f27a323f0e3 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:01.339807 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:01.401656 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 339/666] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:51:01.402890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462db969-decf-4371-93b3-8b73656d6088 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:01.454787 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a481497-b247-4f1f-a5a1-9f27a323f0e3 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:01.489731 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a76f99-c0c9-424b-a94d-06dc17fbc9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:01.490568 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a76f99-c0c9-424b-a94d-06dc17fbc9a5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:01.490856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a76f99-c0c9-424b-a94d-06dc17fbc9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:01.491131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a76f99-c0c9-424b-a94d-06dc17fbc9a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:01.502490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a76f99-c0c9-424b-a94d-06dc17fbc9a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:01.504009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a76f99-c0c9-424b-a94d-06dc17fbc9a5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:01.513910 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462db969-decf-4371-93b3-8b73656d6088 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:01.521593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-d9a76f99-c0c9-424b-a94d-06dc17fbc9a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:01.521593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:01.521593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:01.521593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:01.521593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:01.521593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:01.521593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:01.521593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:01.521593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:01.521593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:01.521593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:01.521593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:01.521593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:01.521593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:01.521593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:01.522771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:01.523805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:01.523805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:01.523805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:01.523805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:01.523805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 328/667] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:01.524125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a76f99-c0c9-424b-a94d-06dc17fbc9a5 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:01.549577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:01.556187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebe6f7-9e56-48b3-9ec0-4e7bcd934df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:01.557409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebe6f7-9e56-48b3-9ec0-4e7bcd934df3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:01.557715 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebe6f7-9e56-48b3-9ec0-4e7bcd934df3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:01.557987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebe6f7-9e56-48b3-9ec0-4e7bcd934df3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:01.576865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:01.634030 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a76f99-c0c9-424b-a94d-06dc17fbc9a5 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:01.636491 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebe6f7-9e56-48b3-9ec0-4e7bcd934df3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:01.638005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebe6f7-9e56-48b3-9ec0-4e7bcd934df3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:01.676540 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df26e1a-1b25-446a-b320-7b2cb7384b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:01.677209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df26e1a-1b25-446a-b320-7b2cb7384b08 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:01.677491 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df26e1a-1b25-446a-b320-7b2cb7384b08 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:01.677763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df26e1a-1b25-446a-b320-7b2cb7384b08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:01.678668 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df26e1a-1b25-446a-b320-7b2cb7384b08 None admin] RBAC: Adding query filter params (role.id=3387bd4782b047ae8d880e386b0351f0, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=64296c932944405abbf8e767570c84f1) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:01.678984 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df26e1a-1b25-446a-b320-7b2cb7384b08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:01.680434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df26e1a-1b25-446a-b320-7b2cb7384b08 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:01.686626 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 329/668] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:01 2022] GET /identity/v3/role_assignments?user.id=64296c932944405abbf8e767570c84f1&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=3387bd4782b047ae8d880e386b0351f0 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:01.687607 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df26e1a-1b25-446a-b320-7b2cb7384b08 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:01.776763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 340/669] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:51:01.777794 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebe6f7-9e56-48b3-9ec0-4e7bcd934df3 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:01.828145 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df26e1a-1b25-446a-b320-7b2cb7384b08 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:01.840835 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:01.858787 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:01.874904 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f384a-f166-4ba0-909f-cf5fd2e72b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:01.876214 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f384a-f166-4ba0-909f-cf5fd2e72b7e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:01.876214 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f384a-f166-4ba0-909f-cf5fd2e72b7e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:01.876405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f384a-f166-4ba0-909f-cf5fd2e72b7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:01.888889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f384a-f166-4ba0-909f-cf5fd2e72b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-932f384a-f166-4ba0-909f-cf5fd2e72b7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:01.897057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:01.898144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:01.899137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:01.899137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:01.899137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:01.899137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:01.899137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:01.899137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:01.899137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:01.899137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:01.899137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:01.899137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:01.899137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:51:01.899137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:01.899137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 330/670] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:51:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:01.899137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f384a-f166-4ba0-909f-cf5fd2e72b7e None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:01.917331 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebe6f7-9e56-48b3-9ec0-4e7bcd934df3 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:01.969028 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb87100-a700-4b9a-afec-0f8eb4bd5b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:01.969708 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb87100-a700-4b9a-afec-0f8eb4bd5b53 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:01.969989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb87100-a700-4b9a-afec-0f8eb4bd5b53 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:01.970295 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb87100-a700-4b9a-afec-0f8eb4bd5b53 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:01.971079 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb87100-a700-4b9a-afec-0f8eb4bd5b53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:01.971492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb87100-a700-4b9a-afec-0f8eb4bd5b53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:01.973680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb87100-a700-4b9a-afec-0f8eb4bd5b53 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:01.988010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 341/671] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:01.988697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb87100-a700-4b9a-afec-0f8eb4bd5b53 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:02.033745 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f384a-f166-4ba0-909f-cf5fd2e72b7e None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:02.072871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc6505-b805-4449-a044-c56ae55bd533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:02.073704 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc6505-b805-4449-a044-c56ae55bd533 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:02.073981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc6505-b805-4449-a044-c56ae55bd533 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:02.074254 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc6505-b805-4449-a044-c56ae55bd533 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:02.086020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc6505-b805-4449-a044-c56ae55bd533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:02.087510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc6505-b805-4449-a044-c56ae55bd533 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:02.090583 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:02.108555 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb87100-a700-4b9a-afec-0f8eb4bd5b53 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:02.111596 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:02.150467 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1b266-0189-4b56-8032-13d70fc47b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:02.151166 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1b266-0189-4b56-8032-13d70fc47b26 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:02.151448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1b266-0189-4b56-8032-13d70fc47b26 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:02.151777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1b266-0189-4b56-8032-13d70fc47b26 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:02.152563 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1b266-0189-4b56-8032-13d70fc47b26 None admin] RBAC: Adding query filter params (name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:02.152873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1b266-0189-4b56-8032-13d70fc47b26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:02.154347 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1b266-0189-4b56-8032-13d70fc47b26 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:02.164910 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 342/672] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:51:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:02.165905 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1b266-0189-4b56-8032-13d70fc47b26 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:02.236430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-43cc6505-b805-4449-a044-c56ae55bd533 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:02.236430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:02.236430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:02.236430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:02.236430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:02.236430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:02.236430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:02.236430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:02.236430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:02.236430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:02.236430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:02.236430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:02.236430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:02.236430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:02.236430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:02.237531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:02.238613 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:02.238613 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:02.238613 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:02.238613 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:02.238613 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 331/673] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:02.239611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc6505-b805-4449-a044-c56ae55bd533 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:02.294901 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1b266-0189-4b56-8032-13d70fc47b26 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:02.343620 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b4a51-43cc-45f4-acfe-e07d9bab1acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:02.344433 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b4a51-43cc-45f4-acfe-e07d9bab1acb None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:02.344839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b4a51-43cc-45f4-acfe-e07d9bab1acb None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:02.345122 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b4a51-43cc-45f4-acfe-e07d9bab1acb None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:02.346562 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc6505-b805-4449-a044-c56ae55bd533 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:02.353854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b4a51-43cc-45f4-acfe-e07d9bab1acb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:02.355902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b4a51-43cc-45f4-acfe-e07d9bab1acb None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:02.373228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-d54b4a51-43cc-45f4-acfe-e07d9bab1acb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:02.373228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:02.373228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:02.373228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:02.373228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:02.373228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:02.373228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:02.373228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:02.373228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:02.373228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:02.373228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:02.373228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:02.373228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:02.373228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:02.373228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:02.375370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:02.377412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:02.377412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:02.377412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:02.377412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:02.377412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:02.377412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:02.377412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:02.377412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:02.377412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 343/674] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:02.377412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b4a51-43cc-45f4-acfe-e07d9bab1acb None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:02.387983 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8ba8d-3dec-416d-a279-3cc4e75e381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:02.389320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8ba8d-3dec-416d-a279-3cc4e75e381f None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:02.389728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8ba8d-3dec-416d-a279-3cc4e75e381f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:02.390646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8ba8d-3dec-416d-a279-3cc4e75e381f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:02.404281 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8ba8d-3dec-416d-a279-3cc4e75e381f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:02.406995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8ba8d-3dec-416d-a279-3cc4e75e381f None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:02.408254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:02.432806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-60a8ba8d-3dec-416d-a279-3cc4e75e381f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:02.432806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:02.432806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:02.432806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:02.432806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:02.432806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:02.432806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:02.432806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:02.432806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:02.432806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:02.432806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:02.432806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:02.432806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:02.432806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:02.432806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:02.434080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:02.435456 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:02.435456 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:02.435456 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:02.435456 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:02.437159 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 332/675] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:02.438105 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8ba8d-3dec-416d-a279-3cc4e75e381f None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:02.489175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b4a51-43cc-45f4-acfe-e07d9bab1acb None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:02.494002 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:02.551584 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2bbef-96f9-4309-b799-2d7006b3d2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:02.552273 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2bbef-96f9-4309-b799-2d7006b3d2aa None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:02.552556 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2bbef-96f9-4309-b799-2d7006b3d2aa None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:02.552828 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2bbef-96f9-4309-b799-2d7006b3d2aa None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:02.553574 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2bbef-96f9-4309-b799-2d7006b3d2aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:02.553880 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2bbef-96f9-4309-b799-2d7006b3d2aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:02.555327 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2bbef-96f9-4309-b799-2d7006b3d2aa None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:02.565326 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8ba8d-3dec-416d-a279-3cc4e75e381f None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:02.568203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 344/676] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:02.569221 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2bbef-96f9-4309-b799-2d7006b3d2aa None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:02.608942 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89270be7-c21e-453d-9dd3-4789661baa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:02.609765 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89270be7-c21e-453d-9dd3-4789661baa25 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:02.610149 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89270be7-c21e-453d-9dd3-4789661baa25 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:02.610739 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89270be7-c21e-453d-9dd3-4789661baa25 None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:02.611446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89270be7-c21e-453d-9dd3-4789661baa25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:02.611891 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89270be7-c21e-453d-9dd3-4789661baa25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:02.613419 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89270be7-c21e-453d-9dd3-4789661baa25 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:02.627666 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 333/677] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:02.628420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89270be7-c21e-453d-9dd3-4789661baa25 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:02.682735 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:02.699500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2bbef-96f9-4309-b799-2d7006b3d2aa None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:02.737395 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e619161-8753-483e-ba09-51a96c5380e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:02.738161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e619161-8753-483e-ba09-51a96c5380e4 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:02.738556 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e619161-8753-483e-ba09-51a96c5380e4 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:02.738900 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e619161-8753-483e-ba09-51a96c5380e4 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:02.739705 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e619161-8753-483e-ba09-51a96c5380e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:02.740084 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e619161-8753-483e-ba09-51a96c5380e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:02.741468 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e619161-8753-483e-ba09-51a96c5380e4 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:02.752905 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89270be7-c21e-453d-9dd3-4789661baa25 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:02.754075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 345/678] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:02.754922 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e619161-8753-483e-ba09-51a96c5380e4 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:02.767923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:02.796618 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e1f44-6e5d-4d74-9a12-2431901dba73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:02.797426 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e1f44-6e5d-4d74-9a12-2431901dba73 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:02.797707 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e1f44-6e5d-4d74-9a12-2431901dba73 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:02.797983 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e1f44-6e5d-4d74-9a12-2431901dba73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:02.805400 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e1f44-6e5d-4d74-9a12-2431901dba73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:02.806991 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e1f44-6e5d-4d74-9a12-2431901dba73 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:02.822520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-2c8e1f44-6e5d-4d74-9a12-2431901dba73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:02.822520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:02.822520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:02.822520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:02.822520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:02.822520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:02.822520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:02.822520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:02.822520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:02.822520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:02.822520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:02.822520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:02.822520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:02.822520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:02.822520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:02.823525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:02.824529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:02.824529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:02.824529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:02.824529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:02.824529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:02.824529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:02.824529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:02.824529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:02.824529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 334/679] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:02.825064 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500a20d-3943-4bed-bd3c-c04a5dbeabaf None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:02.825659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500a20d-3943-4bed-bd3c-c04a5dbeabaf None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:02.825659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500a20d-3943-4bed-bd3c-c04a5dbeabaf None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:02.830446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 335/680] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:02.830446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500a20d-3943-4bed-bd3c-c04a5dbeabaf None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:02.866153 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e619161-8753-483e-ba09-51a96c5380e4 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:02.904106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d17c15-2ae4-40d3-b427-643aa9e5e799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:02.904794 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d17c15-2ae4-40d3-b427-643aa9e5e799 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:02.905144 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d17c15-2ae4-40d3-b427-643aa9e5e799 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:02.905418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d17c15-2ae4-40d3-b427-643aa9e5e799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:02.906469 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d17c15-2ae4-40d3-b427-643aa9e5e799 None admin] RBAC: Adding query filter params (role.id=3387bd4782b047ae8d880e386b0351f0, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=6b89f5a099d24437a25d5d7f21e10e2e) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:02.906780 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d17c15-2ae4-40d3-b427-643aa9e5e799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:02.908219 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d17c15-2ae4-40d3-b427-643aa9e5e799 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:02.922295 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 346/681] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:02 2022] GET /identity/v3/role_assignments?user.id=6b89f5a099d24437a25d5d7f21e10e2e&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=3387bd4782b047ae8d880e386b0351f0 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:02.922295 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d17c15-2ae4-40d3-b427-643aa9e5e799 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:02.933153 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:02.958050 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500a20d-3943-4bed-bd3c-c04a5dbeabaf None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:03.007587 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc653e8-b973-4bb9-81dd-754d7c04ed32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:03.008414 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc653e8-b973-4bb9-81dd-754d7c04ed32 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:03.008698 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc653e8-b973-4bb9-81dd-754d7c04ed32 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:03.008975 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc653e8-b973-4bb9-81dd-754d7c04ed32 None admin] PATH_INFO: `/v3/users/swift` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:03.019192 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc653e8-b973-4bb9-81dd-754d7c04ed32 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:03.020869 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc653e8-b973-4bb9-81dd-754d7c04ed32 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-5dc653e8-b973-4bb9-81dd-754d7c04ed32 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:03.037196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:03.039024 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:03.040297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:03.040297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:03.040297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:03.040297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:03.040297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:03.040297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:03.040297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:03.040297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:03.040297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:51:03.040297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:03.040888 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d17c15-2ae4-40d3-b427-643aa9e5e799 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:03.042795 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 336/682] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:51:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:03.044394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d232b-8edc-41fb-b081-a4c899c18c36 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:03.044686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d232b-8edc-41fb-b081-a4c899c18c36 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:03.044964 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d232b-8edc-41fb-b081-a4c899c18c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:03.064740 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-f85d232b-8edc-41fb-b081-a4c899c18c36 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:03.066066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-f85d232b-8edc-41fb-b081-a4c899c18c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:03.083688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb8111-9afb-439b-adfd-9ebb9e2afd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:03.084557 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb8111-9afb-439b-adfd-9ebb9e2afd38 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:03.084847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb8111-9afb-439b-adfd-9ebb9e2afd38 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:03.085136 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb8111-9afb-439b-adfd-9ebb9e2afd38 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:03.086313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb8111-9afb-439b-adfd-9ebb9e2afd38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:03.086632 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb8111-9afb-439b-adfd-9ebb9e2afd38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:03.088137 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb8111-9afb-439b-adfd-9ebb9e2afd38 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:03.102971 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:03.104914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 347/683] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:03.106058 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb8111-9afb-439b-adfd-9ebb9e2afd38 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:03.159652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 337/684] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:51:03.160495 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d232b-8edc-41fb-b081-a4c899c18c36 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:03.204461 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:03.248121 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb8111-9afb-439b-adfd-9ebb9e2afd38 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:03.290784 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb28e9-d69a-4003-a32c-872678593e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:03.291776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb28e9-d69a-4003-a32c-872678593e5f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:03.292215 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb28e9-d69a-4003-a32c-872678593e5f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:03.292659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb28e9-d69a-4003-a32c-872678593e5f None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:03.293672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb28e9-d69a-4003-a32c-872678593e5f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:03.294156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb28e9-d69a-4003-a32c-872678593e5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:03.294384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d232b-8edc-41fb-b081-a4c899c18c36 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:03.296162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb28e9-d69a-4003-a32c-872678593e5f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:03.324702 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 348/685] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 19:51:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:03.325673 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb28e9-d69a-4003-a32c-872678593e5f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:03.343753 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4ea40-af95-49e5-8092-3f6785b4d073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:03.344424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4ea40-af95-49e5-8092-3f6785b4d073 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:03.345016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4ea40-af95-49e5-8092-3f6785b4d073 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:03.345295 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4ea40-af95-49e5-8092-3f6785b4d073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:03.346513 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4ea40-af95-49e5-8092-3f6785b4d073 None admin] RBAC: Adding query filter params (role.id=3387bd4782b047ae8d880e386b0351f0, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=24d916d5c44c4078958734bd59a220b3) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:03.347146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4ea40-af95-49e5-8092-3f6785b4d073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:03.348824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4ea40-af95-49e5-8092-3f6785b4d073 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:03.357465 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 338/686] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:03 2022] GET /identity/v3/role_assignments?user.id=24d916d5c44c4078958734bd59a220b3&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=3387bd4782b047ae8d880e386b0351f0 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:03.358523 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4ea40-af95-49e5-8092-3f6785b4d073 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:03.387551 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:03.500415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4ea40-af95-49e5-8092-3f6785b4d073 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:03.544842 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e730a-5145-4768-9f3e-f951f393f0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:03.545523 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e730a-5145-4768-9f3e-f951f393f0a1 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:03.546155 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e730a-5145-4768-9f3e-f951f393f0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:03.546504 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e730a-5145-4768-9f3e-f951f393f0a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:03.588360 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb28e9-d69a-4003-a32c-872678593e5f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:03.629931 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e730a-5145-4768-9f3e-f951f393f0a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:03.631660 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e730a-5145-4768-9f3e-f951f393f0a1 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:03.643123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:03.659075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066c67b-9ddb-421e-ad32-b4367996495d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:03.659908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066c67b-9ddb-421e-ad32-b4367996495d None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:03.660195 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066c67b-9ddb-421e-ad32-b4367996495d None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:03.660472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066c67b-9ddb-421e-ad32-b4367996495d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:03.670649 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:03.676047 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066c67b-9ddb-421e-ad32-b4367996495d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-0066c67b-9ddb-421e-ad32-b4367996495d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:03.690420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:03.691564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:03.692522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:03.692522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:03.692522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:03.692522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:03.692522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:03.692522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:03.692522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:03.692522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:03.692522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:03.692522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:03.692522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:03.692522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:03.693243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 349/687] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:51:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:03.699299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066c67b-9ddb-421e-ad32-b4367996495d None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:03.763524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 339/688] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:51:03.770345 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e730a-5145-4768-9f3e-f951f393f0a1 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:03.825834 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066c67b-9ddb-421e-ad32-b4367996495d None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:03.872255 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f60dd-0292-4d1f-bc00-52b127a2d8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:03.873275 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f60dd-0292-4d1f-bc00-52b127a2d8f4 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:03.873753 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f60dd-0292-4d1f-bc00-52b127a2d8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:03.874095 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f60dd-0292-4d1f-bc00-52b127a2d8f4 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:03.874922 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f60dd-0292-4d1f-bc00-52b127a2d8f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:03.875218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f60dd-0292-4d1f-bc00-52b127a2d8f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:03.876726 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f60dd-0292-4d1f-bc00-52b127a2d8f4 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:03.891578 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 350/689] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:51:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:03.892332 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f60dd-0292-4d1f-bc00-52b127a2d8f4 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:03.918999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e730a-5145-4768-9f3e-f951f393f0a1 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:03.946979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:03.960678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9ac85-9ea4-465a-bc73-095cf1cff1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:03.962345 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9ac85-9ea4-465a-bc73-095cf1cff1e1 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:03.962639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9ac85-9ea4-465a-bc73-095cf1cff1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:03.963336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9ac85-9ea4-465a-bc73-095cf1cff1e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:03.981443 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:03.986461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9ac85-9ea4-465a-bc73-095cf1cff1e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:03.987954 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9ac85-9ea4-465a-bc73-095cf1cff1e1 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:04.015564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-b7c9ac85-9ea4-465a-bc73-095cf1cff1e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:04.015564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:04.015564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:04.015564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:04.015564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:04.015564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:04.015564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:04.015564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:04.015564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:04.015564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:04.015564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:04.015564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:04.015564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:04.015564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:04.015564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:04.016635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:04.017667 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:04.017667 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:04.017667 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:04.017667 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:04.017667 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 340/690] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:04.024078 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9ac85-9ea4-465a-bc73-095cf1cff1e1 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:04.066156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f60dd-0292-4d1f-bc00-52b127a2d8f4 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:04.103269 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c54d5-f433-43e3-961e-9eeeddfb4dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:04.104253 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c54d5-f433-43e3-961e-9eeeddfb4dfe None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:04.104537 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c54d5-f433-43e3-961e-9eeeddfb4dfe None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:04.104810 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c54d5-f433-43e3-961e-9eeeddfb4dfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:04.116130 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8c54d5-f433-43e3-961e-9eeeddfb4dfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:04.117593 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8c54d5-f433-43e3-961e-9eeeddfb4dfe None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:04.135296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-cb8c54d5-f433-43e3-961e-9eeeddfb4dfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:04.135296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:04.135296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:04.135296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:04.135296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:04.135296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:04.135296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:04.135296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:04.135296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:04.135296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:04.135296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:04.135296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:04.135296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:04.135296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:04.135296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:04.136353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:04.137369 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:04.137369 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:04.137369 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:04.137369 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:04.137369 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 351/691] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:04.147276 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c54d5-f433-43e3-961e-9eeeddfb4dfe None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:04.185063 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9ac85-9ea4-465a-bc73-095cf1cff1e1 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:04.231062 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ead58c-5148-42a2-83dd-5448c47c4d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:04.232134 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ead58c-5148-42a2-83dd-5448c47c4d99 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:04.232642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ead58c-5148-42a2-83dd-5448c47c4d99 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:04.233166 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ead58c-5148-42a2-83dd-5448c47c4d99 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:04.234136 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ead58c-5148-42a2-83dd-5448c47c4d99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:04.234670 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ead58c-5148-42a2-83dd-5448c47c4d99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:04.236258 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ead58c-5148-42a2-83dd-5448c47c4d99 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:04.254704 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 341/692] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:04.260383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ead58c-5148-42a2-83dd-5448c47c4d99 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:04.331958 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:04.382679 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c54d5-f433-43e3-961e-9eeeddfb4dfe None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:04.384344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:04.412835 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ead58c-5148-42a2-83dd-5448c47c4d99 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:04.429495 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f15e2-7507-43f6-a2f5-37930228e103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:04.431415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f15e2-7507-43f6-a2f5-37930228e103 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:04.431910 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f15e2-7507-43f6-a2f5-37930228e103 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:04.432621 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f15e2-7507-43f6-a2f5-37930228e103 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:04.433934 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f15e2-7507-43f6-a2f5-37930228e103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:04.434468 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f15e2-7507-43f6-a2f5-37930228e103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:04.436069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f15e2-7507-43f6-a2f5-37930228e103 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:04.451435 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32383b15-a354-40db-8062-3805a6079994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:04.452513 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32383b15-a354-40db-8062-3805a6079994 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:04.452876 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32383b15-a354-40db-8062-3805a6079994 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:04.453276 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32383b15-a354-40db-8062-3805a6079994 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:04.453908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 352/693] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:04.455465 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad538fb-7b04-470d-a0fb-0a56df585581 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:04.455924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad538fb-7b04-470d-a0fb-0a56df585581 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:04.456369 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad538fb-7b04-470d-a0fb-0a56df585581 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:04.458392 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 353/694] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:04.459322 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad538fb-7b04-470d-a0fb-0a56df585581 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:04.470336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32383b15-a354-40db-8062-3805a6079994 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:04.472140 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32383b15-a354-40db-8062-3805a6079994 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:04.487766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-32383b15-a354-40db-8062-3805a6079994 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:04.487766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:04.487766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:04.487766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:04.487766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:04.487766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:04.487766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:04.487766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:04.487766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:04.487766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:04.487766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:04.487766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:04.487766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:04.487766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:04.487766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:04.488847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:04.489856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:04.489856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:04.489856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:04.489856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:04.489856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:04.489856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:04.489856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:04.489856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:04.493949 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 342/695] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:04.495494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83acbc5-4c53-40b5-8a18-c1040d009f3f None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:04.495772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83acbc5-4c53-40b5-8a18-c1040d009f3f None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:04.496101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83acbc5-4c53-40b5-8a18-c1040d009f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:04.514982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-a83acbc5-4c53-40b5-8a18-c1040d009f3f None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:04.516241 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-a83acbc5-4c53-40b5-8a18-c1040d009f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:04.598416 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 343/696] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:51:04.598416 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83acbc5-4c53-40b5-8a18-c1040d009f3f None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:04.599457 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad538fb-7b04-470d-a0fb-0a56df585581 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:04.635841 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:04.657054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3efc47-8c7d-439c-9305-7faf3fc6925c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:04.658520 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3efc47-8c7d-439c-9305-7faf3fc6925c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:04.658907 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3efc47-8c7d-439c-9305-7faf3fc6925c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:04.659552 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3efc47-8c7d-439c-9305-7faf3fc6925c None admin] PATH_INFO: `/v3/users/nova` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:04.669453 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3efc47-8c7d-439c-9305-7faf3fc6925c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:04.671971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3efc47-8c7d-439c-9305-7faf3fc6925c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-5b3efc47-8c7d-439c-9305-7faf3fc6925c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:04.691925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:04.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:04.694126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:04.694126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:04.694126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:04.694126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:04.694126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:04.694126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:04.694126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:04.694126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:04.694126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:51:04.694126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:04.701188 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 354/697] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:51:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:04.705729 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3efc47-8c7d-439c-9305-7faf3fc6925c None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:04.730142 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83acbc5-4c53-40b5-8a18-c1040d009f3f None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:04.737996 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:04.775013 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de056b3-5a9d-4892-ac71-c6cff10b4196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:04.776033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de056b3-5a9d-4892-ac71-c6cff10b4196 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:04.776645 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de056b3-5a9d-4892-ac71-c6cff10b4196 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:04.777119 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de056b3-5a9d-4892-ac71-c6cff10b4196 None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:04.778146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de056b3-5a9d-4892-ac71-c6cff10b4196 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:04.778696 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de056b3-5a9d-4892-ac71-c6cff10b4196 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:04.780407 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de056b3-5a9d-4892-ac71-c6cff10b4196 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:04.797821 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 344/698] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:04.799126 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de056b3-5a9d-4892-ac71-c6cff10b4196 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:04.828793 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3efc47-8c7d-439c-9305-7faf3fc6925c None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:04.875811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c2590-90f7-4f46-bff3-2ece865ab2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:04.876538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c2590-90f7-4f46-bff3-2ece865ab2e6 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:04.876819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c2590-90f7-4f46-bff3-2ece865ab2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:04.877091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c2590-90f7-4f46-bff3-2ece865ab2e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:04.890108 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:04.891489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c2590-90f7-4f46-bff3-2ece865ab2e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:04.893293 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c2590-90f7-4f46-bff3-2ece865ab2e6 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:04.909454 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de056b3-5a9d-4892-ac71-c6cff10b4196 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:04.917212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-6b0c2590-90f7-4f46-bff3-2ece865ab2e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:04.917212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:04.917212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:04.917212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:04.917212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:04.917212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:04.917212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:04.917212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:04.917212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:04.917212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:04.917212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:04.917212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:04.917212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:04.917212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:04.917212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:04.918214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:04.919246 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:04.919246 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:04.919246 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:04.919246 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:04.922414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 355/699] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:04.923760 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c2590-90f7-4f46-bff3-2ece865ab2e6 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:04.950213 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8d0eb-a933-4f67-ac11-61a979288a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:04.951281 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8d0eb-a933-4f67-ac11-61a979288a8e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:04.951798 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8d0eb-a933-4f67-ac11-61a979288a8e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:04.952324 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8d0eb-a933-4f67-ac11-61a979288a8e None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:04.953380 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8d0eb-a933-4f67-ac11-61a979288a8e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:04.953901 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8d0eb-a933-4f67-ac11-61a979288a8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:04.955592 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8d0eb-a933-4f67-ac11-61a979288a8e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:04.980135 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 345/700] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:51:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:04.981383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8d0eb-a933-4f67-ac11-61a979288a8e None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:05.022336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:05.050743 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c2590-90f7-4f46-bff3-2ece865ab2e6 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:05.089951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbf8e1-14e8-4aa3-8232-ba433471e220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:05.090679 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbf8e1-14e8-4aa3-8232-ba433471e220 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:05.090967 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbf8e1-14e8-4aa3-8232-ba433471e220 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:05.091247 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbf8e1-14e8-4aa3-8232-ba433471e220 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:05.092203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbf8e1-14e8-4aa3-8232-ba433471e220 None admin] RBAC: Adding query filter params (role.id=3387bd4782b047ae8d880e386b0351f0, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=acebbcf1f90d487aa21a8eb99102ce09) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:05.092574 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbf8e1-14e8-4aa3-8232-ba433471e220 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:05.094496 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbf8e1-14e8-4aa3-8232-ba433471e220 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:05.101980 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 356/701] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:04 2022] GET /identity/v3/role_assignments?user.id=acebbcf1f90d487aa21a8eb99102ce09&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=3387bd4782b047ae8d880e386b0351f0 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:05.103010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbf8e1-14e8-4aa3-8232-ba433471e220 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:05.110188 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8d0eb-a933-4f67-ac11-61a979288a8e None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:05.147785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f6d4c-1f25-4662-ac23-24cf27e1c201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:05.148849 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f6d4c-1f25-4662-ac23-24cf27e1c201 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:05.149348 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f6d4c-1f25-4662-ac23-24cf27e1c201 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:05.149846 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f6d4c-1f25-4662-ac23-24cf27e1c201 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:05.150838 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f6d4c-1f25-4662-ac23-24cf27e1c201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:05.151354 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f6d4c-1f25-4662-ac23-24cf27e1c201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:05.152955 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f6d4c-1f25-4662-ac23-24cf27e1c201 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:05.166335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 346/702] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:05.172284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f6d4c-1f25-4662-ac23-24cf27e1c201 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:05.175640 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:05.240009 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbf8e1-14e8-4aa3-8232-ba433471e220 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:05.290633 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f6d4c-1f25-4662-ac23-24cf27e1c201 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:05.298176 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf52828-0520-4046-825f-6ecd6ff2b9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:05.300603 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf52828-0520-4046-825f-6ecd6ff2b9d5 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:05.300603 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf52828-0520-4046-825f-6ecd6ff2b9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:05.300603 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf52828-0520-4046-825f-6ecd6ff2b9d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:05.310168 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:05.368125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ac9ab-2715-4177-8066-b332be9a7b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:05.368922 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ac9ab-2715-4177-8066-b332be9a7b32 None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:05.369270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ac9ab-2715-4177-8066-b332be9a7b32 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:05.369878 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ac9ab-2715-4177-8066-b332be9a7b32 None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:05.371880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ac9ab-2715-4177-8066-b332be9a7b32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:05.373983 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ac9ab-2715-4177-8066-b332be9a7b32 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:05.387135 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf52828-0520-4046-825f-6ecd6ff2b9d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:05.389310 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf52828-0520-4046-825f-6ecd6ff2b9d5 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:05.407066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 347/703] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:51:05 2022] POST /identity/v3/users => generated 254 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:51:05.515669 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2143f-a1ec-4e72-89d3-a7d8664b0dee None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:05.515935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2143f-a1ec-4e72-89d3-a7d8664b0dee None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:05.516588 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2143f-a1ec-4e72-89d3-a7d8664b0dee None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:05.522460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 348/704] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:05.524272 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798989a-b181-4050-a72e-f472a4c95c0c None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:05.524564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798989a-b181-4050-a72e-f472a4c95c0c None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:05.524850 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798989a-b181-4050-a72e-f472a4c95c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:05.531377 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 357/705] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:51:05.540885 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf52828-0520-4046-825f-6ecd6ff2b9d5 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:05.548129 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-9798989a-b181-4050-a72e-f472a4c95c0c None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:05.549744 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-9798989a-b181-4050-a72e-f472a4c95c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:05.584618 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:05.617175 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:05.630919 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 349/706] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:51:05.654054 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798989a-b181-4050-a72e-f472a4c95c0c None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:05.659614 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf52828-0520-4046-825f-6ecd6ff2b9d5 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:05.700795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0f6f2-b16c-4985-9bcd-ff0b3b1a59b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:05.701762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0f6f2-b16c-4985-9bcd-ff0b3b1a59b0 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:05.702215 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0f6f2-b16c-4985-9bcd-ff0b3b1a59b0 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:05.702544 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0f6f2-b16c-4985-9bcd-ff0b3b1a59b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:05.716861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0f6f2-b16c-4985-9bcd-ff0b3b1a59b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:05.719925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0f6f2-b16c-4985-9bcd-ff0b3b1a59b0 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:05.739472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-1ca0f6f2-b16c-4985-9bcd-ff0b3b1a59b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:05.739472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:05.739472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:05.739472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:05.739472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:05.739472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:05.739472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:05.739472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:05.739472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:05.739472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:05.739472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:05.739472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:05.739472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:05.739472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:05.739472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:05.740723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:05.741947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:05.741947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:05.741947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:05.741947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:05.741947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 358/707] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:05.747853 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0f6f2-b16c-4985-9bcd-ff0b3b1a59b0 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:05.801637 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798989a-b181-4050-a72e-f472a4c95c0c None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:05.840014 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7797aa-9774-42d3-a3f2-c62f0560b524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:05.841306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7797aa-9774-42d3-a3f2-c62f0560b524 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:05.841792 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7797aa-9774-42d3-a3f2-c62f0560b524 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:05.842295 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7797aa-9774-42d3-a3f2-c62f0560b524 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:05.843221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7797aa-9774-42d3-a3f2-c62f0560b524 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:05.844970 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7797aa-9774-42d3-a3f2-c62f0560b524 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:05.854329 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-9f7797aa-9774-42d3-a3f2-c62f0560b524 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:05.863382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:05.864516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:05.865460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:05.865460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:05.865460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:05.865460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:05.865460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:05.865460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:05.865460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:05.865460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:05.865460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:05.865460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:05.865460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:05.865460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:05.865460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:05.865460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:51:05.865460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:05.866402 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0f6f2-b16c-4985-9bcd-ff0b3b1a59b0 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:05.868701 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 350/708] 10.222.0.84 () {56 vars in 1129 bytes} [Thu May 19 19:51:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:05.875087 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7797aa-9774-42d3-a3f2-c62f0560b524 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:05.900787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375147e9-84c1-42e8-914b-c8a43c12874c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:05.901485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375147e9-84c1-42e8-914b-c8a43c12874c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:05.901785 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375147e9-84c1-42e8-914b-c8a43c12874c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:05.902184 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375147e9-84c1-42e8-914b-c8a43c12874c None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:05.902888 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375147e9-84c1-42e8-914b-c8a43c12874c None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:05.903364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375147e9-84c1-42e8-914b-c8a43c12874c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:05.904795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375147e9-84c1-42e8-914b-c8a43c12874c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:05.917248 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 359/709] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:05.926144 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375147e9-84c1-42e8-914b-c8a43c12874c None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:05.939759 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:05.993516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7797aa-9774-42d3-a3f2-c62f0560b524 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:06.030354 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d5944-1893-4e07-a150-c116b14586f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:06.031506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d5944-1893-4e07-a150-c116b14586f5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:06.032021 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d5944-1893-4e07-a150-c116b14586f5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:06.032511 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d5944-1893-4e07-a150-c116b14586f5 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:06.033429 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d5944-1893-4e07-a150-c116b14586f5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:06.033978 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d5944-1893-4e07-a150-c116b14586f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:06.035584 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d5944-1893-4e07-a150-c116b14586f5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:06.041116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 351/710] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 19:51:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:06.047243 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d5944-1893-4e07-a150-c116b14586f5 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:06.100637 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375147e9-84c1-42e8-914b-c8a43c12874c None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:06.159275 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d5944-1893-4e07-a150-c116b14586f5 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:06.160375 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4373e8-bd76-4cc8-a186-5f3598ba36fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:06.161225 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4373e8-bd76-4cc8-a186-5f3598ba36fe None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:06.161511 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4373e8-bd76-4cc8-a186-5f3598ba36fe None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:06.161790 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4373e8-bd76-4cc8-a186-5f3598ba36fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:06.174400 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4373e8-bd76-4cc8-a186-5f3598ba36fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:06.175797 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4373e8-bd76-4cc8-a186-5f3598ba36fe None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:06.181329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:06.202597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4d8a4-294c-4a41-9c75-059a988a5530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:06.203647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4d8a4-294c-4a41-9c75-059a988a5530 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:06.204159 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4d8a4-294c-4a41-9c75-059a988a5530 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:06.204648 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4d8a4-294c-4a41-9c75-059a988a5530 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:06.205598 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4d8a4-294c-4a41-9c75-059a988a5530 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:06.206115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4d8a4-294c-4a41-9c75-059a988a5530 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:06.207753 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4d8a4-294c-4a41-9c75-059a988a5530 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:06.210932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-1d4373e8-bd76-4cc8-a186-5f3598ba36fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:06.210932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:06.210932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:06.210932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:06.210932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:06.210932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:06.210932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:06.210932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:06.210932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:06.210932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:06.210932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:06.210932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:06.210932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:06.210932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:06.210932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:06.211914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:06.212945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:06.212945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:06.212945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:06.212945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:06.212945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:06.212945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:06.212945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:06.212945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:06.212945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 360/711] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:06.220339 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4373e8-bd76-4cc8-a186-5f3598ba36fe None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:06.224308 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 352/712] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 19:51:06 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:06.248299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:06.385017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4373e8-bd76-4cc8-a186-5f3598ba36fe None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:06.431487 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c27cb-4f7a-41d1-8860-bb338c4dff53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:06.432118 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c27cb-4f7a-41d1-8860-bb338c4dff53 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:06.432350 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c27cb-4f7a-41d1-8860-bb338c4dff53 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:06.432586 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c27cb-4f7a-41d1-8860-bb338c4dff53 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:06.433391 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c27cb-4f7a-41d1-8860-bb338c4dff53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:06.433706 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c27cb-4f7a-41d1-8860-bb338c4dff53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:06.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c27cb-4f7a-41d1-8860-bb338c4dff53 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:06.448710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 361/713] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:06.454952 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4d8a4-294c-4a41-9c75-059a988a5530 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:06.489485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a03c2b-d7d8-475a-af62-b50a6d83178a None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:06.489827 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a03c2b-d7d8-475a-af62-b50a6d83178a None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:06.490059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a03c2b-d7d8-475a-af62-b50a6d83178a None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:06.491744 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 362/714] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:06.500334 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b4a6b-1686-421b-9500-850c9eaaa483 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:06.500626 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b4a6b-1686-421b-9500-850c9eaaa483 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:06.500905 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b4a6b-1686-421b-9500-850c9eaaa483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:06.538491 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-405b4a6b-1686-421b-9500-850c9eaaa483 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:06.539701 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-405b4a6b-1686-421b-9500-850c9eaaa483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:06.565799 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4d8a4-294c-4a41-9c75-059a988a5530 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:06.609604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9cb04-53f0-4fb4-bc68-04af1f28245f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:06.610290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9cb04-53f0-4fb4-bc68-04af1f28245f None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:06.610577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9cb04-53f0-4fb4-bc68-04af1f28245f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:06.610846 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9cb04-53f0-4fb4-bc68-04af1f28245f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:06.611694 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9cb04-53f0-4fb4-bc68-04af1f28245f None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=0da7f490d9604107b974e9e461ebc38b) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:06.612005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9cb04-53f0-4fb4-bc68-04af1f28245f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:06.613453 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9cb04-53f0-4fb4-bc68-04af1f28245f None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:06.632294 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 353/715] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:06 2022] GET /identity/v3/role_assignments?user.id=0da7f490d9604107b974e9e461ebc38b&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:06.654979 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:06.670309 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 363/716] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:51:06.707214 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9cb04-53f0-4fb4-bc68-04af1f28245f None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:06.781561 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:06.856099 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9cb04-53f0-4fb4-bc68-04af1f28245f None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:06.901175 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0527e-679d-4196-a1fb-1949604f26a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:06.902017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0527e-679d-4196-a1fb-1949604f26a6 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:06.902317 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0527e-679d-4196-a1fb-1949604f26a6 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:06.902594 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0527e-679d-4196-a1fb-1949604f26a6 None admin] PATH_INFO: `/v3/services/network` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:06.903279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0527e-679d-4196-a1fb-1949604f26a6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:06.904697 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0527e-679d-4196-a1fb-1949604f26a6 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-97f0527e-679d-4196-a1fb-1949604f26a6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:06.922771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:06.923959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:06.925559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:06.925559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:06.925559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:06.925559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:06.925559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:06.925559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:06.925559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:06.925559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:06.925559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:06.925559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:06.925559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:06.925559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:06.925559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:51:06.925559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:06.925559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 354/717] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:06 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:06.929926 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b4a6b-1686-421b-9500-850c9eaaa483 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:07.044468 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b4a6b-1686-421b-9500-850c9eaaa483 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:07.081105 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778efce-c6d2-4eaa-baa3-dd46f8ebd93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:07.081798 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778efce-c6d2-4eaa-baa3-dd46f8ebd93f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:07.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778efce-c6d2-4eaa-baa3-dd46f8ebd93f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:07.082374 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778efce-c6d2-4eaa-baa3-dd46f8ebd93f None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:07.083108 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778efce-c6d2-4eaa-baa3-dd46f8ebd93f None admin] RBAC: Adding query filter params (name=network) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:07.083412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778efce-c6d2-4eaa-baa3-dd46f8ebd93f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:07.084823 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778efce-c6d2-4eaa-baa3-dd46f8ebd93f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:07.090625 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 364/718] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:51:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:07.099418 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0527e-679d-4196-a1fb-1949604f26a6 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:07.235085 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0527e-679d-4196-a1fb-1949604f26a6 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:07.244595 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:07.271170 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711121d5-d3b7-4231-81db-69e1fe5f254d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:07.271846 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711121d5-d3b7-4231-81db-69e1fe5f254d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:07.272128 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711121d5-d3b7-4231-81db-69e1fe5f254d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:07.272403 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711121d5-d3b7-4231-81db-69e1fe5f254d None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:07.273119 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711121d5-d3b7-4231-81db-69e1fe5f254d None admin] RBAC: Adding query filter params (type=network) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:07.273428 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711121d5-d3b7-4231-81db-69e1fe5f254d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:07.274874 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711121d5-d3b7-4231-81db-69e1fe5f254d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:07.279779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 355/719] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:51:07 2022] GET /identity/v3/services?type=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:08.367863 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992403d2-78c4-45b0-b23a-19892d556931 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:08.368412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992403d2-78c4-45b0-b23a-19892d556931 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:08.368412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992403d2-78c4-45b0-b23a-19892d556931 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:08.370062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 365/720] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:08.376625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b8083-f1e9-402e-a9d3-ae45fb9c529d None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:08.377994 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b8083-f1e9-402e-a9d3-ae45fb9c529d None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:08.377994 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b8083-f1e9-402e-a9d3-ae45fb9c529d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:08.385929 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a73efd-c427-40f0-91af-ee76a25cb77c None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:08.386552 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a73efd-c427-40f0-91af-ee76a25cb77c None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:08.387043 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a73efd-c427-40f0-91af-ee76a25cb77c None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:08.388848 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 366/721] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:08.394908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df930c-aa65-4cb8-bfe6-b124a4926de6 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:08.395167 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df930c-aa65-4cb8-bfe6-b124a4926de6 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:08.395448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df930c-aa65-4cb8-bfe6-b124a4926de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:08.399822 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-403b8083-f1e9-402e-a9d3-ae45fb9c529d None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:08.401293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-403b8083-f1e9-402e-a9d3-ae45fb9c529d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:08.423500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-d9df930c-aa65-4cb8-bfe6-b124a4926de6 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:08.424708 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-d9df930c-aa65-4cb8-bfe6-b124a4926de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:08.491647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 356/722] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:51:08.516876 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b8083-f1e9-402e-a9d3-ae45fb9c529d None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:08.540945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:08.555997 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 367/723] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:51:08.583401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df930c-aa65-4cb8-bfe6-b124a4926de6 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:08.716570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b8083-f1e9-402e-a9d3-ae45fb9c529d None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:08.720306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:08.755290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df930c-aa65-4cb8-bfe6-b124a4926de6 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:08.795263 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebd96b-2537-4736-bdb4-481108c82512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:08.797241 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebd96b-2537-4736-bdb4-481108c82512 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:08.797744 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebd96b-2537-4736-bdb4-481108c82512 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:08.798180 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebd96b-2537-4736-bdb4-481108c82512 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:08.802543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad944293-2412-49b5-9596-5530b69c44b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:08.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad944293-2412-49b5-9596-5530b69c44b4 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:08.804128 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad944293-2412-49b5-9596-5530b69c44b4 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:08.804579 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad944293-2412-49b5-9596-5530b69c44b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:08.830351 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebd96b-2537-4736-bdb4-481108c82512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-eeebd96b-2537-4736-bdb4-481108c82512 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:08.838576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:08.839857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:08.841011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:08.841011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:08.841011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:08.841011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:08.841011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:08.841011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:08.841011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:08.841011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:08.841011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:08.841011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:08.841011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:51:08.841011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:08.843750 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad944293-2412-49b5-9596-5530b69c44b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-ad944293-2412-49b5-9596-5530b69c44b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:08.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:08.853330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:08.854518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:08.854518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:08.854518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:08.854518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:08.854518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:08.854518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:08.854518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:08.854518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:08.854518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:08.854518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:08.854518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:08.854518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:08.856963 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 368/724] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:51:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:08.862991 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 357/725] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:51:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:08.866046 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebd96b-2537-4736-bdb4-481108c82512 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:08.870041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad944293-2412-49b5-9596-5530b69c44b4 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:08.927768 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:08.985165 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad944293-2412-49b5-9596-5530b69c44b4 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:09.022086 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323832c0-c003-485c-a07b-4fb546452305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:09.023237 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323832c0-c003-485c-a07b-4fb546452305 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:09.023754 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323832c0-c003-485c-a07b-4fb546452305 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:09.024257 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323832c0-c003-485c-a07b-4fb546452305 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:09.024379 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebd96b-2537-4736-bdb4-481108c82512 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:09.025369 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323832c0-c003-485c-a07b-4fb546452305 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:09.025900 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323832c0-c003-485c-a07b-4fb546452305 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:09.027688 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323832c0-c003-485c-a07b-4fb546452305 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:09.042519 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 358/726] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:51:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:09.044456 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b2c62-27e3-474f-ae45-ba5e763f049f None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:09.045027 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b2c62-27e3-474f-ae45-ba5e763f049f None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:09.045580 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b2c62-27e3-474f-ae45-ba5e763f049f None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:09.047693 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 359/727] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:09.053575 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7efa8-8b90-4470-9475-f7b0269a6546 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:09.054253 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7efa8-8b90-4470-9475-f7b0269a6546 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:09.054772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7efa8-8b90-4470-9475-f7b0269a6546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:09.068361 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0870493-7703-4fbb-a2fe-f4b90a3bc841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:09.069043 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0870493-7703-4fbb-a2fe-f4b90a3bc841 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:09.069322 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0870493-7703-4fbb-a2fe-f4b90a3bc841 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:09.069592 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0870493-7703-4fbb-a2fe-f4b90a3bc841 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:09.070380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0870493-7703-4fbb-a2fe-f4b90a3bc841 None admin] RBAC: Adding query filter params (name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:09.070686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0870493-7703-4fbb-a2fe-f4b90a3bc841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:09.072159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0870493-7703-4fbb-a2fe-f4b90a3bc841 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:09.077953 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-b4a7efa8-8b90-4470-9475-f7b0269a6546 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:09.079644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-b4a7efa8-8b90-4470-9475-f7b0269a6546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:09.081405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 369/728] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:51:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:09.082285 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0870493-7703-4fbb-a2fe-f4b90a3bc841 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:09.101144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:09.159351 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 360/729] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:51:09.160743 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7efa8-8b90-4470-9475-f7b0269a6546 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:09.214915 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0870493-7703-4fbb-a2fe-f4b90a3bc841 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:09.235359 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:09.255264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be88a0-647b-490b-a116-f8c3ec7990c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:09.256066 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be88a0-647b-490b-a116-f8c3ec7990c7 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:09.256346 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be88a0-647b-490b-a116-f8c3ec7990c7 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:09.256622 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be88a0-647b-490b-a116-f8c3ec7990c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:09.268216 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be88a0-647b-490b-a116-f8c3ec7990c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:09.269890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be88a0-647b-490b-a116-f8c3ec7990c7 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:09.279385 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7efa8-8b90-4470-9475-f7b0269a6546 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:09.287674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-d3be88a0-647b-490b-a116-f8c3ec7990c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:09.287674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:09.287674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:09.287674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:09.287674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:09.287674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:09.287674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:09.287674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:09.287674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:09.287674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:09.287674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:09.287674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:09.287674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:09.287674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:09.287674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:09.288678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:09.289689 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:09.289689 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:09.289689 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:09.289689 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:09.293575 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 370/730] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:09.294627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be88a0-647b-490b-a116-f8c3ec7990c7 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:09.321289 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602eb65f-6a8e-4790-b08e-679a86a20906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:09.322523 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eb65f-6a8e-4790-b08e-679a86a20906 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:09.323069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eb65f-6a8e-4790-b08e-679a86a20906 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:09.323860 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eb65f-6a8e-4790-b08e-679a86a20906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:09.335634 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602eb65f-6a8e-4790-b08e-679a86a20906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:09.337469 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602eb65f-6a8e-4790-b08e-679a86a20906 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:09.357421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-602eb65f-6a8e-4790-b08e-679a86a20906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:09.357421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:09.357421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:09.357421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:09.357421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:09.357421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:09.357421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:09.357421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:09.357421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:09.357421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:09.357421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:09.357421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:09.357421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:09.357421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:09.357421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:09.358636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:09.359629 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:09.359629 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:09.359629 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:09.359629 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:09.362162 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 361/731] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:09.363585 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602eb65f-6a8e-4790-b08e-679a86a20906 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:09.373629 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:09.404510 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be88a0-647b-490b-a116-f8c3ec7990c7 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:09.441603 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746104ea-1268-4300-8922-e965e15ab2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:09.442587 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746104ea-1268-4300-8922-e965e15ab2f9 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:09.443073 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746104ea-1268-4300-8922-e965e15ab2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:09.443494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746104ea-1268-4300-8922-e965e15ab2f9 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:09.444487 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746104ea-1268-4300-8922-e965e15ab2f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:09.446128 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746104ea-1268-4300-8922-e965e15ab2f9 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:09.470716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 371/732] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:51:09 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:51:09.472104 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746104ea-1268-4300-8922-e965e15ab2f9 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:09.472869 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602eb65f-6a8e-4790-b08e-679a86a20906 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:09.518062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:09.529750 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24025cc-9e75-4492-a4e3-6dd1ec9e9574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:09.531642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24025cc-9e75-4492-a4e3-6dd1ec9e9574 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:09.532198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24025cc-9e75-4492-a4e3-6dd1ec9e9574 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:09.532692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24025cc-9e75-4492-a4e3-6dd1ec9e9574 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:09.533836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24025cc-9e75-4492-a4e3-6dd1ec9e9574 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:09.534541 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24025cc-9e75-4492-a4e3-6dd1ec9e9574 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:09.536318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24025cc-9e75-4492-a4e3-6dd1ec9e9574 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:09.551804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 362/733] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:09.553804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893f635-3b8f-4a0d-b51e-54124423148b None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:09.554391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893f635-3b8f-4a0d-b51e-54124423148b None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:09.554933 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893f635-3b8f-4a0d-b51e-54124423148b None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:09.557213 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 363/734] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:09.558425 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893f635-3b8f-4a0d-b51e-54124423148b None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:09.633652 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746104ea-1268-4300-8922-e965e15ab2f9 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:09.640380 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:09.682063 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad1291-e71d-4b99-bb32-80e603fe6e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:09.683666 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad1291-e71d-4b99-bb32-80e603fe6e9c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:09.683953 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad1291-e71d-4b99-bb32-80e603fe6e9c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:09.684231 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad1291-e71d-4b99-bb32-80e603fe6e9c None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:09.684978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad1291-e71d-4b99-bb32-80e603fe6e9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:09.685288 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad1291-e71d-4b99-bb32-80e603fe6e9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:09.686854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad1291-e71d-4b99-bb32-80e603fe6e9c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:09.689228 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893f635-3b8f-4a0d-b51e-54124423148b None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:09.700269 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 372/735] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:09.701683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c521493-c8e2-46b4-8947-a1dee2f164fd None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:09.702123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c521493-c8e2-46b4-8947-a1dee2f164fd None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:09.702513 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c521493-c8e2-46b4-8947-a1dee2f164fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:09.729073 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-3c521493-c8e2-46b4-8947-a1dee2f164fd None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:09.729472 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee1bbc-8a44-4714-ae66-a74fee128584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:09.730296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-3c521493-c8e2-46b4-8947-a1dee2f164fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:09.730420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1bbc-8a44-4714-ae66-a74fee128584 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:09.730659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1bbc-8a44-4714-ae66-a74fee128584 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:09.730944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1bbc-8a44-4714-ae66-a74fee128584 None admin] PATH_INFO: `/v3/users/swift` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:09.739667 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee1bbc-8a44-4714-ae66-a74fee128584 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:09.741681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee1bbc-8a44-4714-ae66-a74fee128584 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-ecee1bbc-8a44-4714-ae66-a74fee128584 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:09.757178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:09.758384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:09.759460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:09.759460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:09.759460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:09.759460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:09.759460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:09.759460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:09.759460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:09.759460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:09.759460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:51:09.759460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:09.759460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 364/736] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:51:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:09.760166 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee1bbc-8a44-4714-ae66-a74fee128584 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:09.821566 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 373/737] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:09 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:51:09.823038 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c521493-c8e2-46b4-8947-a1dee2f164fd None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:09.848080 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:09.878414 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee1bbc-8a44-4714-ae66-a74fee128584 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:09.903203 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:09.916384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64f570-8370-4862-b2fa-f0a95c2b9678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:09.917371 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64f570-8370-4862-b2fa-f0a95c2b9678 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:09.917754 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64f570-8370-4862-b2fa-f0a95c2b9678 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:09.918153 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64f570-8370-4862-b2fa-f0a95c2b9678 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:09.927256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e64f570-8370-4862-b2fa-f0a95c2b9678 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:09.928928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e64f570-8370-4862-b2fa-f0a95c2b9678 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-7e64f570-8370-4862-b2fa-f0a95c2b9678 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:09.946185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:09.947320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:09.948376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:09.948376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:09.948376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:09.948376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:09.948376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:09.948376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:09.948376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:09.948376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:09.948376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:51:09.948376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:09.952311 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 365/738] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:51:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:09.953282 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64f570-8370-4862-b2fa-f0a95c2b9678 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:09.961777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c521493-c8e2-46b4-8947-a1dee2f164fd None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:10.009388 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d20177-484f-45e5-b912-4849bd9f44a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:10.010618 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d20177-484f-45e5-b912-4849bd9f44a7 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:10.010903 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d20177-484f-45e5-b912-4849bd9f44a7 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:10.011469 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d20177-484f-45e5-b912-4849bd9f44a7 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:10.012575 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d20177-484f-45e5-b912-4849bd9f44a7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:10.012895 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d20177-484f-45e5-b912-4849bd9f44a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:10.015128 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d20177-484f-45e5-b912-4849bd9f44a7 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:10.040101 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 374/739] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 19:51:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:10.041558 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d20177-484f-45e5-b912-4849bd9f44a7 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:10.060037 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64f570-8370-4862-b2fa-f0a95c2b9678 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:10.098378 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d20b35-4a5b-47c9-84b5-d735966b6313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:10.099184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d20b35-4a5b-47c9-84b5-d735966b6313 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:10.099506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d20b35-4a5b-47c9-84b5-d735966b6313 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:10.099792 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d20b35-4a5b-47c9-84b5-d735966b6313 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:10.111066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d20b35-4a5b-47c9-84b5-d735966b6313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-19d20b35-4a5b-47c9-84b5-d735966b6313 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:10.119223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:10.121476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:10.123730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:10.123730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:10.123730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:10.123730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:10.123730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:10.123730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:10.123730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:10.123730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:10.123730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:10.123730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:10.123730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:10.123730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:10.134289 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 366/740] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:51:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:10.134289 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d20b35-4a5b-47c9-84b5-d735966b6313 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:10.138549 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:10.166901 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d20177-484f-45e5-b912-4849bd9f44a7 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:10.174451 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:10.204037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e954fc-55c8-4131-b3fa-931c9c2d57b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:10.205007 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e954fc-55c8-4131-b3fa-931c9c2d57b6 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:10.205531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e954fc-55c8-4131-b3fa-931c9c2d57b6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:10.205803 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e954fc-55c8-4131-b3fa-931c9c2d57b6 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:10.206700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e954fc-55c8-4131-b3fa-931c9c2d57b6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:10.207011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e954fc-55c8-4131-b3fa-931c9c2d57b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:10.208449 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e954fc-55c8-4131-b3fa-931c9c2d57b6 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:10.232158 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 375/741] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 19:51:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:10.232158 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e954fc-55c8-4131-b3fa-931c9c2d57b6 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:10.242082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d20b35-4a5b-47c9-84b5-d735966b6313 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:10.275589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c386e54-dad4-42bd-b1fb-a77efbda28dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:10.276250 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c386e54-dad4-42bd-b1fb-a77efbda28dc None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:10.276537 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c386e54-dad4-42bd-b1fb-a77efbda28dc None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:10.276815 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c386e54-dad4-42bd-b1fb-a77efbda28dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:10.335417 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e954fc-55c8-4131-b3fa-931c9c2d57b6 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:10.353849 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c386e54-dad4-42bd-b1fb-a77efbda28dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:10.355333 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c386e54-dad4-42bd-b1fb-a77efbda28dc None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:10.373453 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a337bd-6d36-4245-89d8-df3b709d56fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:10.374234 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a337bd-6d36-4245-89d8-df3b709d56fa None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:10.374576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a337bd-6d36-4245-89d8-df3b709d56fa None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:10.374798 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a337bd-6d36-4245-89d8-df3b709d56fa None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:10.375553 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a337bd-6d36-4245-89d8-df3b709d56fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:10.375861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a337bd-6d36-4245-89d8-df3b709d56fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:10.377241 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a337bd-6d36-4245-89d8-df3b709d56fa None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:10.378686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:10.401224 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 376/742] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:51:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:10.401224 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e712ac-fdf1-4cf1-b333-176c6fbe683e None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:10.401224 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e712ac-fdf1-4cf1-b333-176c6fbe683e None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:10.401224 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e712ac-fdf1-4cf1-b333-176c6fbe683e None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:10.401224 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 377/743] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:10.401224 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e712ac-fdf1-4cf1-b333-176c6fbe683e None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:10.408373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:10.474667 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 367/744] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:10 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:51:10.475393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c386e54-dad4-42bd-b1fb-a77efbda28dc None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:10.511559 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e712ac-fdf1-4cf1-b333-176c6fbe683e None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:10.546113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf13a70-3628-4750-b159-d7b58ebea033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:10.546814 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf13a70-3628-4750-b159-d7b58ebea033 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:10.547100 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf13a70-3628-4750-b159-d7b58ebea033 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:10.547382 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf13a70-3628-4750-b159-d7b58ebea033 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:10.581042 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c386e54-dad4-42bd-b1fb-a77efbda28dc None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:10.610042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:10.617059 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb1dfb-da86-4317-94fc-d936388f6c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:10.617862 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb1dfb-da86-4317-94fc-d936388f6c77 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:10.618148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb1dfb-da86-4317-94fc-d936388f6c77 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:10.618366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf13a70-3628-4750-b159-d7b58ebea033 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:10.618482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb1dfb-da86-4317-94fc-d936388f6c77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:10.619786 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf13a70-3628-4750-b159-d7b58ebea033 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:10.629094 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb1dfb-da86-4317-94fc-d936388f6c77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:10.630562 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb1dfb-da86-4317-94fc-d936388f6c77 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:10.632656 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:10.650379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-8abb1dfb-da86-4317-94fc-d936388f6c77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:10.650379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:10.650379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:10.650379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:10.650379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:10.650379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:10.650379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:10.650379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:10.650379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:10.650379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:10.650379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:10.650379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:10.650379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:10.650379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:10.650379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:10.651394 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:10.652404 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:10.652404 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:10.652404 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:10.652404 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:10.652404 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 368/745] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:10.653147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d02405-6e46-4937-a793-0ee932d54f76 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:10.653147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d02405-6e46-4937-a793-0ee932d54f76 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:10.653391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d02405-6e46-4937-a793-0ee932d54f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:10.670119 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-97d02405-6e46-4937-a793-0ee932d54f76 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:10.671520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-97d02405-6e46-4937-a793-0ee932d54f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:10.741447 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 378/746] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:10 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:51:10.741447 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf13a70-3628-4750-b159-d7b58ebea033 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:10.753911 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 369/747] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:10 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:51:10.759221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d02405-6e46-4937-a793-0ee932d54f76 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:10.840943 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:10.848888 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf13a70-3628-4750-b159-d7b58ebea033 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:10.868501 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d02405-6e46-4937-a793-0ee932d54f76 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:10.886276 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67048a2b-7441-4d4c-bafa-f752ae998d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:10.887079 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67048a2b-7441-4d4c-bafa-f752ae998d23 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:10.887356 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67048a2b-7441-4d4c-bafa-f752ae998d23 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:10.887627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67048a2b-7441-4d4c-bafa-f752ae998d23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:10.893253 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:10.898767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67048a2b-7441-4d4c-bafa-f752ae998d23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:10.900765 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67048a2b-7441-4d4c-bafa-f752ae998d23 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:10.904840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190f5a2-a912-48ac-89e2-d1e4e7fabe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:10.905514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190f5a2-a912-48ac-89e2-d1e4e7fabe44 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:10.905796 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190f5a2-a912-48ac-89e2-d1e4e7fabe44 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:10.906071 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190f5a2-a912-48ac-89e2-d1e4e7fabe44 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:10.906835 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190f5a2-a912-48ac-89e2-d1e4e7fabe44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:10.907149 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190f5a2-a912-48ac-89e2-d1e4e7fabe44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:10.908542 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190f5a2-a912-48ac-89e2-d1e4e7fabe44 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:10.918290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-67048a2b-7441-4d4c-bafa-f752ae998d23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:10.918290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:10.918290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:10.918290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:10.918290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:10.918290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:10.918290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:10.918290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:10.918290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:10.918290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:10.918290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:10.918290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:10.918290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:10.918290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:10.918290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:10.919264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:10.920285 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:10.920285 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:10.920285 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:10.920285 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:10.920285 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 379/748] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:10.920758 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67048a2b-7441-4d4c-bafa-f752ae998d23 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:10.925139 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 370/749] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:10.925817 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190f5a2-a912-48ac-89e2-d1e4e7fabe44 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:11.030146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190f5a2-a912-48ac-89e2-d1e4e7fabe44 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:11.030545 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67048a2b-7441-4d4c-bafa-f752ae998d23 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:11.065337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873251c-d1d5-4c02-b619-17ba54002e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:11.065781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88a482-7af3-496d-8f7a-b12d13ca78f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:11.066079 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873251c-d1d5-4c02-b619-17ba54002e71 None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:11.066420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873251c-d1d5-4c02-b619-17ba54002e71 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:11.066703 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873251c-d1d5-4c02-b619-17ba54002e71 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:11.066826 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88a482-7af3-496d-8f7a-b12d13ca78f8 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:11.067032 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88a482-7af3-496d-8f7a-b12d13ca78f8 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:11.067301 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88a482-7af3-496d-8f7a-b12d13ca78f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:11.067477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873251c-d1d5-4c02-b619-17ba54002e71 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:11.068817 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873251c-d1d5-4c02-b619-17ba54002e71 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:11.078272 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88a482-7af3-496d-8f7a-b12d13ca78f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:11.079880 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88a482-7af3-496d-8f7a-b12d13ca78f8 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:11.094290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 371/750] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:51:10 2022] POST /identity/v3/services => generated 240 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:51:11.094290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873251c-d1d5-4c02-b619-17ba54002e71 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:11.098763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-0f88a482-7af3-496d-8f7a-b12d13ca78f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:11.098763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:11.098763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:11.098763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:11.098763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:11.098763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:11.098763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:11.098763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:11.098763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:11.098763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:11.098763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:11.098763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:11.098763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:11.098763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:11.098763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:11.099832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:11.100906 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:11.100906 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:11.100906 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:11.100906 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:11.107867 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 380/751] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:11.107867 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:11.107867 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88a482-7af3-496d-8f7a-b12d13ca78f8 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:11.171090 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:11.256926 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88a482-7af3-496d-8f7a-b12d13ca78f8 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:11.257144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873251c-d1d5-4c02-b619-17ba54002e71 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:11.297329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e75a9d-d115-4e60-9058-02359637ca5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:11.298442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e75a9d-d115-4e60-9058-02359637ca5c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:11.298962 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e75a9d-d115-4e60-9058-02359637ca5c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:11.299463 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e75a9d-d115-4e60-9058-02359637ca5c None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:11.300473 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e75a9d-d115-4e60-9058-02359637ca5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:11.300709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242f82b-a66f-423d-9c7d-74ea57e58e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:11.301204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e75a9d-d115-4e60-9058-02359637ca5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:11.301524 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f82b-a66f-423d-9c7d-74ea57e58e2f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:11.301811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f82b-a66f-423d-9c7d-74ea57e58e2f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:11.302097 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f82b-a66f-423d-9c7d-74ea57e58e2f None admin] PATH_INFO: `/v3/users/nova` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:11.302863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e75a9d-d115-4e60-9058-02359637ca5c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:11.311559 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242f82b-a66f-423d-9c7d-74ea57e58e2f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:11.314478 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242f82b-a66f-423d-9c7d-74ea57e58e2f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:11.317362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 372/752] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:11.318551 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e75a9d-d115-4e60-9058-02359637ca5c None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-7242f82b-a66f-423d-9c7d-74ea57e58e2f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:11.334703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:11.335925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:11.337228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:11.337228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:11.337228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:11.337228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:11.337228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:11.337228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:11.337228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:11.337228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:11.337228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:51:11.337228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:11.337228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 381/753] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:51:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:11.337228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242f82b-a66f-423d-9c7d-74ea57e58e2f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:11.426754 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:11.445766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:11.448902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e75a9d-d115-4e60-9058-02359637ca5c None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:11.461351 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242f82b-a66f-423d-9c7d-74ea57e58e2f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:11.487132 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5ee9b-c0bc-4c4e-87ea-8b7620d3a373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:11.487964 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5ee9b-c0bc-4c4e-87ea-8b7620d3a373 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:11.488347 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5ee9b-c0bc-4c4e-87ea-8b7620d3a373 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:11.488750 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5ee9b-c0bc-4c4e-87ea-8b7620d3a373 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:11.489597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5ee9b-c0bc-4c4e-87ea-8b7620d3a373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:11.490009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5ee9b-c0bc-4c4e-87ea-8b7620d3a373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:11.491534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5ee9b-c0bc-4c4e-87ea-8b7620d3a373 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:11.500012 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d55bf-7b83-45b7-b164-5aaf72e799f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:11.501179 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d55bf-7b83-45b7-b164-5aaf72e799f7 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:11.501674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d55bf-7b83-45b7-b164-5aaf72e799f7 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:11.502160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d55bf-7b83-45b7-b164-5aaf72e799f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:11.509484 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 373/754] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:11.510248 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d55bf-7b83-45b7-b164-5aaf72e799f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:11.510601 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5ee9b-c0bc-4c4e-87ea-8b7620d3a373 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:11.512162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d55bf-7b83-45b7-b164-5aaf72e799f7 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:11.528667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-707d55bf-7b83-45b7-b164-5aaf72e799f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:11.528667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:11.528667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:11.528667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:11.528667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:11.528667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:11.528667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:11.528667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:11.528667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:11.528667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:11.528667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:11.528667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:11.528667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:11.528667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:11.528667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:11.529751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:11.531448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:11.531448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:11.531448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:11.531448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:11.531448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:11.531448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:11.531448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:11.531448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:11.538224 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 382/755] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:11.539528 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d55bf-7b83-45b7-b164-5aaf72e799f7 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:11.618545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5ee9b-c0bc-4c4e-87ea-8b7620d3a373 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:11.654896 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d55bf-7b83-45b7-b164-5aaf72e799f7 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:11.656558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622d211-ea7b-42db-b9d9-f29f7d7a4ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:11.657354 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622d211-ea7b-42db-b9d9-f29f7d7a4ebc None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:11.657738 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622d211-ea7b-42db-b9d9-f29f7d7a4ebc None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:11.658156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622d211-ea7b-42db-b9d9-f29f7d7a4ebc None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:11.659132 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622d211-ea7b-42db-b9d9-f29f7d7a4ebc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:11.659546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622d211-ea7b-42db-b9d9-f29f7d7a4ebc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:11.661097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622d211-ea7b-42db-b9d9-f29f7d7a4ebc None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:11.679150 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:11.682058 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 374/756] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:51:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:11.683095 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622d211-ea7b-42db-b9d9-f29f7d7a4ebc None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:11.696200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32702366-cc0e-4fd2-9a89-9ae8f50f5e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:11.697363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32702366-cc0e-4fd2-9a89-9ae8f50f5e71 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:11.697814 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32702366-cc0e-4fd2-9a89-9ae8f50f5e71 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:11.698371 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32702366-cc0e-4fd2-9a89-9ae8f50f5e71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:11.713335 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32702366-cc0e-4fd2-9a89-9ae8f50f5e71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:11.716847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32702366-cc0e-4fd2-9a89-9ae8f50f5e71 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:11.719584 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:11.735964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-32702366-cc0e-4fd2-9a89-9ae8f50f5e71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:11.735964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:11.735964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:11.735964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:11.735964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:11.735964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:11.735964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:11.735964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:11.735964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:11.735964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:11.735964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:11.735964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:11.735964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:11.735964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:11.735964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:11.737054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:11.738087 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:11.738087 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:11.738087 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:11.738087 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:11.738087 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:11.738087 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:11.738087 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:11.738087 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:11.740796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 383/757] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:11.741910 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32702366-cc0e-4fd2-9a89-9ae8f50f5e71 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:11.806417 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622d211-ea7b-42db-b9d9-f29f7d7a4ebc None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:11.842019 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e0f31-07e2-4c39-8971-209a4f138757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:11.842748 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e0f31-07e2-4c39-8971-209a4f138757 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:11.843030 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e0f31-07e2-4c39-8971-209a4f138757 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:11.843448 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e0f31-07e2-4c39-8971-209a4f138757 None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:11.844529 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e0f31-07e2-4c39-8971-209a4f138757 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:11.844841 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e0f31-07e2-4c39-8971-209a4f138757 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:11.846294 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e0f31-07e2-4c39-8971-209a4f138757 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:11.847657 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32702366-cc0e-4fd2-9a89-9ae8f50f5e71 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:11.866052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 375/758] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:11.866052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e0f31-07e2-4c39-8971-209a4f138757 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:11.890606 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9c7e1-a4fd-4d2d-a21a-5ccd37125582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:11.891416 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9c7e1-a4fd-4d2d-a21a-5ccd37125582 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:11.891695 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9c7e1-a4fd-4d2d-a21a-5ccd37125582 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:11.891970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9c7e1-a4fd-4d2d-a21a-5ccd37125582 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:11.903325 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9c7e1-a4fd-4d2d-a21a-5ccd37125582 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:11.904802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9c7e1-a4fd-4d2d-a21a-5ccd37125582 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:11.922795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-22d9c7e1-a4fd-4d2d-a21a-5ccd37125582 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:11.922795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:11.922795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:11.922795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:11.922795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:11.922795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:11.922795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:11.922795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:11.922795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:11.922795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:11.922795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:11.922795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:11.922795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:11.922795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:11.922795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:11.923873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:11.924944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:11.924944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:11.924944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:11.924944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:11.924944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 384/759] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:11.925611 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095712c4-abb2-4c99-bb21-f5ab78ee5f91 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:11.925611 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095712c4-abb2-4c99-bb21-f5ab78ee5f91 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:11.925761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095712c4-abb2-4c99-bb21-f5ab78ee5f91 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:11.930389 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 385/760] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:11.930389 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095712c4-abb2-4c99-bb21-f5ab78ee5f91 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:11.952377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:11.963170 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:11.979044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e0f31-07e2-4c39-8971-209a4f138757 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:12.013152 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600f581-7f3b-4232-8b14-97ef066d2ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:12.013825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600f581-7f3b-4232-8b14-97ef066d2ebf None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:12.014101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600f581-7f3b-4232-8b14-97ef066d2ebf None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:12.014491 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600f581-7f3b-4232-8b14-97ef066d2ebf None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:12.015289 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600f581-7f3b-4232-8b14-97ef066d2ebf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:12.015598 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600f581-7f3b-4232-8b14-97ef066d2ebf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:12.017034 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600f581-7f3b-4232-8b14-97ef066d2ebf None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:12.034606 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 376/761] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:12.035818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600f581-7f3b-4232-8b14-97ef066d2ebf None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:12.040778 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095712c4-abb2-4c99-bb21-f5ab78ee5f91 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:12.076723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c6d56-62e4-4a50-9465-107d29efbc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:12.077706 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c6d56-62e4-4a50-9465-107d29efbc6e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:12.078134 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c6d56-62e4-4a50-9465-107d29efbc6e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:12.078610 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c6d56-62e4-4a50-9465-107d29efbc6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:12.079692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c6d56-62e4-4a50-9465-107d29efbc6e None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=acebbcf1f90d487aa21a8eb99102ce09) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:12.080156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c6d56-62e4-4a50-9465-107d29efbc6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:12.081856 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c6d56-62e4-4a50-9465-107d29efbc6e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:12.088039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 386/762] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:11 2022] GET /identity/v3/role_assignments?user.id=acebbcf1f90d487aa21a8eb99102ce09&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:12.089336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4e876-737d-4e12-a9a1-520baa6d8d20 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:12.089620 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4e876-737d-4e12-a9a1-520baa6d8d20 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:12.089891 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4e876-737d-4e12-a9a1-520baa6d8d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:12.114599 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-06e4e876-737d-4e12-a9a1-520baa6d8d20 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:12.115958 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-06e4e876-737d-4e12-a9a1-520baa6d8d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:12.153594 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600f581-7f3b-4232-8b14-97ef066d2ebf None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:12.203925 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff424963-bca7-4966-8d11-d9375ea46770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:12.204963 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff424963-bca7-4966-8d11-d9375ea46770 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:12.205452 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff424963-bca7-4966-8d11-d9375ea46770 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:12.205914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff424963-bca7-4966-8d11-d9375ea46770 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:12.206887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff424963-bca7-4966-8d11-d9375ea46770 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:12.207346 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 387/763] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:12.207599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff424963-bca7-4966-8d11-d9375ea46770 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:12.208369 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4e876-737d-4e12-a9a1-520baa6d8d20 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:12.209384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff424963-bca7-4966-8d11-d9375ea46770 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:12.218764 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:12.226323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 377/764] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:12.235751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff424963-bca7-4966-8d11-d9375ea46770 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:12.264003 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:12.348214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4e876-737d-4e12-a9a1-520baa6d8d20 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:12.351161 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff424963-bca7-4966-8d11-d9375ea46770 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:12.385267 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd64333-58f4-404a-944d-74e08b3997ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:12.386421 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd64333-58f4-404a-944d-74e08b3997ca None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:12.386899 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd64333-58f4-404a-944d-74e08b3997ca None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:12.387352 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd64333-58f4-404a-944d-74e08b3997ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:12.388378 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd64333-58f4-404a-944d-74e08b3997ca None admin] RBAC: Adding query filter params (role.id=3387bd4782b047ae8d880e386b0351f0, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=8683be51c17341f7b0e85d366a74869f) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:12.388881 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd64333-58f4-404a-944d-74e08b3997ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:12.390592 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd64333-58f4-404a-944d-74e08b3997ca None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:12.391292 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ae743-01f0-4ee9-bece-2dca6cf6eb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:12.392166 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ae743-01f0-4ee9-bece-2dca6cf6eb75 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:12.392515 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ae743-01f0-4ee9-bece-2dca6cf6eb75 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:12.392856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ae743-01f0-4ee9-bece-2dca6cf6eb75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:12.398171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 388/765] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:12 2022] GET /identity/v3/role_assignments?user.id=8683be51c17341f7b0e85d366a74869f&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=3387bd4782b047ae8d880e386b0351f0 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:12.399329 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd64333-58f4-404a-944d-74e08b3997ca None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:12.414555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ae743-01f0-4ee9-bece-2dca6cf6eb75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:12.416387 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ae743-01f0-4ee9-bece-2dca6cf6eb75 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:12.449543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-263ae743-01f0-4ee9-bece-2dca6cf6eb75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:12.449543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:12.449543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:12.449543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:12.449543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:12.449543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:12.449543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:12.449543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:12.449543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:12.449543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:12.449543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:12.449543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:12.449543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:12.449543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:12.449543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:12.450690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:12.451796 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:12.451796 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:12.451796 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:12.451796 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:12.451796 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:12.451796 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:12.451796 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:12.451796 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:12.463286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 378/766] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:12.464523 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ae743-01f0-4ee9-bece-2dca6cf6eb75 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:12.583392 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:12.598514 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:12.621492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd64333-58f4-404a-944d-74e08b3997ca None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:12.636905 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ae743-01f0-4ee9-bece-2dca6cf6eb75 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:12.676101 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8004f-0585-45e5-8b45-ff374016e048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:12.676934 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8004f-0585-45e5-8b45-ff374016e048 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:12.677218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8004f-0585-45e5-8b45-ff374016e048 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:12.677493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8004f-0585-45e5-8b45-ff374016e048 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:12.678167 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8004f-0585-45e5-8b45-ff374016e048 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:12.679599 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8004f-0585-45e5-8b45-ff374016e048 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:12.682448 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc02d28-7634-4c54-b427-256c8bc4a55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:12.683123 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc02d28-7634-4c54-b427-256c8bc4a55c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:12.683404 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc02d28-7634-4c54-b427-256c8bc4a55c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:12.683678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc02d28-7634-4c54-b427-256c8bc4a55c None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:12.684483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc02d28-7634-4c54-b427-256c8bc4a55c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:12.684795 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc02d28-7634-4c54-b427-256c8bc4a55c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:12.686377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc02d28-7634-4c54-b427-256c8bc4a55c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:12.703996 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 379/767] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-7eb8004f-0585-45e5-8b45-ff374016e048 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:12.707728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:12.709005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:12.710313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 389/768] 10.222.0.84 () {56 vars in 1129 bytes} [Thu May 19 19:51:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:12.711263 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc02d28-7634-4c54-b427-256c8bc4a55c None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:12.727318 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8004f-0585-45e5-8b45-ff374016e048 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:12.820881 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc02d28-7634-4c54-b427-256c8bc4a55c None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:12.846969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:12.861262 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbdd18-53f3-4366-ac1b-ea84be8ac844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:12.862177 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbdd18-53f3-4366-ac1b-ea84be8ac844 None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:12.862639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbdd18-53f3-4366-ac1b-ea84be8ac844 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:12.862932 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbdd18-53f3-4366-ac1b-ea84be8ac844 None admin] PATH_INFO: `/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/0da7f490d9604107b974e9e461ebc38b/roles/2cd6d3dd613c4824a26cacf53fb6c07f` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:12.865109 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbdd18-53f3-4366-ac1b-ea84be8ac844 None admin] RBAC: Authorizing `identity:create_grant(project_id=d638a29a793d41e7bfa686c0fcc29149, user_id=0da7f490d9604107b974e9e461ebc38b, role_id=2cd6d3dd613c4824a26cacf53fb6c07f)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:12.866611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbdd18-53f3-4366-ac1b-ea84be8ac844 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:12.895794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 380/769] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:51:12 2022] PUT /identity/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/0da7f490d9604107b974e9e461ebc38b/roles/2cd6d3dd613c4824a26cacf53fb6c07f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:51:12.966824 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8004f-0585-45e5-8b45-ff374016e048 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:13.015215 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9170e8b-ec43-49e2-b91c-d9d4ae104b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:13.015839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9170e8b-ec43-49e2-b91c-d9d4ae104b66 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:13.016123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9170e8b-ec43-49e2-b91c-d9d4ae104b66 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:13.016397 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9170e8b-ec43-49e2-b91c-d9d4ae104b66 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:13.017135 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9170e8b-ec43-49e2-b91c-d9d4ae104b66 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:13.017438 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9170e8b-ec43-49e2-b91c-d9d4ae104b66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:13.019102 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9170e8b-ec43-49e2-b91c-d9d4ae104b66 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:13.024822 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 390/770] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 19:51:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:13.031436 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbdd18-53f3-4366-ac1b-ea84be8ac844 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:13.158104 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbdd18-53f3-4366-ac1b-ea84be8ac844 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:13.195186 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2681a-c59f-4cbd-bfd1-f994ecf9f0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:13.195879 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2681a-c59f-4cbd-bfd1-f994ecf9f0a1 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:13.196158 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2681a-c59f-4cbd-bfd1-f994ecf9f0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:13.196430 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2681a-c59f-4cbd-bfd1-f994ecf9f0a1 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:13.197167 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2681a-c59f-4cbd-bfd1-f994ecf9f0a1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:13.197473 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2681a-c59f-4cbd-bfd1-f994ecf9f0a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:13.198906 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2681a-c59f-4cbd-bfd1-f994ecf9f0a1 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:13.205492 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 381/771] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 19:51:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:13.211472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9170e8b-ec43-49e2-b91c-d9d4ae104b66 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:13.233603 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:13.322711 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9170e8b-ec43-49e2-b91c-d9d4ae104b66 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:13.357235 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33b6cd-0648-4ee1-bb69-71240819c6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:13.358053 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33b6cd-0648-4ee1-bb69-71240819c6df None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:13.358370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33b6cd-0648-4ee1-bb69-71240819c6df None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:13.358645 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33b6cd-0648-4ee1-bb69-71240819c6df None admin] PATH_INFO: `/v3/endpoints` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:13.359446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33b6cd-0648-4ee1-bb69-71240819c6df None admin] RBAC: Adding query filter params (interface=public, service_id=73038104868d4aaa85a21ed9dcb45d2a, region_id=RegionOne) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:13.359753 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33b6cd-0648-4ee1-bb69-71240819c6df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:13.361125 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33b6cd-0648-4ee1-bb69-71240819c6df None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:13.369240 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 391/772] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:51:13 2022] GET /identity/v3/endpoints?service_id=73038104868d4aaa85a21ed9dcb45d2a&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:13.374938 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2681a-c59f-4cbd-bfd1-f994ecf9f0a1 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:13.426072 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:13.485674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2681a-c59f-4cbd-bfd1-f994ecf9f0a1 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:13.520841 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd54a93-872e-4233-8972-a606dfea3d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:13.521514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd54a93-872e-4233-8972-a606dfea3d80 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:13.521841 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd54a93-872e-4233-8972-a606dfea3d80 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:13.522118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd54a93-872e-4233-8972-a606dfea3d80 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:13.522875 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd54a93-872e-4233-8972-a606dfea3d80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:13.524252 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd54a93-872e-4233-8972-a606dfea3d80 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:13.530452 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 382/773] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:51:13 2022] GET /identity/v3/services => generated 772 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:13.745117 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa07bae-eeb7-4a99-b76e-3fb1899539f2 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:13.745661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa07bae-eeb7-4a99-b76e-3fb1899539f2 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:13.745661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa07bae-eeb7-4a99-b76e-3fb1899539f2 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:13.747371 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 392/774] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:13.753216 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3576744-d735-4a85-a2ea-0d0e0bf811c7 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:13.753684 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3576744-d735-4a85-a2ea-0d0e0bf811c7 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:13.753807 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3576744-d735-4a85-a2ea-0d0e0bf811c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:13.775995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-a3576744-d735-4a85-a2ea-0d0e0bf811c7 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:13.777213 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-a3576744-d735-4a85-a2ea-0d0e0bf811c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:13.849113 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 383/775] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:13.872626 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa07bae-eeb7-4a99-b76e-3fb1899539f2 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:13.981557 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:14.011973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa07bae-eeb7-4a99-b76e-3fb1899539f2 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:14.060260 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea20ab-507a-499b-aa57-865e90bdc71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:14.061268 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea20ab-507a-499b-aa57-865e90bdc71d None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:14.061392 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea20ab-507a-499b-aa57-865e90bdc71d None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:14.061685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea20ab-507a-499b-aa57-865e90bdc71d None admin] PATH_INFO: `/v3/services/network` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:14.062370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea20ab-507a-499b-aa57-865e90bdc71d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:14.063797 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea20ab-507a-499b-aa57-865e90bdc71d None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-edea20ab-507a-499b-aa57-865e90bdc71d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:14.079722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:14.080851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:14.081804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:14.081804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:14.081804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:14.081804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:14.081804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:14.081804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:14.081804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:14.081804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:14.081804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:14.081804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:14.081804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:14.081804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:14.081804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:51:14.081804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:14.081804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 393/776] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:13 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:14.086915 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3576744-d735-4a85-a2ea-0d0e0bf811c7 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:14.155387 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:14.254987 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3576744-d735-4a85-a2ea-0d0e0bf811c7 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:14.316872 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899fb91-1bec-4133-a363-4bebeb6b78f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:14.317573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899fb91-1bec-4133-a363-4bebeb6b78f0 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:14.317854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899fb91-1bec-4133-a363-4bebeb6b78f0 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:14.318127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899fb91-1bec-4133-a363-4bebeb6b78f0 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:14.318890 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7899fb91-1bec-4133-a363-4bebeb6b78f0 None admin] RBAC: Adding query filter params (name=network) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:14.319196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7899fb91-1bec-4133-a363-4bebeb6b78f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:14.321168 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7899fb91-1bec-4133-a363-4bebeb6b78f0 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:14.331234 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 384/777] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:51:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:14.336378 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea20ab-507a-499b-aa57-865e90bdc71d None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:14.473329 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea20ab-507a-499b-aa57-865e90bdc71d None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:14.514145 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd814913-d3d5-4dbc-95dc-78198baab3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:14.515902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd814913-d3d5-4dbc-95dc-78198baab3c7 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:14.516185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd814913-d3d5-4dbc-95dc-78198baab3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:14.516460 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd814913-d3d5-4dbc-95dc-78198baab3c7 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:14.517184 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd814913-d3d5-4dbc-95dc-78198baab3c7 None admin] RBAC: Adding query filter params (type=network) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:14.517488 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd814913-d3d5-4dbc-95dc-78198baab3c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:14.518892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd814913-d3d5-4dbc-95dc-78198baab3c7 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:14.526208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 394/778] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:51:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:14.531760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899fb91-1bec-4133-a363-4bebeb6b78f0 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:14.667426 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:14.671009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899fb91-1bec-4133-a363-4bebeb6b78f0 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:14.714978 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ab7ab-bfd9-4156-a829-da72b296e564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:14.716602 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ab7ab-bfd9-4156-a829-da72b296e564 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:14.717238 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ab7ab-bfd9-4156-a829-da72b296e564 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:14.717752 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ab7ab-bfd9-4156-a829-da72b296e564 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:14.718985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ab7ab-bfd9-4156-a829-da72b296e564 None admin] RBAC: Adding query filter params (interface=public, service_id=e4ee67172673452789bc72df63c1f2da, region_id=RegionOne) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:14.719521 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ab7ab-bfd9-4156-a829-da72b296e564 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:14.721135 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ab7ab-bfd9-4156-a829-da72b296e564 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:14.737184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 385/779] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:51:14 2022] GET /identity/v3/endpoints?service_id=e4ee67172673452789bc72df63c1f2da&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:14.743253 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd814913-d3d5-4dbc-95dc-78198baab3c7 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:14.766747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:14.773627 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142e9d6-2925-4a65-83a2-b095fcf0e17e None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:14.774526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142e9d6-2925-4a65-83a2-b095fcf0e17e None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:14.775040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142e9d6-2925-4a65-83a2-b095fcf0e17e None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:14.776917 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 386/780] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:14.782420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63745eab-cfd9-4ce6-b5bf-b66ff6a40bc0 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:14.782713 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63745eab-cfd9-4ce6-b5bf-b66ff6a40bc0 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:14.782989 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63745eab-cfd9-4ce6-b5bf-b66ff6a40bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:14.811364 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-63745eab-cfd9-4ce6-b5bf-b66ff6a40bc0 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:14.812667 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-63745eab-cfd9-4ce6-b5bf-b66ff6a40bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:14.931189 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd814913-d3d5-4dbc-95dc-78198baab3c7 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:14.937778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 387/781] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:14.973089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54311706-5c0a-4b65-bb85-d5c05834bbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:14.974110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54311706-5c0a-4b65-bb85-d5c05834bbc2 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:14.974602 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54311706-5c0a-4b65-bb85-d5c05834bbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:14.975084 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54311706-5c0a-4b65-bb85-d5c05834bbc2 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:14.975965 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54311706-5c0a-4b65-bb85-d5c05834bbc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:14.977532 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54311706-5c0a-4b65-bb85-d5c05834bbc2 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:14.981744 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63745eab-cfd9-4ce6-b5bf-b66ff6a40bc0 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:14.997191 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 395/782] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:51:14 2022] GET /identity/v3/services => generated 772 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:15.094073 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:15.102178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63745eab-cfd9-4ce6-b5bf-b66ff6a40bc0 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:15.140596 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13763a14-ae15-4001-9544-d2d162d6d5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:15.141765 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13763a14-ae15-4001-9544-d2d162d6d5be None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:15.142240 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13763a14-ae15-4001-9544-d2d162d6d5be None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:15.142761 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13763a14-ae15-4001-9544-d2d162d6d5be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:15.155090 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13763a14-ae15-4001-9544-d2d162d6d5be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-13763a14-ae15-4001-9544-d2d162d6d5be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:15.163422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:15.164526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:15.165481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:15.165481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:15.165481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:15.165481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:15.165481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:15.165481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:15.165481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:15.165481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:15.165481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:15.165481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:15.165481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:15.165481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:15.168163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 388/783] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:51:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:15.175412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54311706-5c0a-4b65-bb85-d5c05834bbc2 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:15.274918 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:15.302303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54311706-5c0a-4b65-bb85-d5c05834bbc2 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:15.358749 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0442c-b1e9-4576-81ee-52596ac83c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:15.359330 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0442c-b1e9-4576-81ee-52596ac83c2e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:15.359610 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0442c-b1e9-4576-81ee-52596ac83c2e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:15.359883 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0442c-b1e9-4576-81ee-52596ac83c2e None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:15.360650 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0442c-b1e9-4576-81ee-52596ac83c2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:15.360958 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0442c-b1e9-4576-81ee-52596ac83c2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:15.362369 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0442c-b1e9-4576-81ee-52596ac83c2e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:15.377960 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 396/784] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:51:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:15.383335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13763a14-ae15-4001-9544-d2d162d6d5be None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:15.504946 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13763a14-ae15-4001-9544-d2d162d6d5be None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:15.505965 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e269743-0141-45d0-8d4d-2ccf4c28847d None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:15.506271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e269743-0141-45d0-8d4d-2ccf4c28847d None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:15.506552 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e269743-0141-45d0-8d4d-2ccf4c28847d None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:15.508296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 397/785] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:15.513166 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fa2f3-f5bd-42ae-86d4-4d6a38e32fbd None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:15.513453 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fa2f3-f5bd-42ae-86d4-4d6a38e32fbd None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:15.514549 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fa2f3-f5bd-42ae-86d4-4d6a38e32fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:15.542713 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f953d1-1a23-4fa2-b5e9-216c2e4dcdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:15.543962 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f953d1-1a23-4fa2-b5e9-216c2e4dcdcd None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:15.544488 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f953d1-1a23-4fa2-b5e9-216c2e4dcdcd None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:15.544968 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f953d1-1a23-4fa2-b5e9-216c2e4dcdcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:15.551291 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-494fa2f3-f5bd-42ae-86d4-4d6a38e32fbd None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:15.552544 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-494fa2f3-f5bd-42ae-86d4-4d6a38e32fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:15.557440 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f953d1-1a23-4fa2-b5e9-216c2e4dcdcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:15.559472 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f953d1-1a23-4fa2-b5e9-216c2e4dcdcd None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:15.581681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-56f953d1-1a23-4fa2-b5e9-216c2e4dcdcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:15.581681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:15.581681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:15.581681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:15.581681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:15.581681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:15.581681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:15.581681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:15.581681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:15.581681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:15.581681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:15.581681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:15.581681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:15.581681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:15.581681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:15.582778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:15.583884 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:15.583884 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:15.583884 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:15.583884 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:15.586775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 389/786] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:15.592594 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f953d1-1a23-4fa2-b5e9-216c2e4dcdcd None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:15.609278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:15.665865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 398/787] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:15.668363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b05359-2c46-45a1-8df2-2184e294952a None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:15.668886 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b05359-2c46-45a1-8df2-2184e294952a None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:15.669387 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b05359-2c46-45a1-8df2-2184e294952a None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:15.671323 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 399/788] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:15.677105 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348af29-1653-4653-8186-99efee27b5ba None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:15.678018 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348af29-1653-4653-8186-99efee27b5ba None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:15.678548 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348af29-1653-4653-8186-99efee27b5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:15.701197 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-7348af29-1653-4653-8186-99efee27b5ba None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:15.707058 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-7348af29-1653-4653-8186-99efee27b5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:15.720271 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f953d1-1a23-4fa2-b5e9-216c2e4dcdcd None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:15.757610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1d6ba-7997-4f2b-9974-4c7c9c1fabae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:15.758829 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1d6ba-7997-4f2b-9974-4c7c9c1fabae None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:15.759374 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1d6ba-7997-4f2b-9974-4c7c9c1fabae None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:15.759896 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1d6ba-7997-4f2b-9974-4c7c9c1fabae None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:15.760901 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1d6ba-7997-4f2b-9974-4c7c9c1fabae None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:15.761418 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1d6ba-7997-4f2b-9974-4c7c9c1fabae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:15.763150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1d6ba-7997-4f2b-9974-4c7c9c1fabae None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:15.772772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:15.780551 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 390/789] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:15.782055 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 400/790] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:15.782468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1d6ba-7997-4f2b-9974-4c7c9c1fabae None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:15.785602 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348af29-1653-4653-8186-99efee27b5ba None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:15.851360 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:15.888191 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1d6ba-7997-4f2b-9974-4c7c9c1fabae None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:15.927953 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e5154-8162-466b-a137-f28afc1a7112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:15.928775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e5154-8162-466b-a137-f28afc1a7112 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:15.929059 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e5154-8162-466b-a137-f28afc1a7112 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:15.929335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e5154-8162-466b-a137-f28afc1a7112 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:15.941643 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348af29-1653-4653-8186-99efee27b5ba None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:15.943251 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e5154-8162-466b-a137-f28afc1a7112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-cc4e5154-8162-466b-a137-f28afc1a7112 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:15.953047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:15.954280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:15.955230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:15.955230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:15.955230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:15.955230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:15.955230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:15.955230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:15.955230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:15.955230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:15.955230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:15.955230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:15.955230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:51:15.955230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:15.955230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 391/791] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:51:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:15.956331 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e5154-8162-466b-a137-f28afc1a7112 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:15.986030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8172e-8b61-4975-b6f5-a3f3f2e938ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:15.986883 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8172e-8b61-4975-b6f5-a3f3f2e938ed None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:15.987167 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8172e-8b61-4975-b6f5-a3f3f2e938ed None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:15.987443 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8172e-8b61-4975-b6f5-a3f3f2e938ed None admin] PATH_INFO: `/v3/users/swift` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:15.995540 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8172e-8b61-4975-b6f5-a3f3f2e938ed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:15.997004 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8172e-8b61-4975-b6f5-a3f3f2e938ed None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:16.080308 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e5154-8162-466b-a137-f28afc1a7112 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:16.111862 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:16.123244 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f85d38-fd53-44c1-8252-d4f162aa63b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:16.124281 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f85d38-fd53-44c1-8252-d4f162aa63b5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:16.124657 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f85d38-fd53-44c1-8252-d4f162aa63b5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:16.125038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f85d38-fd53-44c1-8252-d4f162aa63b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:16.136662 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f85d38-fd53-44c1-8252-d4f162aa63b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-71d8172e-8b61-4975-b6f5-a3f3f2e938ed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:16.141600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:16.143838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:16.146207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:16.146207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:16.146207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:16.146207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:16.146207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:16.146207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:16.146207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:16.146207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:16.146207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:51:16.146207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-c7f85d38-fd53-44c1-8252-d4f162aa63b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:16.147571 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:16.158834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:16.160947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:16.160947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:16.160947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:16.160947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:16.160947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:16.160947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:16.160947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:16.160947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:16.160947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:16.160947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:16.160947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:16.160947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:16.162580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 401/792] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:51:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:16.162580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8172e-8b61-4975-b6f5-a3f3f2e938ed None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:16.162901 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 392/793] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:51:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:16.162901 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f85d38-fd53-44c1-8252-d4f162aa63b5 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:16.210706 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:16.257690 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8172e-8b61-4975-b6f5-a3f3f2e938ed None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:16.258687 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f85d38-fd53-44c1-8252-d4f162aa63b5 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:16.293672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1737e-1398-4642-8217-14bb09cda9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:16.294425 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22e5d2-89ea-4077-8ca7-91fde88b969e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:16.294711 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1737e-1398-4642-8217-14bb09cda9d2 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:16.294711 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1737e-1398-4642-8217-14bb09cda9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:16.295207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1737e-1398-4642-8217-14bb09cda9d2 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:16.295498 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22e5d2-89ea-4077-8ca7-91fde88b969e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:16.296069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22e5d2-89ea-4077-8ca7-91fde88b969e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:16.296550 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22e5d2-89ea-4077-8ca7-91fde88b969e None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:16.297330 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1737e-1398-4642-8217-14bb09cda9d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:16.297512 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22e5d2-89ea-4077-8ca7-91fde88b969e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:16.297755 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1737e-1398-4642-8217-14bb09cda9d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:16.297921 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22e5d2-89ea-4077-8ca7-91fde88b969e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:16.299062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1737e-1398-4642-8217-14bb09cda9d2 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:16.299356 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22e5d2-89ea-4077-8ca7-91fde88b969e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:16.309575 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 402/794] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:51:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:16.310593 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1737e-1398-4642-8217-14bb09cda9d2 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:16.327377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 393/795] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 19:51:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:16.328322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22e5d2-89ea-4077-8ca7-91fde88b969e None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:16.389846 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:16.464142 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22e5d2-89ea-4077-8ca7-91fde88b969e None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:16.466146 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1737e-1398-4642-8217-14bb09cda9d2 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:16.489485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:16.502505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee5304-9499-42dc-8a08-a10802be1cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:16.503395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee5304-9499-42dc-8a08-a10802be1cbe None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:16.503703 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee5304-9499-42dc-8a08-a10802be1cbe None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:16.504041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee5304-9499-42dc-8a08-a10802be1cbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:16.510481 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232151b-cd89-4673-b254-ff047d24e8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:16.511278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232151b-cd89-4673-b254-ff047d24e8ec None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:16.511506 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232151b-cd89-4673-b254-ff047d24e8ec None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:16.511830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232151b-cd89-4673-b254-ff047d24e8ec None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:16.512799 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232151b-cd89-4673-b254-ff047d24e8ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:16.513122 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232151b-cd89-4673-b254-ff047d24e8ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:16.514724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232151b-cd89-4673-b254-ff047d24e8ec None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:16.516606 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee5304-9499-42dc-8a08-a10802be1cbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:16.518061 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee5304-9499-42dc-8a08-a10802be1cbe None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:16.524041 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 403/796] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:51:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:16.526316 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232151b-cd89-4673-b254-ff047d24e8ec None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:16.538912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-f1ee5304-9499-42dc-8a08-a10802be1cbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:16.538912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:16.538912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:16.538912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:16.538912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:16.538912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:16.538912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:16.538912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:16.538912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:16.538912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:16.538912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:16.538912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:16.538912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:16.538912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:16.538912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:16.539927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:16.546555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:16.546555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:16.546555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:16.546555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:16.546555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 394/797] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:16.546555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c3445-d88d-42c1-9f58-4426a92dbe29 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:16.546555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c3445-d88d-42c1-9f58-4426a92dbe29 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:16.546555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c3445-d88d-42c1-9f58-4426a92dbe29 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:16.546555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 395/798] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:16.547256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c3445-d88d-42c1-9f58-4426a92dbe29 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:16.632946 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232151b-cd89-4673-b254-ff047d24e8ec None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:16.647770 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:16.656749 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c3445-d88d-42c1-9f58-4426a92dbe29 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:16.668138 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8695c-0597-44e6-a195-411a97839544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:16.668938 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8695c-0597-44e6-a195-411a97839544 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:16.669215 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8695c-0597-44e6-a195-411a97839544 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:16.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8695c-0597-44e6-a195-411a97839544 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:16.679645 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8695c-0597-44e6-a195-411a97839544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:16.681097 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8695c-0597-44e6-a195-411a97839544 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:16.693696 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f3143-fe51-41ad-a87e-fe56c5d87e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:16.694796 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f3143-fe51-41ad-a87e-fe56c5d87e57 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:16.695283 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f3143-fe51-41ad-a87e-fe56c5d87e57 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:16.695751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f3143-fe51-41ad-a87e-fe56c5d87e57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:16.698172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-60d8695c-0597-44e6-a195-411a97839544 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:16.698172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:16.698172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:16.698172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:16.698172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:16.698172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:16.698172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:16.698172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:16.698172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:16.698172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:16.698172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:16.698172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:16.698172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:16.698172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:16.698172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:16.699314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:16.700320 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:16.700320 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:16.700320 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:16.700320 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:16.700320 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 404/799] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:16.700640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8695c-0597-44e6-a195-411a97839544 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:16.714421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f3143-fe51-41ad-a87e-fe56c5d87e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:16.716188 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f3143-fe51-41ad-a87e-fe56c5d87e57 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:16.734823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-963f3143-fe51-41ad-a87e-fe56c5d87e57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:16.734823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:16.734823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:16.734823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:16.734823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:16.734823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:16.734823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:16.734823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:16.734823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:16.734823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:16.734823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:16.734823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:16.734823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:16.734823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:16.734823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:16.735865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:16.736856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:16.736856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:16.736856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:16.736856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:16.736856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 396/800] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:16.737882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b61b2-4872-4648-9e07-c30e3f73498e None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:16.738166 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b61b2-4872-4648-9e07-c30e3f73498e None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:16.738464 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b61b2-4872-4648-9e07-c30e3f73498e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:16.749729 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:16.756479 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-1b8b61b2-4872-4648-9e07-c30e3f73498e None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:16.757912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-1b8b61b2-4872-4648-9e07-c30e3f73498e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:16.824377 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8695c-0597-44e6-a195-411a97839544 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:16.827197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 397/801] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:16.828631 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b61b2-4872-4648-9e07-c30e3f73498e None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:16.869887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78287c56-b9b3-42b2-83ab-6f136afeab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:16.870600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78287c56-b9b3-42b2-83ab-6f136afeab6f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:16.870884 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78287c56-b9b3-42b2-83ab-6f136afeab6f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:16.871158 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78287c56-b9b3-42b2-83ab-6f136afeab6f None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:16.871903 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78287c56-b9b3-42b2-83ab-6f136afeab6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:16.872208 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78287c56-b9b3-42b2-83ab-6f136afeab6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:16.873784 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78287c56-b9b3-42b2-83ab-6f136afeab6f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:16.887201 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 405/802] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:16.888048 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78287c56-b9b3-42b2-83ab-6f136afeab6f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:16.918045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:16.945675 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b61b2-4872-4648-9e07-c30e3f73498e None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:16.984229 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:16.986283 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edef55e-79d5-4af1-a398-a5ef1f3a19b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:16.986995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edef55e-79d5-4af1-a398-a5ef1f3a19b4 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:16.987544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edef55e-79d5-4af1-a398-a5ef1f3a19b4 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:16.987829 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edef55e-79d5-4af1-a398-a5ef1f3a19b4 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:16.988575 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edef55e-79d5-4af1-a398-a5ef1f3a19b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:16.988889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edef55e-79d5-4af1-a398-a5ef1f3a19b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:16.990307 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edef55e-79d5-4af1-a398-a5ef1f3a19b4 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:16.993848 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78287c56-b9b3-42b2-83ab-6f136afeab6f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:17.006570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 398/803] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:17.006570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edef55e-79d5-4af1-a398-a5ef1f3a19b4 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:17.033439 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9205341-fa74-4cf7-95a8-907afd36424a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:17.034105 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9205341-fa74-4cf7-95a8-907afd36424a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:17.034422 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9205341-fa74-4cf7-95a8-907afd36424a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:17.034700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9205341-fa74-4cf7-95a8-907afd36424a None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:17.035431 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9205341-fa74-4cf7-95a8-907afd36424a None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:17.035742 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9205341-fa74-4cf7-95a8-907afd36424a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:17.037109 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9205341-fa74-4cf7-95a8-907afd36424a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:17.048522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 406/804] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:17.049346 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9205341-fa74-4cf7-95a8-907afd36424a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:17.119684 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edef55e-79d5-4af1-a398-a5ef1f3a19b4 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:17.155872 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9205341-fa74-4cf7-95a8-907afd36424a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:17.157443 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13755fb-0772-436b-ace4-b17a90fd5990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:17.158281 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13755fb-0772-436b-ace4-b17a90fd5990 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:17.158574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13755fb-0772-436b-ace4-b17a90fd5990 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:17.158857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13755fb-0772-436b-ace4-b17a90fd5990 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:17.159518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13755fb-0772-436b-ace4-b17a90fd5990 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:17.160899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13755fb-0772-436b-ace4-b17a90fd5990 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-d13755fb-0772-436b-ace4-b17a90fd5990 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:17.173393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:17.174503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:17.175565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 399/805] 10.222.0.84 () {56 vars in 1129 bytes} [Thu May 19 19:51:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:17.176520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13755fb-0772-436b-ace4-b17a90fd5990 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:17.182230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:17.200596 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcf716-ec13-4635-8ebf-b559fb77a5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:17.201419 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcf716-ec13-4635-8ebf-b559fb77a5a5 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:17.201796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcf716-ec13-4635-8ebf-b559fb77a5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:17.202077 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcf716-ec13-4635-8ebf-b559fb77a5a5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:17.210495 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcf716-ec13-4635-8ebf-b559fb77a5a5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:17.212183 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcf716-ec13-4635-8ebf-b559fb77a5a5 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-9edcf716-ec13-4635-8ebf-b559fb77a5a5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:17.228059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:17.229366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:17.230451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:17.230451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:17.230451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:17.230451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:17.230451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:17.230451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:17.230451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:17.230451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:17.230451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:51:17.230451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:17.230451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 407/806] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:51:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:17.230451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcf716-ec13-4635-8ebf-b559fb77a5a5 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:17.252418 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:17.294949 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13755fb-0772-436b-ace4-b17a90fd5990 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:17.333339 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e97cf-1c97-4336-9ea9-ce849c6d6695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:17.334141 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e97cf-1c97-4336-9ea9-ce849c6d6695 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:17.334470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e97cf-1c97-4336-9ea9-ce849c6d6695 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:17.334827 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e97cf-1c97-4336-9ea9-ce849c6d6695 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:17.342540 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e97cf-1c97-4336-9ea9-ce849c6d6695 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:17.344482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e97cf-1c97-4336-9ea9-ce849c6d6695 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:17.347259 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcf716-ec13-4635-8ebf-b559fb77a5a5 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:17.360255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-699e97cf-1c97-4336-9ea9-ce849c6d6695 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:17.360255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:17.360255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:17.360255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:17.360255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:17.360255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:17.360255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:17.360255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:17.360255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:17.360255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:17.360255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:17.360255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:17.360255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:17.360255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:17.360255 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:17.361461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:17.362674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:17.362674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:17.362674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:17.362674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:17.362674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:17.362674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:17.362674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:17.362674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:17.365092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 400/807] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:17.366018 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e97cf-1c97-4336-9ea9-ce849c6d6695 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:17.384284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9eee7-36d9-47b4-8ba8-c11a35aa411c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:17.385110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9eee7-36d9-47b4-8ba8-c11a35aa411c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:17.385394 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9eee7-36d9-47b4-8ba8-c11a35aa411c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:17.385672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9eee7-36d9-47b4-8ba8-c11a35aa411c None admin] PATH_INFO: `/v3/users/nova` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:17.393339 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9eee7-36d9-47b4-8ba8-c11a35aa411c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:17.394819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9eee7-36d9-47b4-8ba8-c11a35aa411c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-3fd9eee7-36d9-47b4-8ba8-c11a35aa411c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:17.409380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:17.410663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:17.411717 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:17.411717 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:17.411717 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:17.411717 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:17.411717 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:17.411717 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:17.411717 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:17.411717 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:17.411717 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:51:17.411717 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:17.415277 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 408/808] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:51:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:17.416127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9eee7-36d9-47b4-8ba8-c11a35aa411c None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:17.436586 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:17.476893 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e97cf-1c97-4336-9ea9-ce849c6d6695 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:17.489881 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:17.513205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846a533-ee23-4b5a-9f45-2951f4ec44a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:17.513809 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846a533-ee23-4b5a-9f45-2951f4ec44a5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:17.514108 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846a533-ee23-4b5a-9f45-2951f4ec44a5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:17.514422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846a533-ee23-4b5a-9f45-2951f4ec44a5 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:17.515140 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846a533-ee23-4b5a-9f45-2951f4ec44a5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:17.515451 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846a533-ee23-4b5a-9f45-2951f4ec44a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:17.516835 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846a533-ee23-4b5a-9f45-2951f4ec44a5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:17.520728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9eee7-36d9-47b4-8ba8-c11a35aa411c None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:17.521536 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 401/809] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 19:51:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:17.522348 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846a533-ee23-4b5a-9f45-2951f4ec44a5 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:17.560774 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387267c5-6f2c-4f4d-bcd3-70de8d5fd3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:17.562063 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387267c5-6f2c-4f4d-bcd3-70de8d5fd3f1 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:17.562406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387267c5-6f2c-4f4d-bcd3-70de8d5fd3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:17.562691 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387267c5-6f2c-4f4d-bcd3-70de8d5fd3f1 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:17.563552 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387267c5-6f2c-4f4d-bcd3-70de8d5fd3f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:17.563865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387267c5-6f2c-4f4d-bcd3-70de8d5fd3f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:17.565326 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387267c5-6f2c-4f4d-bcd3-70de8d5fd3f1 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:17.583470 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 409/810] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 19:51:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:17.585214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387267c5-6f2c-4f4d-bcd3-70de8d5fd3f1 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:17.623599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846a533-ee23-4b5a-9f45-2951f4ec44a5 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:17.649447 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:17.658350 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f907b3-1e94-4b31-abee-f1044dfc0c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:17.659014 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f907b3-1e94-4b31-abee-f1044dfc0c3d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:17.659298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f907b3-1e94-4b31-abee-f1044dfc0c3d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:17.659577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f907b3-1e94-4b31-abee-f1044dfc0c3d None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:17.660370 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f907b3-1e94-4b31-abee-f1044dfc0c3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:17.660680 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f907b3-1e94-4b31-abee-f1044dfc0c3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:17.662185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f907b3-1e94-4b31-abee-f1044dfc0c3d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:17.674219 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 402/811] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:17.675594 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896910c4-5474-4a22-bf91-dea93d1555f1 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:17.675895 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896910c4-5474-4a22-bf91-dea93d1555f1 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:17.676176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896910c4-5474-4a22-bf91-dea93d1555f1 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:17.677569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 403/812] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:17.678113 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896910c4-5474-4a22-bf91-dea93d1555f1 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:17.688378 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387267c5-6f2c-4f4d-bcd3-70de8d5fd3f1 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:17.723160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:17.729114 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b70091-db7c-4997-83d3-4021d9bff544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:17.729828 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b70091-db7c-4997-83d3-4021d9bff544 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:17.730129 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b70091-db7c-4997-83d3-4021d9bff544 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:17.730430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b70091-db7c-4997-83d3-4021d9bff544 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:17.731280 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b70091-db7c-4997-83d3-4021d9bff544 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:17.731591 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b70091-db7c-4997-83d3-4021d9bff544 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:17.733650 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b70091-db7c-4997-83d3-4021d9bff544 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:17.751907 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 410/813] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:51:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:17.752741 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b70091-db7c-4997-83d3-4021d9bff544 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:17.792031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896910c4-5474-4a22-bf91-dea93d1555f1 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:17.825368 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bca340-f397-47c4-aea4-ecfff03d2272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:17.826036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bca340-f397-47c4-aea4-ecfff03d2272 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:17.826358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bca340-f397-47c4-aea4-ecfff03d2272 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:17.826801 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bca340-f397-47c4-aea4-ecfff03d2272 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:17.827524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bca340-f397-47c4-aea4-ecfff03d2272 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:17.828069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bca340-f397-47c4-aea4-ecfff03d2272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:17.829221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bca340-f397-47c4-aea4-ecfff03d2272 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:17.834219 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 404/814] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 19:51:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:17.835013 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bca340-f397-47c4-aea4-ecfff03d2272 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:17.854847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b70091-db7c-4997-83d3-4021d9bff544 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:17.888603 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:17.891420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2dccde-1d7a-43a3-843a-2c34ee4f1892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:17.892212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2dccde-1d7a-43a3-843a-2c34ee4f1892 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:17.892497 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2dccde-1d7a-43a3-843a-2c34ee4f1892 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:17.893052 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2dccde-1d7a-43a3-843a-2c34ee4f1892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:17.903390 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2dccde-1d7a-43a3-843a-2c34ee4f1892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:17.904884 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2dccde-1d7a-43a3-843a-2c34ee4f1892 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:17.922627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-cd2dccde-1d7a-43a3-843a-2c34ee4f1892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:17.922627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:17.922627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:17.922627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:17.922627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:17.922627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:17.922627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:17.922627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:17.922627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:17.922627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:17.922627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:17.922627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:17.922627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:17.922627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:17.922627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:17.923746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:17.924756 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:17.924756 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:17.924756 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:17.924756 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:17.924756 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 411/815] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:17.925120 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc90eb0-2f53-4c2e-ac5d-7fb9262f637f None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:17.925422 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc90eb0-2f53-4c2e-ac5d-7fb9262f637f None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:17.925678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc90eb0-2f53-4c2e-ac5d-7fb9262f637f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:17.942016 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-ddc90eb0-2f53-4c2e-ac5d-7fb9262f637f None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:17.942729 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bca340-f397-47c4-aea4-ecfff03d2272 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:17.943249 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-ddc90eb0-2f53-4c2e-ac5d-7fb9262f637f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:17.978206 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540502e-d3fd-4911-9e86-93c1363c4288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:17.979134 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540502e-d3fd-4911-9e86-93c1363c4288 None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:17.979417 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540502e-d3fd-4911-9e86-93c1363c4288 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:17.979693 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540502e-d3fd-4911-9e86-93c1363c4288 None admin] PATH_INFO: `/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/acebbcf1f90d487aa21a8eb99102ce09/roles/2cd6d3dd613c4824a26cacf53fb6c07f` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:17.980742 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:17.982578 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540502e-d3fd-4911-9e86-93c1363c4288 None admin] RBAC: Authorizing `identity:create_grant(project_id=d638a29a793d41e7bfa686c0fcc29149, user_id=acebbcf1f90d487aa21a8eb99102ce09, role_id=2cd6d3dd613c4824a26cacf53fb6c07f)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:17.983767 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540502e-d3fd-4911-9e86-93c1363c4288 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:17.996815 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 405/816] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:51:17 2022] PUT /identity/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/acebbcf1f90d487aa21a8eb99102ce09/roles/2cd6d3dd613c4824a26cacf53fb6c07f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:51:17.997606 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540502e-d3fd-4911-9e86-93c1363c4288 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:18.022537 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 412/817] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:18.022537 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc90eb0-2f53-4c2e-ac5d-7fb9262f637f None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:18.121899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540502e-d3fd-4911-9e86-93c1363c4288 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:18.134720 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:18.153305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc90eb0-2f53-4c2e-ac5d-7fb9262f637f None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:18.163960 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83840bb-11f6-4392-9ca5-4d7f19be8c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:18.164996 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83840bb-11f6-4392-9ca5-4d7f19be8c7f None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:18.165285 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83840bb-11f6-4392-9ca5-4d7f19be8c7f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:18.165878 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83840bb-11f6-4392-9ca5-4d7f19be8c7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:18.193470 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a3799-a221-42f8-9c15-6cd3d7fc7c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:18.194149 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a3799-a221-42f8-9c15-6cd3d7fc7c28 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:18.194477 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a3799-a221-42f8-9c15-6cd3d7fc7c28 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:18.194753 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a3799-a221-42f8-9c15-6cd3d7fc7c28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:18.195514 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a3799-a221-42f8-9c15-6cd3d7fc7c28 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:18.196901 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a3799-a221-42f8-9c15-6cd3d7fc7c28 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:18.219683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 413/818] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:51:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:51:18.221185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a3799-a221-42f8-9c15-6cd3d7fc7c28 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:18.246747 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83840bb-11f6-4392-9ca5-4d7f19be8c7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:18.248238 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83840bb-11f6-4392-9ca5-4d7f19be8c7f None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:18.256863 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:18.364974 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a3799-a221-42f8-9c15-6cd3d7fc7c28 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:18.369442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 406/819] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:51:18.370602 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83840bb-11f6-4392-9ca5-4d7f19be8c7f None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:18.397114 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:18.419581 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558e627-e9d7-4f44-8629-60b01bc4b57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:18.420726 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558e627-e9d7-4f44-8629-60b01bc4b57b None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:18.421613 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558e627-e9d7-4f44-8629-60b01bc4b57b None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:18.422281 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558e627-e9d7-4f44-8629-60b01bc4b57b None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:18.423369 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558e627-e9d7-4f44-8629-60b01bc4b57b None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:18.423879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558e627-e9d7-4f44-8629-60b01bc4b57b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:18.426461 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558e627-e9d7-4f44-8629-60b01bc4b57b None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:18.443087 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 414/820] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:18.444080 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558e627-e9d7-4f44-8629-60b01bc4b57b None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:18.501634 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83840bb-11f6-4392-9ca5-4d7f19be8c7f None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:18.544303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:18.545733 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e49d58-3426-4e3b-816a-375f74fa60cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:18.548367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e49d58-3426-4e3b-816a-375f74fa60cc None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:18.548367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e49d58-3426-4e3b-816a-375f74fa60cc None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:18.548367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e49d58-3426-4e3b-816a-375f74fa60cc None admin] PATH_INFO: `/v3/services/network` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:18.548367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e49d58-3426-4e3b-816a-375f74fa60cc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:18.549481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e49d58-3426-4e3b-816a-375f74fa60cc None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:18.567345 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558e627-e9d7-4f44-8629-60b01bc4b57b None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-f2e49d58-3426-4e3b-816a-375f74fa60cc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:18.568412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:18.569494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:18.570563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:18.570563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:18.570563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:18.570563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:18.570563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:18.570563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:18.570563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:18.570563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:18.570563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:18.570563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:18.570563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:18.570563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:18.570563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:51:18.570563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:18.570563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 407/821] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:18 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:18.571446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e49d58-3426-4e3b-816a-375f74fa60cc None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:18.620527 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fed0a-9394-4ba2-8b99-e0d041a78e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:18.621330 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fed0a-9394-4ba2-8b99-e0d041a78e29 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:18.621609 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fed0a-9394-4ba2-8b99-e0d041a78e29 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:18.621881 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fed0a-9394-4ba2-8b99-e0d041a78e29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:18.635862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2fed0a-9394-4ba2-8b99-e0d041a78e29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:18.637767 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2fed0a-9394-4ba2-8b99-e0d041a78e29 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:18.661091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-9c2fed0a-9394-4ba2-8b99-e0d041a78e29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:18.661091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:18.661091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:18.661091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:18.661091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:18.661091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:18.661091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:18.661091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:18.661091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:18.661091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:18.661091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:18.661091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:18.661091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:18.661091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:18.661091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:18.662161 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:18.663197 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:18.663197 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:18.663197 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:18.663197 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:18.666412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 415/822] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:18.667511 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fed0a-9394-4ba2-8b99-e0d041a78e29 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:18.670633 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:18.697505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e49d58-3426-4e3b-816a-375f74fa60cc None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:18.747919 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d038d-c802-474f-83c5-04b940b41a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:18.749298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d038d-c802-474f-83c5-04b940b41a19 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:18.749593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d038d-c802-474f-83c5-04b940b41a19 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:18.749874 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d038d-c802-474f-83c5-04b940b41a19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:18.757172 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d038d-c802-474f-83c5-04b940b41a19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:18.758677 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d038d-c802-474f-83c5-04b940b41a19 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:18.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-1b4d038d-c802-474f-83c5-04b940b41a19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:18.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:18.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:18.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:18.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:18.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:18.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:18.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:18.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:18.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:18.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:18.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:18.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:18.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:18.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:18.774985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:18.776089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:18.776089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:18.776089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:18.776089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:18.776089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:18.776089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:18.776089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:18.776089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:18.780597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 408/823] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:18.781570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d038d-c802-474f-83c5-04b940b41a19 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:18.802648 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fed0a-9394-4ba2-8b99-e0d041a78e29 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:18.843615 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363459b-73a2-481d-86fa-c5cb56e9505a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:18.844294 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363459b-73a2-481d-86fa-c5cb56e9505a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:18.844578 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363459b-73a2-481d-86fa-c5cb56e9505a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:18.844854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363459b-73a2-481d-86fa-c5cb56e9505a None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:18.846038 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363459b-73a2-481d-86fa-c5cb56e9505a None admin] RBAC: Adding query filter params (name=network) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:18.846458 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363459b-73a2-481d-86fa-c5cb56e9505a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:18.848485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363459b-73a2-481d-86fa-c5cb56e9505a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:18.849940 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:18.859687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 416/824] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:51:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:18.859687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363459b-73a2-481d-86fa-c5cb56e9505a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:18.915819 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d038d-c802-474f-83c5-04b940b41a19 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:18.953913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07880fd3-66ce-49fa-a2ee-dfe6b13d54e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:18.954870 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07880fd3-66ce-49fa-a2ee-dfe6b13d54e6 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:18.955250 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07880fd3-66ce-49fa-a2ee-dfe6b13d54e6 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:18.955624 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07880fd3-66ce-49fa-a2ee-dfe6b13d54e6 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:18.956460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07880fd3-66ce-49fa-a2ee-dfe6b13d54e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:18.956869 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07880fd3-66ce-49fa-a2ee-dfe6b13d54e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:18.958653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07880fd3-66ce-49fa-a2ee-dfe6b13d54e6 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:18.960090 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:18.977759 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 409/825] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:18.977759 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07880fd3-66ce-49fa-a2ee-dfe6b13d54e6 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:18.979849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363459b-73a2-481d-86fa-c5cb56e9505a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:19.015593 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d884f8-95a6-4dd9-8e79-eac35f3ad5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:19.016550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d884f8-95a6-4dd9-8e79-eac35f3ad5e3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:19.017011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d884f8-95a6-4dd9-8e79-eac35f3ad5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:19.017448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d884f8-95a6-4dd9-8e79-eac35f3ad5e3 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:19.018436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d884f8-95a6-4dd9-8e79-eac35f3ad5e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:19.018888 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d884f8-95a6-4dd9-8e79-eac35f3ad5e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:19.020598 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d884f8-95a6-4dd9-8e79-eac35f3ad5e3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:19.033307 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 417/826] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:19.034366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d884f8-95a6-4dd9-8e79-eac35f3ad5e3 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:19.095408 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07880fd3-66ce-49fa-a2ee-dfe6b13d54e6 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:19.107987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:19.137178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833a60d-7915-451d-a013-f8d93fc656b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:19.138111 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833a60d-7915-451d-a013-f8d93fc656b9 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:19.138521 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833a60d-7915-451d-a013-f8d93fc656b9 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:19.139189 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833a60d-7915-451d-a013-f8d93fc656b9 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:19.140062 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833a60d-7915-451d-a013-f8d93fc656b9 None admin] RBAC: Adding query filter params (type=network) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:19.140470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833a60d-7915-451d-a013-f8d93fc656b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:19.142007 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833a60d-7915-451d-a013-f8d93fc656b9 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:19.152587 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 410/827] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:51:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:19.152587 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833a60d-7915-451d-a013-f8d93fc656b9 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:19.161892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d884f8-95a6-4dd9-8e79-eac35f3ad5e3 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:19.199125 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a8421-4001-4e86-ba53-1e94fad58582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:19.200297 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a8421-4001-4e86-ba53-1e94fad58582 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:19.200737 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a8421-4001-4e86-ba53-1e94fad58582 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:19.201172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a8421-4001-4e86-ba53-1e94fad58582 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:19.209083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a8421-4001-4e86-ba53-1e94fad58582 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:19.210763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a8421-4001-4e86-ba53-1e94fad58582 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:19.214499 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:19.228242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-ee2a8421-4001-4e86-ba53-1e94fad58582 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:19.228242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:19.228242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:19.228242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:19.228242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:19.228242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:19.228242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:19.228242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:19.228242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:19.228242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:19.228242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:19.228242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:19.228242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:19.228242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:19.228242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:19.229452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:19.230722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:19.230722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:19.230722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:19.230722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:19.230722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:19.230722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:19.230722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:19.230722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:19.232851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 418/828] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:19.233756 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a8421-4001-4e86-ba53-1e94fad58582 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:19.272884 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833a60d-7915-451d-a013-f8d93fc656b9 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:19.312027 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad43c5-af2a-440e-867d-b447b74b6847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:19.313209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad43c5-af2a-440e-867d-b447b74b6847 None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:19.313572 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad43c5-af2a-440e-867d-b447b74b6847 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:19.313943 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad43c5-af2a-440e-867d-b447b74b6847 None admin] PATH_INFO: `/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/8683be51c17341f7b0e85d366a74869f/roles/3387bd4782b047ae8d880e386b0351f0` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:19.330104 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad43c5-af2a-440e-867d-b447b74b6847 None admin] RBAC: Authorizing `identity:create_grant(project_id=d638a29a793d41e7bfa686c0fcc29149, user_id=8683be51c17341f7b0e85d366a74869f, role_id=3387bd4782b047ae8d880e386b0351f0)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:19.331910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad43c5-af2a-440e-867d-b447b74b6847 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:19.351675 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 411/829] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:51:19 2022] PUT /identity/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/8683be51c17341f7b0e85d366a74869f/roles/3387bd4782b047ae8d880e386b0351f0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:51:19.353261 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad43c5-af2a-440e-867d-b447b74b6847 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:19.359167 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a8421-4001-4e86-ba53-1e94fad58582 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:19.381395 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:19.407958 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693b2ff-8770-40d0-bb6c-629b47052214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:19.409010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693b2ff-8770-40d0-bb6c-629b47052214 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:19.409505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693b2ff-8770-40d0-bb6c-629b47052214 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:19.409994 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693b2ff-8770-40d0-bb6c-629b47052214 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:19.411044 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3693b2ff-8770-40d0-bb6c-629b47052214 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:19.412684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3693b2ff-8770-40d0-bb6c-629b47052214 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:19.458328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 419/830] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:51:19 2022] POST /identity/v3/endpoints => generated 327 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:51:19.493556 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad43c5-af2a-440e-867d-b447b74b6847 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:19.505845 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:19.544727 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b92f4-cbb3-4140-b11b-0c39e37ee40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:19.546107 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b92f4-cbb3-4140-b11b-0c39e37ee40d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:19.546532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b92f4-cbb3-4140-b11b-0c39e37ee40d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:19.546829 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b92f4-cbb3-4140-b11b-0c39e37ee40d None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:19.547657 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b92f4-cbb3-4140-b11b-0c39e37ee40d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:19.547975 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b92f4-cbb3-4140-b11b-0c39e37ee40d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:19.549454 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b92f4-cbb3-4140-b11b-0c39e37ee40d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:19.568123 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 412/831] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:19.569822 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693b2ff-8770-40d0-bb6c-629b47052214 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:19.691560 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693b2ff-8770-40d0-bb6c-629b47052214 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:19.745869 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f415fa-aec1-49af-ab1c-b9e04b57f637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:19.746671 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f415fa-aec1-49af-ab1c-b9e04b57f637 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:19.746952 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f415fa-aec1-49af-ab1c-b9e04b57f637 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:19.747224 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f415fa-aec1-49af-ab1c-b9e04b57f637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:19.748084 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f415fa-aec1-49af-ab1c-b9e04b57f637 None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=0da7f490d9604107b974e9e461ebc38b) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:19.748601 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f415fa-aec1-49af-ab1c-b9e04b57f637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:19.750703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f415fa-aec1-49af-ab1c-b9e04b57f637 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:19.758072 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 420/832] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:19 2022] GET /identity/v3/role_assignments?user.id=0da7f490d9604107b974e9e461ebc38b&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:20.295528 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c6182-e0b9-42fe-baf0-59d1e5860b7e None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:20.296113 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c6182-e0b9-42fe-baf0-59d1e5860b7e None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:20.296113 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c6182-e0b9-42fe-baf0-59d1e5860b7e None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:20.297693 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 413/833] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:20.303705 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10a9c2-92a7-4efe-ad40-05baa3164705 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:20.306196 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10a9c2-92a7-4efe-ad40-05baa3164705 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:20.306196 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10a9c2-92a7-4efe-ad40-05baa3164705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:20.322816 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-de10a9c2-92a7-4efe-ad40-05baa3164705 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:20.324011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-de10a9c2-92a7-4efe-ad40-05baa3164705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:20.356536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:20.397614 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 421/834] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:20.421478 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c6182-e0b9-42fe-baf0-59d1e5860b7e None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:20.539385 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c6182-e0b9-42fe-baf0-59d1e5860b7e None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:20.574802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fae3e2-e231-4199-aedb-8403b13ae89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:20.575535 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fae3e2-e231-4199-aedb-8403b13ae89b None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:20.575818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fae3e2-e231-4199-aedb-8403b13ae89b None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:20.576095 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fae3e2-e231-4199-aedb-8403b13ae89b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:20.587499 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fae3e2-e231-4199-aedb-8403b13ae89b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-47fae3e2-e231-4199-aedb-8403b13ae89b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:20.595270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:20.596337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:20.597945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:20.597945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:20.597945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:20.597945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:20.597945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:20.597945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:20.597945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:20.597945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:20.597945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:20.597945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:20.597945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:20.597945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:20.597945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 414/835] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:51:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:20.602878 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10a9c2-92a7-4efe-ad40-05baa3164705 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:20.682883 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9bab7-afc5-4ea5-b021-da0e594a9a48 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:20.683216 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9bab7-afc5-4ea5-b021-da0e594a9a48 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:20.684338 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9bab7-afc5-4ea5-b021-da0e594a9a48 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:20.686096 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 415/836] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:51:20.693014 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dd52c-80f5-4a39-bbe7-07f391bd3904 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:20.693589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dd52c-80f5-4a39-bbe7-07f391bd3904 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:20.693873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dd52c-80f5-4a39-bbe7-07f391bd3904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:20.719996 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-128dd52c-80f5-4a39-bbe7-07f391bd3904 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:20.722366 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-128dd52c-80f5-4a39-bbe7-07f391bd3904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:20.725685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10a9c2-92a7-4efe-ad40-05baa3164705 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:20.747171 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:20.763611 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5972d1-82d1-4f53-8d74-5c5263cde798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:20.764339 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5972d1-82d1-4f53-8d74-5c5263cde798 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:20.764622 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5972d1-82d1-4f53-8d74-5c5263cde798 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:20.764898 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5972d1-82d1-4f53-8d74-5c5263cde798 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:20.765675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5972d1-82d1-4f53-8d74-5c5263cde798 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:20.765981 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5972d1-82d1-4f53-8d74-5c5263cde798 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:20.767446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5972d1-82d1-4f53-8d74-5c5263cde798 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:20.782488 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 422/837] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:51:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:20.783443 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5972d1-82d1-4f53-8d74-5c5263cde798 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:20.816621 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:20.826392 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 416/838] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:20.849582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128dd52c-80f5-4a39-bbe7-07f391bd3904 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:20.910569 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5972d1-82d1-4f53-8d74-5c5263cde798 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:20.957454 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098a579-618f-46a0-9b18-e88e8bb81d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:20.958549 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098a579-618f-46a0-9b18-e88e8bb81d0f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:20.958676 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128dd52c-80f5-4a39-bbe7-07f391bd3904 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:20.959094 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098a579-618f-46a0-9b18-e88e8bb81d0f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:20.959547 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098a579-618f-46a0-9b18-e88e8bb81d0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:20.974752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098a579-618f-46a0-9b18-e88e8bb81d0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:20.976716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098a579-618f-46a0-9b18-e88e8bb81d0f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:20.995906 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d330b0-b100-4f01-bdb7-c005a5e5d619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:20.996846 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d330b0-b100-4f01-bdb7-c005a5e5d619 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:20.997223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d330b0-b100-4f01-bdb7-c005a5e5d619 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:20.997590 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d330b0-b100-4f01-bdb7-c005a5e5d619 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:20.998531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d330b0-b100-4f01-bdb7-c005a5e5d619 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:21.000012 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d330b0-b100-4f01-bdb7-c005a5e5d619 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:21.001014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-4098a579-618f-46a0-9b18-e88e8bb81d0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:21.001014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:21.001014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:21.001014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:21.001014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:21.001014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:21.001014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:21.001014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:21.001014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:21.001014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:21.001014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:21.001014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:21.001014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:21.001014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:21.001014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:21.002199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:21.003484 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:21.003484 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:21.003484 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:21.003484 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:21.006586 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 423/839] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:21.011921 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098a579-618f-46a0-9b18-e88e8bb81d0f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-21d330b0-b100-4f01-bdb7-c005a5e5d619 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:21.021620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:21.022781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:21.023728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:21.023728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:21.023728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:21.023728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:21.023728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:21.023728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:21.023728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:21.023728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:21.023728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:21.023728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:21.023728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:21.023728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:21.023728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:51:21.023728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:21.026747 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 417/840] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:51:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:21.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d330b0-b100-4f01-bdb7-c005a5e5d619 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:21.060913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:21.113926 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:21.137572 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098a579-618f-46a0-9b18-e88e8bb81d0f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:21.156141 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d330b0-b100-4f01-bdb7-c005a5e5d619 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:21.184769 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4d96e-a296-4579-8c1f-0d9c0ca3992f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:21.185449 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4d96e-a296-4579-8c1f-0d9c0ca3992f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:21.186138 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4d96e-a296-4579-8c1f-0d9c0ca3992f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:21.186450 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4d96e-a296-4579-8c1f-0d9c0ca3992f None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:21.187471 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4d96e-a296-4579-8c1f-0d9c0ca3992f None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:21.187786 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4d96e-a296-4579-8c1f-0d9c0ca3992f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:21.189356 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4d96e-a296-4579-8c1f-0d9c0ca3992f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:21.197718 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d76f59-cc0e-4ce4-9c9a-12df45a36cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:21.198807 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d76f59-cc0e-4ce4-9c9a-12df45a36cf2 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:21.199367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d76f59-cc0e-4ce4-9c9a-12df45a36cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:21.199844 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d76f59-cc0e-4ce4-9c9a-12df45a36cf2 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:21.200837 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d76f59-cc0e-4ce4-9c9a-12df45a36cf2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:21.201392 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d76f59-cc0e-4ce4-9c9a-12df45a36cf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:21.203006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d76f59-cc0e-4ce4-9c9a-12df45a36cf2 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:21.204816 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 424/841] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:21.211116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4d96e-a296-4579-8c1f-0d9c0ca3992f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:21.211904 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 418/842] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:51:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:21.219269 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d76f59-cc0e-4ce4-9c9a-12df45a36cf2 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:21.309596 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:21.337097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d76f59-cc0e-4ce4-9c9a-12df45a36cf2 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:21.344328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4d96e-a296-4579-8c1f-0d9c0ca3992f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:21.375610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d903290-316a-46b2-b8ba-c5958cf70bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:21.376618 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d903290-316a-46b2-b8ba-c5958cf70bee None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:21.377160 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d903290-316a-46b2-b8ba-c5958cf70bee None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:21.377592 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d903290-316a-46b2-b8ba-c5958cf70bee None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:21.378491 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d903290-316a-46b2-b8ba-c5958cf70bee None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:21.378943 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d903290-316a-46b2-b8ba-c5958cf70bee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:21.380654 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d903290-316a-46b2-b8ba-c5958cf70bee None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:21.385570 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac445cd-5b5d-46e4-bde6-236b0eb64c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:21.390295 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac445cd-5b5d-46e4-bde6-236b0eb64c1a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:21.390295 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac445cd-5b5d-46e4-bde6-236b0eb64c1a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:21.390295 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac445cd-5b5d-46e4-bde6-236b0eb64c1a None admin] PATH_INFO: `/v3/users/swift` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:21.391162 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 419/843] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:51:21 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:21.399447 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac445cd-5b5d-46e4-bde6-236b0eb64c1a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:21.401204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac445cd-5b5d-46e4-bde6-236b0eb64c1a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-eac445cd-5b5d-46e4-bde6-236b0eb64c1a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:21.418036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:21.419405 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:21.420683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:21.420683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:21.420683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:21.420683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:21.420683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:21.420683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:21.420683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:21.420683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:21.420683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:51:21.420683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:21.423066 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 425/844] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:51:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:21.429183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d903290-316a-46b2-b8ba-c5958cf70bee None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:21.541444 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d903290-316a-46b2-b8ba-c5958cf70bee None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:21.579410 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c5207-5f5f-477e-a552-0f16ad94816c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:21.580437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c5207-5f5f-477e-a552-0f16ad94816c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:21.580869 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c5207-5f5f-477e-a552-0f16ad94816c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:21.581307 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c5207-5f5f-477e-a552-0f16ad94816c None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:21.582348 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c5207-5f5f-477e-a552-0f16ad94816c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:21.582807 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c5207-5f5f-477e-a552-0f16ad94816c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:21.584434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c5207-5f5f-477e-a552-0f16ad94816c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:21.605451 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 420/845] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 19:51:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:21.611543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac445cd-5b5d-46e4-bde6-236b0eb64c1a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:21.614707 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:21.731653 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac445cd-5b5d-46e4-bde6-236b0eb64c1a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:21.768693 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6c237-de26-4101-a212-0136dfb280bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:21.769616 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6c237-de26-4101-a212-0136dfb280bc None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:21.770156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6c237-de26-4101-a212-0136dfb280bc None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:21.770655 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6c237-de26-4101-a212-0136dfb280bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:21.840765 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6c237-de26-4101-a212-0136dfb280bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:21.842550 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6c237-de26-4101-a212-0136dfb280bc None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:21.844620 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:21.900459 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2bfc4-48c7-4fc1-802a-d1f1f7cc3c7e None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:21.901065 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2bfc4-48c7-4fc1-802a-d1f1f7cc3c7e None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:21.901551 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2bfc4-48c7-4fc1-802a-d1f1f7cc3c7e None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:21.903517 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 421/846] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:51:21.909079 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a776c95-35ab-4407-aff1-880c9a231041 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:21.909561 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a776c95-35ab-4407-aff1-880c9a231041 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:21.910010 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a776c95-35ab-4407-aff1-880c9a231041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:21.916530 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:21.929697 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-8a776c95-35ab-4407-aff1-880c9a231041 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:21.931278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-8a776c95-35ab-4407-aff1-880c9a231041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:21.952521 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 426/847] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:21 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:51:21.953967 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6c237-de26-4101-a212-0136dfb280bc None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:22.034429 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 422/848] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:22.059401 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a776c95-35ab-4407-aff1-880c9a231041 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:22.083143 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6c237-de26-4101-a212-0136dfb280bc None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:22.085256 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:22.132115 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3d6b4-9833-4853-822c-5dbff7a117d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:22.133221 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3d6b4-9833-4853-822c-5dbff7a117d4 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:22.133921 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3d6b4-9833-4853-822c-5dbff7a117d4 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:22.134380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3d6b4-9833-4853-822c-5dbff7a117d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:22.148698 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3d6b4-9833-4853-822c-5dbff7a117d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:22.150797 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3d6b4-9833-4853-822c-5dbff7a117d4 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:22.170960 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a776c95-35ab-4407-aff1-880c9a231041 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:22.172314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-7be3d6b4-9833-4853-822c-5dbff7a117d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:22.172314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:22.172314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:22.172314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:22.172314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:22.172314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:22.172314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:22.172314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:22.172314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:22.172314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:22.172314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:22.172314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:22.172314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:22.172314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:22.172314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:22.173430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:22.174904 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:22.174904 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:22.174904 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:22.174904 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:22.178234 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 427/849] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:22.179637 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d4920-b3a9-48fb-a1fe-29a4c228ac4e None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:22.179926 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d4920-b3a9-48fb-a1fe-29a4c228ac4e None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:22.180193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d4920-b3a9-48fb-a1fe-29a4c228ac4e None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:22.182065 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 428/850] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:22.182191 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d4920-b3a9-48fb-a1fe-29a4c228ac4e None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:22.213165 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6fe5b-9038-4d18-ae1f-9eb97927017e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:22.213942 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6fe5b-9038-4d18-ae1f-9eb97927017e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:22.214223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6fe5b-9038-4d18-ae1f-9eb97927017e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:22.214518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6fe5b-9038-4d18-ae1f-9eb97927017e None admin] PATH_INFO: `/v3/roles/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:22.216557 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:22.226722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6fe5b-9038-4d18-ae1f-9eb97927017e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-83a6fe5b-9038-4d18-ae1f-9eb97927017e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:22.234558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:22.235794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:22.236950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:22.236950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:22.236950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:22.236950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:22.236950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:22.236950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:22.236950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:22.236950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:22.236950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:22.236950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:22.236950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:51:22.236950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:22.241114 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 423/851] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:51:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:22.242282 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541998f4-5ed1-4297-9e1a-1577550d34d6 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:22.242625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541998f4-5ed1-4297-9e1a-1577550d34d6 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:22.242834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541998f4-5ed1-4297-9e1a-1577550d34d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:22.259648 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-541998f4-5ed1-4297-9e1a-1577550d34d6 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:22.260840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-541998f4-5ed1-4297-9e1a-1577550d34d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:22.301961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d4920-b3a9-48fb-a1fe-29a4c228ac4e None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:22.335088 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 424/852] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:22.336349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2e602-6d23-4546-9133-eb37daafe81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:22.336767 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541998f4-5ed1-4297-9e1a-1577550d34d6 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:22.337023 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e602-6d23-4546-9133-eb37daafe81f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:22.337303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e602-6d23-4546-9133-eb37daafe81f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:22.337598 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e602-6d23-4546-9133-eb37daafe81f None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:22.338329 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2e602-6d23-4546-9133-eb37daafe81f None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:22.338640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2e602-6d23-4546-9133-eb37daafe81f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:22.340013 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2e602-6d23-4546-9133-eb37daafe81f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:22.358571 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 429/853] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:22.359519 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2e602-6d23-4546-9133-eb37daafe81f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:22.375783 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:22.455156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541998f4-5ed1-4297-9e1a-1577550d34d6 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:22.463510 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2e602-6d23-4546-9133-eb37daafe81f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:22.486190 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:22.499960 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b3793-fe91-4d1c-a714-aba77d63689c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:22.500173 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6a71e-c7b7-480c-a745-6d7f5a4ce01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:22.500646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b3793-fe91-4d1c-a714-aba77d63689c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:22.500838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6a71e-c7b7-480c-a745-6d7f5a4ce01f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:22.500952 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b3793-fe91-4d1c-a714-aba77d63689c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:22.501151 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6a71e-c7b7-480c-a745-6d7f5a4ce01f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:22.501265 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b3793-fe91-4d1c-a714-aba77d63689c None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:22.501346 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6a71e-c7b7-480c-a745-6d7f5a4ce01f None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:22.501949 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b3793-fe91-4d1c-a714-aba77d63689c None admin] RBAC: Adding query filter params (name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:22.502273 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b3793-fe91-4d1c-a714-aba77d63689c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:22.503955 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b3793-fe91-4d1c-a714-aba77d63689c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:22.508593 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6a71e-c7b7-480c-a745-6d7f5a4ce01f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:22.510031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6a71e-c7b7-480c-a745-6d7f5a4ce01f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:22.518908 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 425/854] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:51:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:22.518908 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b3793-fe91-4d1c-a714-aba77d63689c None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:22.525515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-e0f6a71e-c7b7-480c-a745-6d7f5a4ce01f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:22.525515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:22.525515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:22.525515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:22.525515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:22.525515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:22.525515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:22.525515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:22.525515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:22.525515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:22.525515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:22.525515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:22.525515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:22.525515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:22.525515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:22.526738 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:22.527947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:22.527947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:22.527947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:22.527947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:22.527947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:22.527947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:22.527947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:22.527947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:22.527947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 430/855] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:22.527947 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6a71e-c7b7-480c-a745-6d7f5a4ce01f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:22.616907 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:22.644063 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b3793-fe91-4d1c-a714-aba77d63689c None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:22.646610 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6a71e-c7b7-480c-a745-6d7f5a4ce01f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:22.682095 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff7f38-884f-41ac-8fb0-4613d7379a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:22.682392 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc5582-17d7-426c-979c-c87b11cd56a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:22.683093 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff7f38-884f-41ac-8fb0-4613d7379a3f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:22.683356 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc5582-17d7-426c-979c-c87b11cd56a9 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:22.683473 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff7f38-884f-41ac-8fb0-4613d7379a3f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:22.683763 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc5582-17d7-426c-979c-c87b11cd56a9 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:22.683877 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff7f38-884f-41ac-8fb0-4613d7379a3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:22.684162 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc5582-17d7-426c-979c-c87b11cd56a9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:22.685082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc5582-17d7-426c-979c-c87b11cd56a9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:22.687405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc5582-17d7-426c-979c-c87b11cd56a9 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:22.695106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff7f38-884f-41ac-8fb0-4613d7379a3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:22.696561 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff7f38-884f-41ac-8fb0-4613d7379a3f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-2cfc5582-17d7-426c-979c-c87b11cd56a9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:22.700811 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:22.702839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:22.703873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:22.703873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:22.703873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:22.703873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:22.703873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:22.703873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:22.703873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:22.703873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:22.703873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:22.703873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:22.703873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:22.703873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:22.703873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:22.703873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:51:22.703873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:22.708313 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 426/856] 10.222.0.84 () {56 vars in 1132 bytes} [Thu May 19 19:51:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:22.709095 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc5582-17d7-426c-979c-c87b11cd56a9 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:22.722030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-34ff7f38-884f-41ac-8fb0-4613d7379a3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:22.722030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:22.722030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:22.722030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:22.722030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:22.722030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:22.722030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:22.722030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:22.722030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:22.722030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:22.722030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:22.722030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:22.722030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:22.722030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:22.722030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:22.724638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:22.726878 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:22.726878 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:22.726878 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:22.726878 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:22.728335 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 431/857] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:22.729401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff7f38-884f-41ac-8fb0-4613d7379a3f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:22.774689 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:22.831774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc5582-17d7-426c-979c-c87b11cd56a9 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:22.838441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff7f38-884f-41ac-8fb0-4613d7379a3f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:22.865810 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:22.874743 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97a349-fd90-4979-9ebe-5c38fc4e1c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:22.875355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97a349-fd90-4979-9ebe-5c38fc4e1c62 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:22.875638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97a349-fd90-4979-9ebe-5c38fc4e1c62 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:22.875912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97a349-fd90-4979-9ebe-5c38fc4e1c62 None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:22.876553 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e972f-f3a3-4bde-87ac-ab8fc81fdfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:22.876687 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97a349-fd90-4979-9ebe-5c38fc4e1c62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:22.877006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97a349-fd90-4979-9ebe-5c38fc4e1c62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:22.877233 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e972f-f3a3-4bde-87ac-ab8fc81fdfa6 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:22.877505 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e972f-f3a3-4bde-87ac-ab8fc81fdfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:22.877778 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e972f-f3a3-4bde-87ac-ab8fc81fdfa6 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:22.878474 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97a349-fd90-4979-9ebe-5c38fc4e1c62 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:22.878590 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e972f-f3a3-4bde-87ac-ab8fc81fdfa6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:22.878814 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e972f-f3a3-4bde-87ac-ab8fc81fdfa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:22.880335 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e972f-f3a3-4bde-87ac-ab8fc81fdfa6 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:22.887569 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 432/858] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:51:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:22.887569 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e972f-f3a3-4bde-87ac-ab8fc81fdfa6 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:22.896938 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 427/859] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:22.897950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97a349-fd90-4979-9ebe-5c38fc4e1c62 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:22.994046 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e972f-f3a3-4bde-87ac-ab8fc81fdfa6 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:23.008842 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97a349-fd90-4979-9ebe-5c38fc4e1c62 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:23.033034 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:23.049322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894c018-760d-4ebf-9506-f2f6f3767d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:23.050004 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894c018-760d-4ebf-9506-f2f6f3767d6f None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:23.050316 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894c018-760d-4ebf-9506-f2f6f3767d6f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:23.050601 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894c018-760d-4ebf-9506-f2f6f3767d6f None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:23.051328 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894c018-760d-4ebf-9506-f2f6f3767d6f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:23.051410 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537790b8-232b-4853-a305-2287cc83a305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:23.051640 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894c018-760d-4ebf-9506-f2f6f3767d6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:23.052254 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537790b8-232b-4853-a305-2287cc83a305 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:23.052600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537790b8-232b-4853-a305-2287cc83a305 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:23.052938 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537790b8-232b-4853-a305-2287cc83a305 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:23.053012 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894c018-760d-4ebf-9506-f2f6f3767d6f None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:23.053747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537790b8-232b-4853-a305-2287cc83a305 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:23.054109 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537790b8-232b-4853-a305-2287cc83a305 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:23.055516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537790b8-232b-4853-a305-2287cc83a305 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:23.058193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 428/860] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:51:22 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:23.058826 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894c018-760d-4ebf-9506-f2f6f3767d6f None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:23.086455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 433/861] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:23.091704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537790b8-232b-4853-a305-2287cc83a305 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:23.150091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:23.202809 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894c018-760d-4ebf-9506-f2f6f3767d6f None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:23.210074 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537790b8-232b-4853-a305-2287cc83a305 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:23.243476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeace5a7-c7a0-44b0-85c9-4e15ed1cd927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:23.244180 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeace5a7-c7a0-44b0-85c9-4e15ed1cd927 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:23.244471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeace5a7-c7a0-44b0-85c9-4e15ed1cd927 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:23.244748 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeace5a7-c7a0-44b0-85c9-4e15ed1cd927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:23.245744 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeace5a7-c7a0-44b0-85c9-4e15ed1cd927 None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=acebbcf1f90d487aa21a8eb99102ce09) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:23.246068 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeace5a7-c7a0-44b0-85c9-4e15ed1cd927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:23.247630 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeace5a7-c7a0-44b0-85c9-4e15ed1cd927 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:23.251618 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ed6ee-f843-4a65-a7db-7d590c238e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:23.252702 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ed6ee-f843-4a65-a7db-7d590c238e57 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:23.253165 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ed6ee-f843-4a65-a7db-7d590c238e57 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:23.253643 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ed6ee-f843-4a65-a7db-7d590c238e57 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:23.255121 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 429/862] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:23 2022] GET /identity/v3/role_assignments?user.id=acebbcf1f90d487aa21a8eb99102ce09&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:23.263339 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ed6ee-f843-4a65-a7db-7d590c238e57 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:23.266237 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ed6ee-f843-4a65-a7db-7d590c238e57 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-7f0ed6ee-f843-4a65-a7db-7d590c238e57 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:23.283538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:23.284401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:23.285284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:23.285284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:23.285284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:23.285284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:23.285284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:23.285284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:23.285284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:23.285284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:23.285284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:51:23.285284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:23.288066 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 434/863] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:51:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:23.294063 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeace5a7-c7a0-44b0-85c9-4e15ed1cd927 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:23.344307 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:23.417929 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeace5a7-c7a0-44b0-85c9-4e15ed1cd927 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:23.456758 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd41a8-01cf-4520-bfdb-3a055fb103b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:23.457443 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd41a8-01cf-4520-bfdb-3a055fb103b6 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:23.457727 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd41a8-01cf-4520-bfdb-3a055fb103b6 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:23.458006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd41a8-01cf-4520-bfdb-3a055fb103b6 None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:23.458903 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd41a8-01cf-4520-bfdb-3a055fb103b6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:23.459217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd41a8-01cf-4520-bfdb-3a055fb103b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:23.460678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd41a8-01cf-4520-bfdb-3a055fb103b6 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:23.480278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 430/864] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 19:51:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:23.486820 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ed6ee-f843-4a65-a7db-7d590c238e57 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:23.605210 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ed6ee-f843-4a65-a7db-7d590c238e57 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:23.618312 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:23.644461 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8e89b-c4f2-4b60-b0fe-a038dd853586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:23.645523 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8e89b-c4f2-4b60-b0fe-a038dd853586 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:23.645991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8e89b-c4f2-4b60-b0fe-a038dd853586 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:23.646533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8e89b-c4f2-4b60-b0fe-a038dd853586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:23.722289 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8e89b-c4f2-4b60-b0fe-a038dd853586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:23.722289 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8e89b-c4f2-4b60-b0fe-a038dd853586 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:23.791945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbf370-90d5-47db-a953-972b0ffe148e None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:23.792239 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbf370-90d5-47db-a953-972b0ffe148e None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:23.792519 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbf370-90d5-47db-a953-972b0ffe148e None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:23.794035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 431/865] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:23.800876 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6712ec-2404-415f-a6a3-7c84c01c10b0 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:23.801157 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6712ec-2404-415f-a6a3-7c84c01c10b0 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:23.801457 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6712ec-2404-415f-a6a3-7c84c01c10b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:23.819981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-6b6712ec-2404-415f-a6a3-7c84c01c10b0 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:23.821356 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-6b6712ec-2404-415f-a6a3-7c84c01c10b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:23.824684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 435/866] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:23 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:51:23.832679 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8e89b-c4f2-4b60-b0fe-a038dd853586 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:23.842538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:23.908981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 432/867] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:23.931131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6712ec-2404-415f-a6a3-7c84c01c10b0 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:23.940834 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8e89b-c4f2-4b60-b0fe-a038dd853586 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:23.963392 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:23.990465 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99264b-a650-46a2-89af-8b5d9c4c4bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:23.991627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99264b-a650-46a2-89af-8b5d9c4c4bd4 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:23.992101 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99264b-a650-46a2-89af-8b5d9c4c4bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:23.992974 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99264b-a650-46a2-89af-8b5d9c4c4bd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:24.006724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99264b-a650-46a2-89af-8b5d9c4c4bd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:24.009257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99264b-a650-46a2-89af-8b5d9c4c4bd4 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:24.032642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-de99264b-a650-46a2-89af-8b5d9c4c4bd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:24.032642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:24.032642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:24.032642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:24.032642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:24.032642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:24.032642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:24.032642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:24.032642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:24.032642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:24.032642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:24.032642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:24.032642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:24.032642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:24.032642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:24.033445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:24.034185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:24.034185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:24.034185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:24.034185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:24.035770 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 436/868] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:24.040962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99264b-a650-46a2-89af-8b5d9c4c4bd4 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:24.048518 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6712ec-2404-415f-a6a3-7c84c01c10b0 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:24.084376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03106f3-c287-4b3e-b98b-34b05a4e903d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:24.085404 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03106f3-c287-4b3e-b98b-34b05a4e903d None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:24.085881 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03106f3-c287-4b3e-b98b-34b05a4e903d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:24.086383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03106f3-c287-4b3e-b98b-34b05a4e903d None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:24.087342 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03106f3-c287-4b3e-b98b-34b05a4e903d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:24.088924 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03106f3-c287-4b3e-b98b-34b05a4e903d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:24.107114 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 433/869] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:51:23 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:51:24.111441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:24.165073 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99264b-a650-46a2-89af-8b5d9c4c4bd4 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:24.214829 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020b6f3-64ea-464d-a94d-35d6aa81e502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:24.215538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020b6f3-64ea-464d-a94d-35d6aa81e502 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:24.215815 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020b6f3-64ea-464d-a94d-35d6aa81e502 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:24.216087 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020b6f3-64ea-464d-a94d-35d6aa81e502 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:24.216948 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020b6f3-64ea-464d-a94d-35d6aa81e502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:24.217247 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020b6f3-64ea-464d-a94d-35d6aa81e502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:24.218729 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020b6f3-64ea-464d-a94d-35d6aa81e502 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:24.232883 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 437/870] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:24.238501 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03106f3-c287-4b3e-b98b-34b05a4e903d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:24.338019 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:24.357989 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03106f3-c287-4b3e-b98b-34b05a4e903d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:24.394918 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab13fa3-8bac-406b-9a2d-5980d78b978b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:24.395955 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab13fa3-8bac-406b-9a2d-5980d78b978b None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:24.396377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab13fa3-8bac-406b-9a2d-5980d78b978b None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:24.396788 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab13fa3-8bac-406b-9a2d-5980d78b978b None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:24.404038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab13fa3-8bac-406b-9a2d-5980d78b978b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:24.405831 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab13fa3-8bac-406b-9a2d-5980d78b978b None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:24.421357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-4ab13fa3-8bac-406b-9a2d-5980d78b978b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:24.421357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:24.421357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:24.421357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:24.421357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:24.421357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:24.421357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:24.421357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:24.421357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:24.421357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:24.421357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:24.421357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:24.421357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:24.421357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:24.421357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:24.422332 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:24.423283 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:24.423283 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:24.423283 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:24.423283 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:24.423283 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:24.423283 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:24.423283 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:24.423283 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:24.425432 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 434/871] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:24.431369 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020b6f3-64ea-464d-a94d-35d6aa81e502 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:24.550159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020b6f3-64ea-464d-a94d-35d6aa81e502 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:24.568658 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:24.587669 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1243f8d-d686-4ce6-bf1f-a5eb59a55bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:24.588334 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1243f8d-d686-4ce6-bf1f-a5eb59a55bf0 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:24.588610 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1243f8d-d686-4ce6-bf1f-a5eb59a55bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:24.588878 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1243f8d-d686-4ce6-bf1f-a5eb59a55bf0 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:24.589671 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1243f8d-d686-4ce6-bf1f-a5eb59a55bf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:24.589975 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1243f8d-d686-4ce6-bf1f-a5eb59a55bf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:24.591446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1243f8d-d686-4ce6-bf1f-a5eb59a55bf0 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:24.605193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 438/872] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:24.612269 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab13fa3-8bac-406b-9a2d-5980d78b978b None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:24.717918 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab13fa3-8bac-406b-9a2d-5980d78b978b None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:24.752854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71118c53-4014-4586-a29c-e02e005e9df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:24.753514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71118c53-4014-4586-a29c-e02e005e9df2 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:24.753785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71118c53-4014-4586-a29c-e02e005e9df2 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:24.755026 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71118c53-4014-4586-a29c-e02e005e9df2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:24.755879 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71118c53-4014-4586-a29c-e02e005e9df2 None admin] RBAC: Adding query filter params (role.id=3387bd4782b047ae8d880e386b0351f0, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=8683be51c17341f7b0e85d366a74869f) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:24.756196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71118c53-4014-4586-a29c-e02e005e9df2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:24.757637 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71118c53-4014-4586-a29c-e02e005e9df2 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:24.768770 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 435/873] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:24 2022] GET /identity/v3/role_assignments?user.id=8683be51c17341f7b0e85d366a74869f&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=3387bd4782b047ae8d880e386b0351f0 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:25.477655 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef78da9-e9b8-4242-9f8c-de9af7e46c77 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:25.478017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef78da9-e9b8-4242-9f8c-de9af7e46c77 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:25.478221 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef78da9-e9b8-4242-9f8c-de9af7e46c77 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:25.479864 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 439/874] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:25.485589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72c721-9080-4807-884a-6041738539f7 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:25.486068 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72c721-9080-4807-884a-6041738539f7 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:25.486136 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72c721-9080-4807-884a-6041738539f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:25.503369 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-ed72c721-9080-4807-884a-6041738539f7 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:25.504570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-ed72c721-9080-4807-884a-6041738539f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:25.507460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:25.577939 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 436/875] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:25 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:25.601456 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef78da9-e9b8-4242-9f8c-de9af7e46c77 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:25.711426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef78da9-e9b8-4242-9f8c-de9af7e46c77 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:25.741240 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf93cd-6c4a-49ec-91f9-fae491ab8411 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:25.741813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf93cd-6c4a-49ec-91f9-fae491ab8411 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:25.742321 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf93cd-6c4a-49ec-91f9-fae491ab8411 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:25.744982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 437/876] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:25.748130 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad37672-1d9b-4fe8-8419-481605407ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:25.749073 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad37672-1d9b-4fe8-8419-481605407ce8 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:25.749486 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad37672-1d9b-4fe8-8419-481605407ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:25.749588 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d88240-40e8-49e8-9894-fb0cb8221489 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:25.749940 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d88240-40e8-49e8-9894-fb0cb8221489 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:25.750101 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad37672-1d9b-4fe8-8419-481605407ce8 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:25.750437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d88240-40e8-49e8-9894-fb0cb8221489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:25.751138 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad37672-1d9b-4fe8-8419-481605407ce8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:25.752739 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad37672-1d9b-4fe8-8419-481605407ce8 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:25.768722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 440/877] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:51:25 2022] POST /identity/v3/services => generated 269 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:51:25.770237 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-64d88240-40e8-49e8-9894-fb0cb8221489 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:25.771597 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-64d88240-40e8-49e8-9894-fb0cb8221489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:25.855898 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 438/878] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:25.880485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad37672-1d9b-4fe8-8419-481605407ce8 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:25.925790 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:25.994316 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad37672-1d9b-4fe8-8419-481605407ce8 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:26.031277 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08e77c-fbd7-4de7-8663-f781eaf19b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:26.032170 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08e77c-fbd7-4de7-8663-f781eaf19b62 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:26.032399 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08e77c-fbd7-4de7-8663-f781eaf19b62 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:26.032674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08e77c-fbd7-4de7-8663-f781eaf19b62 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:26.033338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08e77c-fbd7-4de7-8663-f781eaf19b62 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:26.034799 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08e77c-fbd7-4de7-8663-f781eaf19b62 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-8c08e77c-fbd7-4de7-8663-f781eaf19b62 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:26.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:26.048271 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:26.048970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:26.048970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:26.048970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:26.048970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:26.048970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:26.048970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:26.048970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:26.048970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:26.048970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:26.048970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:26.048970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:26.048970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:26.048970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:51:26.048970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:26.053775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 441/879] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:51:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:26.055233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d88240-40e8-49e8-9894-fb0cb8221489 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:26.124908 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:26.161520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d88240-40e8-49e8-9894-fb0cb8221489 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:26.196552 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f166a96-8533-48c5-b03c-726e653d18f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:26.197228 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f166a96-8533-48c5-b03c-726e653d18f9 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:26.197511 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f166a96-8533-48c5-b03c-726e653d18f9 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:26.197843 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f166a96-8533-48c5-b03c-726e653d18f9 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:26.198603 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f166a96-8533-48c5-b03c-726e653d18f9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:26.198953 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f166a96-8533-48c5-b03c-726e653d18f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:26.200334 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f166a96-8533-48c5-b03c-726e653d18f9 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:26.210392 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 439/880] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:51:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:26.212304 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08e77c-fbd7-4de7-8663-f781eaf19b62 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:26.326914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08e77c-fbd7-4de7-8663-f781eaf19b62 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:26.329086 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:26.366387 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165088a-2b29-4f6d-8657-119d79a9c0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:26.367121 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165088a-2b29-4f6d-8657-119d79a9c0aa None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:26.367414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165088a-2b29-4f6d-8657-119d79a9c0aa None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:26.367684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165088a-2b29-4f6d-8657-119d79a9c0aa None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:26.368508 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165088a-2b29-4f6d-8657-119d79a9c0aa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:26.368821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165088a-2b29-4f6d-8657-119d79a9c0aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:26.370200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165088a-2b29-4f6d-8657-119d79a9c0aa None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:26.382429 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 442/881] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:51:26 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:26.504461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5cf82-a368-4fd3-b9f8-a8b2a8c739ef None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:26.504967 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5cf82-a368-4fd3-b9f8-a8b2a8c739ef None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:26.505037 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5cf82-a368-4fd3-b9f8-a8b2a8c739ef None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:26.506768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 440/882] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:26.512564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a074c-9c0c-4453-b2cd-3436581ca34d None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:26.512855 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a074c-9c0c-4453-b2cd-3436581ca34d None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:26.513136 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a074c-9c0c-4453-b2cd-3436581ca34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:26.531192 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-882a074c-9c0c-4453-b2cd-3436581ca34d None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:26.532544 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-882a074c-9c0c-4453-b2cd-3436581ca34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:26.607812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 443/883] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:26.629965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5cf82-a368-4fd3-b9f8-a8b2a8c739ef None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:26.748714 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5cf82-a368-4fd3-b9f8-a8b2a8c739ef None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:26.778906 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:26.785070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49aa1c-e72f-46b8-bfee-8eec7e2c03d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:26.785981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49aa1c-e72f-46b8-bfee-8eec7e2c03d7 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:26.786620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49aa1c-e72f-46b8-bfee-8eec7e2c03d7 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:26.786965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49aa1c-e72f-46b8-bfee-8eec7e2c03d7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:26.787789 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49aa1c-e72f-46b8-bfee-8eec7e2c03d7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:26.789358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49aa1c-e72f-46b8-bfee-8eec7e2c03d7 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-8d49aa1c-e72f-46b8-bfee-8eec7e2c03d7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:26.802048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:26.802935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:26.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:26.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:26.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:26.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:26.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:26.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:26.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:26.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:26.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:26.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:26.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:26.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:26.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:51:26.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:26.803674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 441/884] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:51:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:26.809968 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a074c-9c0c-4453-b2cd-3436581ca34d None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:26.904183 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:26.910499 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a074c-9c0c-4453-b2cd-3436581ca34d None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:26.945945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a46a48-40f4-44f1-9285-f342b0fa219c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:26.946669 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a46a48-40f4-44f1-9285-f342b0fa219c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:26.946952 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a46a48-40f4-44f1-9285-f342b0fa219c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:26.947233 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a46a48-40f4-44f1-9285-f342b0fa219c None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:26.947956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a46a48-40f4-44f1-9285-f342b0fa219c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:26.948262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a46a48-40f4-44f1-9285-f342b0fa219c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:26.949640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a46a48-40f4-44f1-9285-f342b0fa219c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:26.955091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 444/885] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:51:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:26.963360 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49aa1c-e72f-46b8-bfee-8eec7e2c03d7 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:27.065161 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49aa1c-e72f-46b8-bfee-8eec7e2c03d7 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:27.106357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d643ebd-3ff6-468d-a573-14997a314f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:27.106357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d643ebd-3ff6-468d-a573-14997a314f5d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:27.106357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d643ebd-3ff6-468d-a573-14997a314f5d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:27.106357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d643ebd-3ff6-468d-a573-14997a314f5d None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:27.106734 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d643ebd-3ff6-468d-a573-14997a314f5d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:27.107219 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d643ebd-3ff6-468d-a573-14997a314f5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:27.108826 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d643ebd-3ff6-468d-a573-14997a314f5d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:27.114611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 442/886] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:51:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:27.120086 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a46a48-40f4-44f1-9285-f342b0fa219c None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:27.220547 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a46a48-40f4-44f1-9285-f342b0fa219c None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:27.254650 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6312a-6914-4abe-92c4-6667dcaea9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:27.255362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6312a-6914-4abe-92c4-6667dcaea9d3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:27.255646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6312a-6914-4abe-92c4-6667dcaea9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:27.255918 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6312a-6914-4abe-92c4-6667dcaea9d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:27.256718 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6312a-6914-4abe-92c4-6667dcaea9d3 None admin] RBAC: Adding query filter params (interface=public, service_id=98c6afc009224cc5a827f2b44ca05d03, region_id=RegionOne) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:27.257027 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6312a-6914-4abe-92c4-6667dcaea9d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:27.258451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6312a-6914-4abe-92c4-6667dcaea9d3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:27.264537 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 445/887] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:51:27 2022] GET /identity/v3/endpoints?service_id=98c6afc009224cc5a827f2b44ca05d03&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:27.270032 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d643ebd-3ff6-468d-a573-14997a314f5d None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:27.339289 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:27.383149 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d643ebd-3ff6-468d-a573-14997a314f5d None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:27.417537 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59680e0a-7b75-4c42-a659-00ec2bbd30a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:27.418211 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59680e0a-7b75-4c42-a659-00ec2bbd30a8 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:27.418577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59680e0a-7b75-4c42-a659-00ec2bbd30a8 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:27.418858 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59680e0a-7b75-4c42-a659-00ec2bbd30a8 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:27.419605 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59680e0a-7b75-4c42-a659-00ec2bbd30a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:27.421225 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59680e0a-7b75-4c42-a659-00ec2bbd30a8 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:27.427498 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 443/888] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:51:27 2022] GET /identity/v3/services => generated 1268 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:51:27.505295 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f708e43-ac52-4f6a-9152-8b0e8526c6bb None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:27.506044 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f708e43-ac52-4f6a-9152-8b0e8526c6bb None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:27.506545 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f708e43-ac52-4f6a-9152-8b0e8526c6bb None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:27.508362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 446/889] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:27.516501 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7542ede-8eb9-44ce-9d45-4c2b0ee350e9 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:27.517029 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7542ede-8eb9-44ce-9d45-4c2b0ee350e9 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:27.517101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7542ede-8eb9-44ce-9d45-4c2b0ee350e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:27.536030 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-a7542ede-8eb9-44ce-9d45-4c2b0ee350e9 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:27.537261 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-a7542ede-8eb9-44ce-9d45-4c2b0ee350e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:27.621177 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 444/890] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:27.645574 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f708e43-ac52-4f6a-9152-8b0e8526c6bb None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:27.673491 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:27.758445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f708e43-ac52-4f6a-9152-8b0e8526c6bb None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:27.793791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeea191-13ca-4587-aaaa-40c230898655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:27.794640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeea191-13ca-4587-aaaa-40c230898655 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:27.794967 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeea191-13ca-4587-aaaa-40c230898655 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:27.795247 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeea191-13ca-4587-aaaa-40c230898655 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:27.805990 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeea191-13ca-4587-aaaa-40c230898655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-3aeea191-13ca-4587-aaaa-40c230898655 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:27.814612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:27.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:27.816175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:27.816175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:27.816175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:27.816175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:27.816175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:27.816175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:27.816175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:27.816175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:27.816175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:27.816175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:27.816175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:51:27.816175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:27.816175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 447/891] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 19:51:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:27.822041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7542ede-8eb9-44ce-9d45-4c2b0ee350e9 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:27.906636 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:27.925300 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7542ede-8eb9-44ce-9d45-4c2b0ee350e9 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:27.961326 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4074fd-21e9-4ead-839b-0ef73dcbf806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:27.962005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4074fd-21e9-4ead-839b-0ef73dcbf806 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:27.962309 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4074fd-21e9-4ead-839b-0ef73dcbf806 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:27.962592 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4074fd-21e9-4ead-839b-0ef73dcbf806 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:27.963351 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4074fd-21e9-4ead-839b-0ef73dcbf806 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:27.963662 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4074fd-21e9-4ead-839b-0ef73dcbf806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:27.965046 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4074fd-21e9-4ead-839b-0ef73dcbf806 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:27.978675 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 445/892] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 19:51:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:27.980345 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeea191-13ca-4587-aaaa-40c230898655 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:28.070459 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:28.087653 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeea191-13ca-4587-aaaa-40c230898655 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:28.126446 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee26062-e9fc-435b-a294-320857bf5e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:28.127284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee26062-e9fc-435b-a294-320857bf5e71 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:28.127567 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee26062-e9fc-435b-a294-320857bf5e71 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:28.127843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee26062-e9fc-435b-a294-320857bf5e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:28.140734 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee26062-e9fc-435b-a294-320857bf5e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:28.142226 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee26062-e9fc-435b-a294-320857bf5e71 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:28.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-0ee26062-e9fc-435b-a294-320857bf5e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:28.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:28.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:28.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:28.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:28.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:28.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:28.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:28.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:28.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:28.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:28.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:28.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:28.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:28.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:28.161825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:28.162606 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:28.162606 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:28.162606 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:28.162606 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:28.162606 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 448/893] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:28.168118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4074fd-21e9-4ead-839b-0ef73dcbf806 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:28.278448 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4074fd-21e9-4ead-839b-0ef73dcbf806 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:28.294760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:28.315989 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878f0ff-2410-4846-99e6-745ff0c44770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:28.316664 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878f0ff-2410-4846-99e6-745ff0c44770 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:28.316944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878f0ff-2410-4846-99e6-745ff0c44770 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:28.317218 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878f0ff-2410-4846-99e6-745ff0c44770 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:28.317948 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878f0ff-2410-4846-99e6-745ff0c44770 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:28.318275 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878f0ff-2410-4846-99e6-745ff0c44770 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:28.320580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cf3dc-5fde-43d9-b0df-89877359a3cc None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:28.320807 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cf3dc-5fde-43d9-b0df-89877359a3cc None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:28.321078 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cf3dc-5fde-43d9-b0df-89877359a3cc None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:28.322752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 449/894] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:51:28.323349 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878f0ff-2410-4846-99e6-745ff0c44770 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:28.336329 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68166210-adb1-411e-bcc9-5b85352d031f None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:28.336518 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68166210-adb1-411e-bcc9-5b85352d031f None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:28.336776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68166210-adb1-411e-bcc9-5b85352d031f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:28.345400 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 446/895] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:28.347404 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878f0ff-2410-4846-99e6-745ff0c44770 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:28.360818 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-68166210-adb1-411e-bcc9-5b85352d031f None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:28.362642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-68166210-adb1-411e-bcc9-5b85352d031f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:28.461567 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878f0ff-2410-4846-99e6-745ff0c44770 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:28.482463 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 450/896] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:28.506524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec23533-0b7d-4d11-af57-9f14df623753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:28.508124 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68166210-adb1-411e-bcc9-5b85352d031f None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:28.511548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec23533-0b7d-4d11-af57-9f14df623753 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:28.511826 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec23533-0b7d-4d11-af57-9f14df623753 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:28.512099 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec23533-0b7d-4d11-af57-9f14df623753 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:28.526779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec23533-0b7d-4d11-af57-9f14df623753 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:28.529406 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec23533-0b7d-4d11-af57-9f14df623753 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-aec23533-0b7d-4d11-af57-9f14df623753 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:28.550913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:28.551825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:28.552647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:28.552647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:28.552647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:28.552647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:28.552647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:28.552647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:28.552647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:28.552647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:28.552647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:51:28.552647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:28.560706 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 447/897] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:51:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:28.561538 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec23533-0b7d-4d11-af57-9f14df623753 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:28.600648 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:28.613290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:28.643037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68166210-adb1-411e-bcc9-5b85352d031f None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:28.683841 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33126b4e-3f11-4ba5-a24b-f042319ab1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:28.685536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33126b4e-3f11-4ba5-a24b-f042319ab1e0 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:28.685993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33126b4e-3f11-4ba5-a24b-f042319ab1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:28.686484 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33126b4e-3f11-4ba5-a24b-f042319ab1e0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:28.687521 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33126b4e-3f11-4ba5-a24b-f042319ab1e0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:28.689025 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33126b4e-3f11-4ba5-a24b-f042319ab1e0 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:28.704118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec23533-0b7d-4d11-af57-9f14df623753 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-33126b4e-3f11-4ba5-a24b-f042319ab1e0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:28.708489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:28.709406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:28.710162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:28.710162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:28.710162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:28.710162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:28.710162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:28.710162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:28.710162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:28.710162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:28.710162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:28.710162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:28.710162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:28.710162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:28.710162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:28.710162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:51:28.710162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:28.718681 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 451/898] 10.222.0.84 () {56 vars in 1132 bytes} [Thu May 19 19:51:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:28.726082 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33126b4e-3f11-4ba5-a24b-f042319ab1e0 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:28.753055 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5211f-dfa1-4ae2-b36b-be4259eef3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:28.753741 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5211f-dfa1-4ae2-b36b-be4259eef3fd None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:28.754022 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5211f-dfa1-4ae2-b36b-be4259eef3fd None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:28.754309 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5211f-dfa1-4ae2-b36b-be4259eef3fd None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:28.755173 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5211f-dfa1-4ae2-b36b-be4259eef3fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:28.755480 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5211f-dfa1-4ae2-b36b-be4259eef3fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:28.756913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5211f-dfa1-4ae2-b36b-be4259eef3fd None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:28.781575 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 448/899] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 19:51:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:28.781575 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379af72-bf2e-480a-9977-fb68bb32c782 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:28.781575 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379af72-bf2e-480a-9977-fb68bb32c782 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:28.781575 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379af72-bf2e-480a-9977-fb68bb32c782 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:28.781575 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 449/900] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:28.785092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379af72-bf2e-480a-9977-fb68bb32c782 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:28.857061 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33126b4e-3f11-4ba5-a24b-f042319ab1e0 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:28.886298 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:28.890365 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:28.893409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7eb38-5f98-4b7e-b527-f12e7aa7ec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:28.894078 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7eb38-5f98-4b7e-b527-f12e7aa7ec6d None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:28.894380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7eb38-5f98-4b7e-b527-f12e7aa7ec6d None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:28.894654 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7eb38-5f98-4b7e-b527-f12e7aa7ec6d None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:28.895365 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7eb38-5f98-4b7e-b527-f12e7aa7ec6d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:28.895674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7eb38-5f98-4b7e-b527-f12e7aa7ec6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:28.897046 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7eb38-5f98-4b7e-b527-f12e7aa7ec6d None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:28.901879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 452/901] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:51:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:28.902453 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379af72-bf2e-480a-9977-fb68bb32c782 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:28.903137 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444719a-8c11-4933-85b5-e55eaceead94 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:28.903427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444719a-8c11-4933-85b5-e55eaceead94 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:28.903698 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444719a-8c11-4933-85b5-e55eaceead94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:28.920524 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-1444719a-8c11-4933-85b5-e55eaceead94 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:28.921707 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-1444719a-8c11-4933-85b5-e55eaceead94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:28.942794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f4316-6fcd-42ca-ba0d-89ebbc3e3713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:28.943560 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f4316-6fcd-42ca-ba0d-89ebbc3e3713 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:28.943850 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f4316-6fcd-42ca-ba0d-89ebbc3e3713 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:28.944129 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f4316-6fcd-42ca-ba0d-89ebbc3e3713 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:28.992754 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 453/902] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:28.993680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444719a-8c11-4933-85b5-e55eaceead94 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:29.026291 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f4316-6fcd-42ca-ba0d-89ebbc3e3713 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:29.027757 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f4316-6fcd-42ca-ba0d-89ebbc3e3713 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:29.109532 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444719a-8c11-4933-85b5-e55eaceead94 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:29.127767 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:29.136390 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:29.142317 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 450/903] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:28 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:51:29.142317 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f4316-6fcd-42ca-ba0d-89ebbc3e3713 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:29.153169 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88d832-6c21-433c-93b2-e13586f4d5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:29.153847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88d832-6c21-433c-93b2-e13586f4d5de None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:29.154166 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88d832-6c21-433c-93b2-e13586f4d5de None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:29.154468 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88d832-6c21-433c-93b2-e13586f4d5de None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:29.155194 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88d832-6c21-433c-93b2-e13586f4d5de None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:29.155599 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88d832-6c21-433c-93b2-e13586f4d5de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:29.156977 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88d832-6c21-433c-93b2-e13586f4d5de None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:29.167578 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 454/904] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:51:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:29.167578 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88d832-6c21-433c-93b2-e13586f4d5de None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:29.250554 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f4316-6fcd-42ca-ba0d-89ebbc3e3713 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:29.273176 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88d832-6c21-433c-93b2-e13586f4d5de None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:29.288012 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36312cd8-8f6c-4ccf-93d0-5407c40263c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:29.288694 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36312cd8-8f6c-4ccf-93d0-5407c40263c8 None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:29.288981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36312cd8-8f6c-4ccf-93d0-5407c40263c8 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:29.289260 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36312cd8-8f6c-4ccf-93d0-5407c40263c8 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:29.290001 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36312cd8-8f6c-4ccf-93d0-5407c40263c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:29.291419 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36312cd8-8f6c-4ccf-93d0-5407c40263c8 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:29.307580 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 451/905] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:51:29 2022] POST /identity/v3/services => generated 241 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:51:29.308414 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36312cd8-8f6c-4ccf-93d0-5407c40263c8 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:29.328188 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b566b5-e694-40bd-9349-a1150804cef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:29.329157 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b566b5-e694-40bd-9349-a1150804cef8 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:29.329509 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b566b5-e694-40bd-9349-a1150804cef8 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:29.329852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b566b5-e694-40bd-9349-a1150804cef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:29.341798 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b566b5-e694-40bd-9349-a1150804cef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:29.343449 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b566b5-e694-40bd-9349-a1150804cef8 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:29.364490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-98b566b5-e694-40bd-9349-a1150804cef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:29.364490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:29.364490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:29.364490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:29.364490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:29.364490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:29.364490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:29.364490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:29.364490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:29.364490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:29.364490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:29.364490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:29.364490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:29.364490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:29.364490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:29.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:29.366534 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:29.366534 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:29.366534 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:29.366534 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:29.372341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 455/906] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:29.373498 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b566b5-e694-40bd-9349-a1150804cef8 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:29.387132 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:29.417448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:29.455772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36312cd8-8f6c-4ccf-93d0-5407c40263c8 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:29.498712 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b566b5-e694-40bd-9349-a1150804cef8 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:29.506004 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcc454-7fb9-42a5-85b9-089c9420a369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:29.506964 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcc454-7fb9-42a5-85b9-089c9420a369 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:29.507390 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcc454-7fb9-42a5-85b9-089c9420a369 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:29.507844 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcc454-7fb9-42a5-85b9-089c9420a369 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:29.508948 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcc454-7fb9-42a5-85b9-089c9420a369 None admin] RBAC: Adding query filter params (interface=public, service_id=2af84fd2d5264439b0f3f1ab8cd21294, region_id=RegionOne) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:29.509449 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcc454-7fb9-42a5-85b9-089c9420a369 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:29.511070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcc454-7fb9-42a5-85b9-089c9420a369 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:29.518014 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 452/907] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:51:29 2022] GET /identity/v3/endpoints?service_id=2af84fd2d5264439b0f3f1ab8cd21294&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:29.523294 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcc454-7fb9-42a5-85b9-089c9420a369 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:29.536527 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd419e-60fd-4221-929e-135804ec612b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:29.537204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd419e-60fd-4221-929e-135804ec612b None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:29.537483 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd419e-60fd-4221-929e-135804ec612b None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:29.537757 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd419e-60fd-4221-929e-135804ec612b None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:29.538522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd419e-60fd-4221-929e-135804ec612b None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:29.538828 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd419e-60fd-4221-929e-135804ec612b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:29.540205 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd419e-60fd-4221-929e-135804ec612b None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:29.552740 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 456/908] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:29.558133 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd419e-60fd-4221-929e-135804ec612b None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:29.646834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcc454-7fb9-42a5-85b9-089c9420a369 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:29.663625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:29.671668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:29.681445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd419e-60fd-4221-929e-135804ec612b None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:29.685928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc252b-4689-4595-aef6-28b8042b9e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:29.686648 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc252b-4689-4595-aef6-28b8042b9e5c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:29.686936 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc252b-4689-4595-aef6-28b8042b9e5c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:29.687216 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc252b-4689-4595-aef6-28b8042b9e5c None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:29.687922 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc252b-4689-4595-aef6-28b8042b9e5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:29.689307 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc252b-4689-4595-aef6-28b8042b9e5c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:29.696138 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 453/909] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:51:29 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:51:29.720816 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b8cf7-6b45-4a39-a511-d1edc1e947d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:29.726435 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b8cf7-6b45-4a39-a511-d1edc1e947d1 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:29.726435 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b8cf7-6b45-4a39-a511-d1edc1e947d1 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:29.726435 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b8cf7-6b45-4a39-a511-d1edc1e947d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:29.734831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b8cf7-6b45-4a39-a511-d1edc1e947d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:29.736548 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b8cf7-6b45-4a39-a511-d1edc1e947d1 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:29.752668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-696b8cf7-6b45-4a39-a511-d1edc1e947d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:29.752668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:29.752668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:29.752668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:29.752668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:29.752668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:29.752668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:29.752668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:29.752668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:29.752668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:29.752668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:29.752668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:29.752668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:29.752668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:29.752668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:29.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:29.754622 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:29.754622 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:29.754622 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:29.754622 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:29.754622 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:29.754622 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:29.754622 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:29.754622 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:29.760595 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 457/910] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:29.761915 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc252b-4689-4595-aef6-28b8042b9e5c None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:29.782333 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c2d73-c792-493e-891f-d2952ff6a9c9 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:29.782333 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c2d73-c792-493e-891f-d2952ff6a9c9 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:29.782333 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c2d73-c792-493e-891f-d2952ff6a9c9 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:29.782333 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 458/911] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:51:29.787086 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1856f-33a4-4b44-bd7e-e35eaef6d813 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:29.787378 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1856f-33a4-4b44-bd7e-e35eaef6d813 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:29.787649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1856f-33a4-4b44-bd7e-e35eaef6d813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:29.807751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-e3a1856f-33a4-4b44-bd7e-e35eaef6d813 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:29.809004 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-e3a1856f-33a4-4b44-bd7e-e35eaef6d813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:29.898413 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc252b-4689-4595-aef6-28b8042b9e5c None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:29.914342 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 459/912] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:29.934111 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1856f-33a4-4b44-bd7e-e35eaef6d813 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:29.945544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e7e21-0cd6-42d6-b2ef-182f5f8a224c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:29.946223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e7e21-0cd6-42d6-b2ef-182f5f8a224c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:29.946543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e7e21-0cd6-42d6-b2ef-182f5f8a224c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:29.946824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e7e21-0cd6-42d6-b2ef-182f5f8a224c None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:29.947629 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e7e21-0cd6-42d6-b2ef-182f5f8a224c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:29.947941 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e7e21-0cd6-42d6-b2ef-182f5f8a224c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:29.949385 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e7e21-0cd6-42d6-b2ef-182f5f8a224c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:29.963758 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 454/913] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:29.969509 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e7e21-0cd6-42d6-b2ef-182f5f8a224c None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:30.021747 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:30.033267 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:30.050879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1856f-33a4-4b44-bd7e-e35eaef6d813 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:30.090096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49711487-bf62-4bb3-80ca-6e48efd392cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:30.091397 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49711487-bf62-4bb3-80ca-6e48efd392cb None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:30.091934 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49711487-bf62-4bb3-80ca-6e48efd392cb None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:30.092377 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49711487-bf62-4bb3-80ca-6e48efd392cb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:30.093381 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49711487-bf62-4bb3-80ca-6e48efd392cb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:30.095063 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49711487-bf62-4bb3-80ca-6e48efd392cb None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:30.097287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e7e21-0cd6-42d6-b2ef-182f5f8a224c None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-49711487-bf62-4bb3-80ca-6e48efd392cb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:30.111341 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:30.112262 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:30.112989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:30.112989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:30.112989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:30.112989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:30.112989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:30.112989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:30.112989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:30.112989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:30.112989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:30.112989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:30.112989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:30.112989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:30.112989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:51:30.112989 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:30.120878 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 460/914] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:51:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:30.122508 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49711487-bf62-4bb3-80ca-6e48efd392cb None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:30.135824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4ade8-b0ba-4ceb-b71a-44fac3c5c729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:30.136500 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4ade8-b0ba-4ceb-b71a-44fac3c5c729 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:30.136782 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4ade8-b0ba-4ceb-b71a-44fac3c5c729 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:30.137061 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4ade8-b0ba-4ceb-b71a-44fac3c5c729 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:30.138240 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4ade8-b0ba-4ceb-b71a-44fac3c5c729 None admin] RBAC: Adding query filter params (role.id=206b3ccf6eb842fc8e1d86592c6500b7, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=8683be51c17341f7b0e85d366a74869f) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:30.138631 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4ade8-b0ba-4ceb-b71a-44fac3c5c729 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:30.140366 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4ade8-b0ba-4ceb-b71a-44fac3c5c729 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:30.148639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 455/915] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:29 2022] GET /identity/v3/role_assignments?user.id=8683be51c17341f7b0e85d366a74869f&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=206b3ccf6eb842fc8e1d86592c6500b7 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:30.252761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49711487-bf62-4bb3-80ca-6e48efd392cb None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:30.298323 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78d08f-1449-4790-8beb-be0dcee9ba16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:30.298323 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78d08f-1449-4790-8beb-be0dcee9ba16 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:30.298323 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78d08f-1449-4790-8beb-be0dcee9ba16 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:30.298323 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78d08f-1449-4790-8beb-be0dcee9ba16 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:30.298707 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78d08f-1449-4790-8beb-be0dcee9ba16 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:30.298707 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78d08f-1449-4790-8beb-be0dcee9ba16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:30.298707 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78d08f-1449-4790-8beb-be0dcee9ba16 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:30.305495 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 461/916] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:51:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:30.311591 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4ade8-b0ba-4ceb-b71a-44fac3c5c729 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:30.423195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4ade8-b0ba-4ceb-b71a-44fac3c5c729 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:30.426913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:30.462524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3bd56-d543-490c-b537-a81e3604fbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:30.463644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3bd56-d543-490c-b537-a81e3604fbe1 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:30.464274 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3bd56-d543-490c-b537-a81e3604fbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:30.464740 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3bd56-d543-490c-b537-a81e3604fbe1 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:30.465698 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3bd56-d543-490c-b537-a81e3604fbe1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:30.466237 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3bd56-d543-490c-b537-a81e3604fbe1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:30.467933 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3bd56-d543-490c-b537-a81e3604fbe1 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:30.478391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 456/917] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 19:51:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:30.480598 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78d08f-1449-4790-8beb-be0dcee9ba16 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:30.498041 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:30.595727 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78d08f-1449-4790-8beb-be0dcee9ba16 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:30.632308 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272b177-32f6-45ff-b8bc-d2c1437297c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:30.632979 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272b177-32f6-45ff-b8bc-d2c1437297c8 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:30.633263 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272b177-32f6-45ff-b8bc-d2c1437297c8 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:30.633538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272b177-32f6-45ff-b8bc-d2c1437297c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:30.634329 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272b177-32f6-45ff-b8bc-d2c1437297c8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:30.635728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272b177-32f6-45ff-b8bc-d2c1437297c8 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:30.656965 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 462/918] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:51:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:51:31.677754 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a588e8f-28e2-4f2d-abf0-89cbd17e5288 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:31.678299 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a588e8f-28e2-4f2d-abf0-89cbd17e5288 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:31.678375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a588e8f-28e2-4f2d-abf0-89cbd17e5288 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:31.679848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 457/919] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:31.685352 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b111562-c533-4b7b-b53d-97028ff22db3 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:31.686286 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b111562-c533-4b7b-b53d-97028ff22db3 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:31.686569 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b111562-c533-4b7b-b53d-97028ff22db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:31.719273 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-1b111562-c533-4b7b-b53d-97028ff22db3 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:31.720491 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-1b111562-c533-4b7b-b53d-97028ff22db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:31.787000 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:31.797780 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 463/920] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:31.820679 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a588e8f-28e2-4f2d-abf0-89cbd17e5288 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:31.939818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a588e8f-28e2-4f2d-abf0-89cbd17e5288 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:31.975646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538faf1c-739a-4dc0-a1cb-229c2c3cb637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:31.976466 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538faf1c-739a-4dc0-a1cb-229c2c3cb637 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:31.976757 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538faf1c-739a-4dc0-a1cb-229c2c3cb637 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:31.977036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538faf1c-739a-4dc0-a1cb-229c2c3cb637 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:31.977704 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538faf1c-739a-4dc0-a1cb-229c2c3cb637 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:31.985367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538faf1c-739a-4dc0-a1cb-229c2c3cb637 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-538faf1c-739a-4dc0-a1cb-229c2c3cb637 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:31.997692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:31.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:31.999902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:31.999902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:31.999902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:31.999902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:31.999902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:31.999902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:31.999902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:31.999902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:31.999902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:31.999902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:31.999902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:31.999902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:31.999902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:51:31.999902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:32.002325 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 458/921] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:51:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:32.010299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b111562-c533-4b7b-b53d-97028ff22db3 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:32.051302 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb811802-3959-4eb3-b0a5-0efa5e11045f None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:32.051586 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb811802-3959-4eb3-b0a5-0efa5e11045f None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:32.052008 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb811802-3959-4eb3-b0a5-0efa5e11045f None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:32.053712 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 459/922] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:51:32.058790 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3c087-5f86-4008-8823-b7d7784fe60a None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:32.059148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3c087-5f86-4008-8823-b7d7784fe60a None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:32.059495 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3c087-5f86-4008-8823-b7d7784fe60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:32.077330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-5ed3c087-5f86-4008-8823-b7d7784fe60a None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:32.078670 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-5ed3c087-5f86-4008-8823-b7d7784fe60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:32.110674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b111562-c533-4b7b-b53d-97028ff22db3 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:32.110870 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:32.145979 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94805025-45bc-4275-bad4-44ae99d95944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:32.146713 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94805025-45bc-4275-bad4-44ae99d95944 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:32.146954 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94805025-45bc-4275-bad4-44ae99d95944 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:32.147228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94805025-45bc-4275-bad4-44ae99d95944 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:32.147944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94805025-45bc-4275-bad4-44ae99d95944 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:32.148250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94805025-45bc-4275-bad4-44ae99d95944 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:32.149627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94805025-45bc-4275-bad4-44ae99d95944 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:32.154634 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 460/923] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:32.155840 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 464/924] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:51:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:32.160524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3c087-5f86-4008-8823-b7d7784fe60a None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:32.177734 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94805025-45bc-4275-bad4-44ae99d95944 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:32.253622 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:32.281251 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3c087-5f86-4008-8823-b7d7784fe60a None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:32.306891 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94805025-45bc-4275-bad4-44ae99d95944 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:32.321517 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4117fe4-7e0c-4018-8cb8-ff5952dbc9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:32.322201 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4117fe4-7e0c-4018-8cb8-ff5952dbc9b1 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:32.322517 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4117fe4-7e0c-4018-8cb8-ff5952dbc9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:32.322799 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4117fe4-7e0c-4018-8cb8-ff5952dbc9b1 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:32.323633 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4117fe4-7e0c-4018-8cb8-ff5952dbc9b1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:32.323950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4117fe4-7e0c-4018-8cb8-ff5952dbc9b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:32.325359 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4117fe4-7e0c-4018-8cb8-ff5952dbc9b1 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:32.332003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 461/925] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:51:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:32.337452 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4117fe4-7e0c-4018-8cb8-ff5952dbc9b1 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:32.354168 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc7430-287a-48ee-ac06-af1c3f6f8bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:32.356142 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc7430-287a-48ee-ac06-af1c3f6f8bd2 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:32.356644 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc7430-287a-48ee-ac06-af1c3f6f8bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:32.357121 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc7430-287a-48ee-ac06-af1c3f6f8bd2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:32.358194 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc7430-287a-48ee-ac06-af1c3f6f8bd2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:32.360179 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc7430-287a-48ee-ac06-af1c3f6f8bd2 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-91fc7430-287a-48ee-ac06-af1c3f6f8bd2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:32.378075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:32.379068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:32.379837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:32.379837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:32.379837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:32.379837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:32.379837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:32.379837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:32.379837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:32.379837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:32.379837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:32.379837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:32.379837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:32.379837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:32.379837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:32.379837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:51:32.379837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:32.384525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:32.386335 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 465/926] 10.222.0.84 () {56 vars in 1132 bytes} [Thu May 19 19:51:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:32.390156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc7430-287a-48ee-ac06-af1c3f6f8bd2 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:32.473161 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4117fe4-7e0c-4018-8cb8-ff5952dbc9b1 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:32.492957 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc7430-287a-48ee-ac06-af1c3f6f8bd2 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:32.519881 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0254d0-61aa-4a3a-beb1-6473c4a646f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:32.521305 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0254d0-61aa-4a3a-beb1-6473c4a646f9 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:32.521736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0254d0-61aa-4a3a-beb1-6473c4a646f9 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:32.522444 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0254d0-61aa-4a3a-beb1-6473c4a646f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:32.523548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0254d0-61aa-4a3a-beb1-6473c4a646f9 None admin] RBAC: Adding query filter params (interface=public, service_id=f4f472fecdae4fa088176a6249dbb33a, region_id=RegionOne) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:32.524005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0254d0-61aa-4a3a-beb1-6473c4a646f9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:32.526423 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0254d0-61aa-4a3a-beb1-6473c4a646f9 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:32.529387 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2bf42-e17a-45be-a8de-ce67f8352a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:32.530300 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2bf42-e17a-45be-a8de-ce67f8352a20 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:32.530660 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2bf42-e17a-45be-a8de-ce67f8352a20 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:32.531025 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2bf42-e17a-45be-a8de-ce67f8352a20 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:32.531898 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2bf42-e17a-45be-a8de-ce67f8352a20 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:32.532230 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2bf42-e17a-45be-a8de-ce67f8352a20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:32.533707 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2bf42-e17a-45be-a8de-ce67f8352a20 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:32.534231 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 462/927] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:51:32 2022] GET /identity/v3/endpoints?service_id=f4f472fecdae4fa088176a6249dbb33a&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:32.539149 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:32.540293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0254d0-61aa-4a3a-beb1-6473c4a646f9 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:32.548438 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 466/928] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:51:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:32.549361 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2bf42-e17a-45be-a8de-ce67f8352a20 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:32.651006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:32.661398 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0254d0-61aa-4a3a-beb1-6473c4a646f9 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:32.661882 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2bf42-e17a-45be-a8de-ce67f8352a20 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:32.698206 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4b092-98cf-41c7-ae30-f97a7315b027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:32.699040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4b092-98cf-41c7-ae30-f97a7315b027 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:32.699328 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4b092-98cf-41c7-ae30-f97a7315b027 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:32.699604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4b092-98cf-41c7-ae30-f97a7315b027 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:32.700303 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4b092-98cf-41c7-ae30-f97a7315b027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:32.701786 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4b092-98cf-41c7-ae30-f97a7315b027 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:32.706754 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37f5e7-533e-4d08-a088-2bc73e912e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:32.707974 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37f5e7-533e-4d08-a088-2bc73e912e99 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:32.708183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 463/929] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:51:32 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 19:51:32.708597 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37f5e7-533e-4d08-a088-2bc73e912e99 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:32.709038 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37f5e7-533e-4d08-a088-2bc73e912e99 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:32.709486 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9a391-4eb7-4893-9654-95b9b9d1bd6a None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:32.709775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9a391-4eb7-4893-9654-95b9b9d1bd6a None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:32.710053 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9a391-4eb7-4893-9654-95b9b9d1bd6a None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:32.710437 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37f5e7-533e-4d08-a088-2bc73e912e99 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:32.710912 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37f5e7-533e-4d08-a088-2bc73e912e99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:32.711493 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 464/930] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:32.712896 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37f5e7-533e-4d08-a088-2bc73e912e99 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:32.720307 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef1b5a-4849-40ca-96e0-708e9bd11c99 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:32.720584 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef1b5a-4849-40ca-96e0-708e9bd11c99 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:32.720857 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef1b5a-4849-40ca-96e0-708e9bd11c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:32.726005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 467/931] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:51:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:32.731466 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37f5e7-533e-4d08-a088-2bc73e912e99 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:32.747501 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-43ef1b5a-4849-40ca-96e0-708e9bd11c99 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:32.749129 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-43ef1b5a-4849-40ca-96e0-708e9bd11c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:32.790712 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:32.838557 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 465/932] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:32.847006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37f5e7-533e-4d08-a088-2bc73e912e99 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:32.855469 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef1b5a-4849-40ca-96e0-708e9bd11c99 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:32.885227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5b7cf-158f-48ce-872d-3df1ed8c09a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:32.885979 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5b7cf-158f-48ce-872d-3df1ed8c09a6 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:32.886294 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5b7cf-158f-48ce-872d-3df1ed8c09a6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:32.886613 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5b7cf-158f-48ce-872d-3df1ed8c09a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:32.887379 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5b7cf-158f-48ce-872d-3df1ed8c09a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:32.889033 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5b7cf-158f-48ce-872d-3df1ed8c09a6 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:32.913662 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 468/933] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:51:32 2022] POST /identity/v3/endpoints => generated 347 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:51:32.952761 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:32.974907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef1b5a-4849-40ca-96e0-708e9bd11c99 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:33.013480 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad2568-fb3a-4046-adc6-775026ba3925 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:33.013805 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad2568-fb3a-4046-adc6-775026ba3925 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:33.014090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad2568-fb3a-4046-adc6-775026ba3925 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:33.015828 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 469/934] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:51:33.024011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bec1a-8f25-4ae5-aa50-95665acce566 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:33.024302 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bec1a-8f25-4ae5-aa50-95665acce566 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:33.024578 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bec1a-8f25-4ae5-aa50-95665acce566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:33.031054 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc0f40-03aa-4344-9446-217920d1cfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:33.031903 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc0f40-03aa-4344-9446-217920d1cfae None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:33.032190 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc0f40-03aa-4344-9446-217920d1cfae None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:33.032468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc0f40-03aa-4344-9446-217920d1cfae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:33.045850 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc0f40-03aa-4344-9446-217920d1cfae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:33.053765 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-246bec1a-8f25-4ae5-aa50-95665acce566 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:33.055036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-246bec1a-8f25-4ae5-aa50-95665acce566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-19dc0f40-03aa-4344-9446-217920d1cfae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:33.056112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:33.056944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:33.057728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:33.057728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:33.057728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:33.057728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:33.057728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:33.057728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:33.057728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:33.057728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:33.057728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:33.057728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:33.057728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:51:33.057728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:33.060969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 466/935] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 19:51:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:33.065762 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc0f40-03aa-4344-9446-217920d1cfae None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:33.150285 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 470/936] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:33.167344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bec1a-8f25-4ae5-aa50-95665acce566 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:33.190632 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc0f40-03aa-4344-9446-217920d1cfae None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:33.214574 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:33.239281 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067481e-f23a-4108-975a-62e6041d90be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:33.240236 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067481e-f23a-4108-975a-62e6041d90be None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:33.240528 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067481e-f23a-4108-975a-62e6041d90be None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:33.240807 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067481e-f23a-4108-975a-62e6041d90be None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:33.241839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067481e-f23a-4108-975a-62e6041d90be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:33.242562 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067481e-f23a-4108-975a-62e6041d90be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:33.244232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067481e-f23a-4108-975a-62e6041d90be None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:33.256941 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 467/937] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 19:51:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:33.262683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067481e-f23a-4108-975a-62e6041d90be None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:33.266967 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:33.284496 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bec1a-8f25-4ae5-aa50-95665acce566 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:33.323318 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d304be-771c-4358-a695-3e8a2b63eadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:33.323917 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d304be-771c-4358-a695-3e8a2b63eadf None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:33.324204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d304be-771c-4358-a695-3e8a2b63eadf None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:33.324483 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d304be-771c-4358-a695-3e8a2b63eadf None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:33.325259 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d304be-771c-4358-a695-3e8a2b63eadf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:33.326878 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d304be-771c-4358-a695-3e8a2b63eadf None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:33.328345 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: WARNING keystone.server.flask.common [None req-49d304be-771c-4358-a695-3e8a2b63eadf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 19:51:33.349455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.sql.core [None req-49d304be-771c-4358-a695-3e8a2b63eadf None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 19:51:33.349455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 19:51:33.349455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [parameters: {'id': '1d996c5a6fb5493f9b5d98b8726c55d5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 19:51:33.349455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=150548) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 19:51:33.368776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-49d304be-771c-4358-a695-3e8a2b63eadf None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1d996c5a6fb5493f9b5d98b8726c55d5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1d996c5a6fb5493f9b5d98b8726c55d5. May 19 19:51:33.368776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:33.368776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:51:33.368776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:51:33.368776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:51:33.368776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:51:33.368776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:51:33.368776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:51:33.368776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:51:33.368776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application conn.query(q) May 19 19:51:33.368776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:51:33.368776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:51:33.368776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application result.read() May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 19:51:33.369607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application next(self.gen) May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application yield resource May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application next(self.gen) May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self.session.rollback() May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self.force_reraise() May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise self.value May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application session.commit() May 19 19:51:33.370485 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._prepare_impl() May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self.session.flush() May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._flush(objects) May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application compat.raise_( May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application flush_context.execute() May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 19:51:33.371299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rec.execute(self) May 19 19:51:33.372116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 19:51:33.372116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 19:51:33.372116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 19:51:33.372116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 19:51:33.372116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 19:51:33.372116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 19:51:33.372116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 19:51:33.372116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 19:51:33.372116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 19:51:33.372116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 19:51:33.372116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 19:51:33.372116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 19:51:33.372116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 19:51:33.372116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 19:51:33.372116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application conn.query(q) May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application result.read() May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:51:33.373031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:51:33.373819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:51:33.373819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:51:33.373819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:51:33.373819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:51:33.373819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:51:33.373819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:51:33.373819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 19:51:33.373819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 19:51:33.373819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [parameters: {'id': '1d996c5a6fb5493f9b5d98b8726c55d5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 19:51:33.373819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 19:51:33.373819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:33.373819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:33.373819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:33.373819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:33.373819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:33.375982 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:33.382922 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:33.382922 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 19:51:33.382922 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 19:51:33.382922 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:33.382922 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:33.382922 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 19:51:33.382922 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 19:51:33.382922 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1d996c5a6fb5493f9b5d98b8726c55d5. May 19 19:51:33.382922 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:33.382922 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 471/938] 10.222.0.84 () {60 vars in 1140 bytes} [Thu May 19 19:51:33 2022] POST /identity/v3/projects => generated 262 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:33.387528 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d304be-771c-4358-a695-3e8a2b63eadf None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:33.388913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067481e-f23a-4108-975a-62e6041d90be None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:33.427369 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83abcb-eb26-4910-9fb5-b025d8f5c5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:33.428176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83abcb-eb26-4910-9fb5-b025d8f5c5dd None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:33.428514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83abcb-eb26-4910-9fb5-b025d8f5c5dd None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:33.428796 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83abcb-eb26-4910-9fb5-b025d8f5c5dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:33.440974 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83abcb-eb26-4910-9fb5-b025d8f5c5dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:33.442478 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83abcb-eb26-4910-9fb5-b025d8f5c5dd None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:33.461176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-8e83abcb-eb26-4910-9fb5-b025d8f5c5dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:33.461176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:33.461176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:33.461176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:33.461176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:33.461176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:33.461176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:33.461176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:33.461176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:33.461176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:33.461176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:33.461176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:33.461176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:33.461176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:33.461176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:33.462040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:33.462902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:33.462902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:33.462902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:33.462902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:33.467742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 468/939] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:33.468875 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83abcb-eb26-4910-9fb5-b025d8f5c5dd None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:33.522737 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:33.530799 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d304be-771c-4358-a695-3e8a2b63eadf None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:33.556631 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:33.576388 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4be86d-33ee-4437-afcb-c15cb22ff5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:33.578176 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4be86d-33ee-4437-afcb-c15cb22ff5c4 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:33.578791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4be86d-33ee-4437-afcb-c15cb22ff5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:33.579508 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4be86d-33ee-4437-afcb-c15cb22ff5c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:33.581047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83abcb-eb26-4910-9fb5-b025d8f5c5dd None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:33.588723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4be86d-33ee-4437-afcb-c15cb22ff5c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:33.591089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4be86d-33ee-4437-afcb-c15cb22ff5c4 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:33.610177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-5c4be86d-33ee-4437-afcb-c15cb22ff5c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:33.610177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:33.610177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:33.610177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:33.610177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:33.610177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:33.610177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:33.610177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:33.610177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:33.610177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:33.610177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:33.610177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:33.610177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:33.610177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:33.610177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:33.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:33.611869 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:33.611869 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:33.611869 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:33.611869 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:33.611869 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:33.611869 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:33.611869 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:33.611869 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:33.618820 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 472/940] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:33.620317 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4be86d-33ee-4437-afcb-c15cb22ff5c4 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:33.626457 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c559b7-3fcc-4ce9-99ab-65942c3ed879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:33.627139 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c559b7-3fcc-4ce9-99ab-65942c3ed879 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:33.627420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c559b7-3fcc-4ce9-99ab-65942c3ed879 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:33.627694 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c559b7-3fcc-4ce9-99ab-65942c3ed879 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:33.628431 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c559b7-3fcc-4ce9-99ab-65942c3ed879 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:33.628737 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c559b7-3fcc-4ce9-99ab-65942c3ed879 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:33.630121 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c559b7-3fcc-4ce9-99ab-65942c3ed879 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:33.643439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 469/941] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:33.648943 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c559b7-3fcc-4ce9-99ab-65942c3ed879 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:33.739180 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4be86d-33ee-4437-afcb-c15cb22ff5c4 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:33.773408 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd56cb-31da-42b1-8d25-34673d7585b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:33.774024 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd56cb-31da-42b1-8d25-34673d7585b6 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:33.774529 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd56cb-31da-42b1-8d25-34673d7585b6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:33.774869 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd56cb-31da-42b1-8d25-34673d7585b6 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:33.775661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd56cb-31da-42b1-8d25-34673d7585b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:33.776062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd56cb-31da-42b1-8d25-34673d7585b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:33.777604 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd56cb-31da-42b1-8d25-34673d7585b6 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:33.780577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c559b7-3fcc-4ce9-99ab-65942c3ed879 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:33.797902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:33.799605 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 473/942] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:51:33 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:33.823587 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75590415-c257-423e-8b0f-fa01630b711a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:33.824685 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75590415-c257-423e-8b0f-fa01630b711a None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:33.825143 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75590415-c257-423e-8b0f-fa01630b711a None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:33.825563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75590415-c257-423e-8b0f-fa01630b711a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:33.834932 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75590415-c257-423e-8b0f-fa01630b711a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:33.836644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75590415-c257-423e-8b0f-fa01630b711a None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-75590415-c257-423e-8b0f-fa01630b711a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:33.854195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:33.855446 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:33.856527 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:33.856527 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:33.856527 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:33.856527 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:33.856527 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:33.856527 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:33.856527 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:33.856527 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:33.856527 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:51:33.856527 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:33.858952 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 470/943] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:51:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:33.867816 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd56cb-31da-42b1-8d25-34673d7585b6 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:33.887815 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:33.990717 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd56cb-31da-42b1-8d25-34673d7585b6 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:34.027519 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819788e-ae6c-4f9f-b2b7-379fa9299923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:34.028200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819788e-ae6c-4f9f-b2b7-379fa9299923 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:34.028481 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819788e-ae6c-4f9f-b2b7-379fa9299923 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:34.028754 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819788e-ae6c-4f9f-b2b7-379fa9299923 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:34.029676 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819788e-ae6c-4f9f-b2b7-379fa9299923 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:34.029983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819788e-ae6c-4f9f-b2b7-379fa9299923 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:34.031470 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819788e-ae6c-4f9f-b2b7-379fa9299923 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:34.050394 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 474/944] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 19:51:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:34.055639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75590415-c257-423e-8b0f-fa01630b711a None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:34.168724 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75590415-c257-423e-8b0f-fa01630b711a None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:34.209159 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd5e99-9c58-469c-9a35-64df064c2afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:34.209967 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd5e99-9c58-469c-9a35-64df064c2afc None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:34.210249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd5e99-9c58-469c-9a35-64df064c2afc None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:34.210552 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd5e99-9c58-469c-9a35-64df064c2afc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:34.221814 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd5e99-9c58-469c-9a35-64df064c2afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:34.223375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd5e99-9c58-469c-9a35-64df064c2afc None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:34.241800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-54cd5e99-9c58-469c-9a35-64df064c2afc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:34.241800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:34.241800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:34.241800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:34.241800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:34.241800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:34.241800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:34.241800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:34.241800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:34.241800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:34.241800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:34.241800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:34.241800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:34.241800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:34.241800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:34.243098 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:34.244410 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:34.244410 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:34.244410 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:34.244410 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:34.244410 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 471/945] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:34.254752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819788e-ae6c-4f9f-b2b7-379fa9299923 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:34.338907 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:34.371979 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819788e-ae6c-4f9f-b2b7-379fa9299923 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:34.409661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e53fc-344b-4fa8-ae18-c2e1daa031d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:34.410647 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e53fc-344b-4fa8-ae18-c2e1daa031d6 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:34.410931 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e53fc-344b-4fa8-ae18-c2e1daa031d6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:34.411200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e53fc-344b-4fa8-ae18-c2e1daa031d6 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:34.411949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e53fc-344b-4fa8-ae18-c2e1daa031d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:34.412258 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e53fc-344b-4fa8-ae18-c2e1daa031d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:34.413640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e53fc-344b-4fa8-ae18-c2e1daa031d6 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:34.426931 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 475/946] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:34.432408 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd5e99-9c58-469c-9a35-64df064c2afc None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:34.462378 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:34.544896 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd5e99-9c58-469c-9a35-64df064c2afc None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:34.589661 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6645d1-866f-47e6-a901-5850253b5694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:34.590520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6645d1-866f-47e6-a901-5850253b5694 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:34.590944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6645d1-866f-47e6-a901-5850253b5694 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:34.591223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6645d1-866f-47e6-a901-5850253b5694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:34.599348 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6645d1-866f-47e6-a901-5850253b5694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:34.600813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6645d1-866f-47e6-a901-5850253b5694 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:34.616335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-af6645d1-866f-47e6-a901-5850253b5694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:34.616335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:34.616335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:34.616335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:34.616335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:34.616335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:34.616335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:34.616335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:34.616335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:34.616335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:34.616335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:34.616335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:34.616335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:34.616335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:34.616335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:34.617183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:34.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:34.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:34.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:34.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:34.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:34.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:34.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:34.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:34.617982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 472/947] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:34.624019 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e53fc-344b-4fa8-ae18-c2e1daa031d6 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:34.744566 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e53fc-344b-4fa8-ae18-c2e1daa031d6 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:34.778671 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:34.781199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad97058-b7fd-47d1-8bd8-06c9e8756970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:34.781786 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad97058-b7fd-47d1-8bd8-06c9e8756970 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:34.782075 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad97058-b7fd-47d1-8bd8-06c9e8756970 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:34.782400 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad97058-b7fd-47d1-8bd8-06c9e8756970 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:34.783218 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad97058-b7fd-47d1-8bd8-06c9e8756970 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:34.783781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad97058-b7fd-47d1-8bd8-06c9e8756970 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:34.785379 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad97058-b7fd-47d1-8bd8-06c9e8756970 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:34.798628 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 476/948] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:34.804484 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6645d1-866f-47e6-a901-5850253b5694 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:34.901819 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:34.926753 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6645d1-866f-47e6-a901-5850253b5694 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:34.962682 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e5425-e0d6-4fb5-a050-1d47cc8cd25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:34.963632 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e5425-e0d6-4fb5-a050-1d47cc8cd25d None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:34.963913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e5425-e0d6-4fb5-a050-1d47cc8cd25d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:34.964189 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e5425-e0d6-4fb5-a050-1d47cc8cd25d None admin] PATH_INFO: `/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/8683be51c17341f7b0e85d366a74869f/roles/206b3ccf6eb842fc8e1d86592c6500b7` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:34.966562 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e5425-e0d6-4fb5-a050-1d47cc8cd25d None admin] RBAC: Authorizing `identity:create_grant(project_id=d638a29a793d41e7bfa686c0fcc29149, user_id=8683be51c17341f7b0e85d366a74869f, role_id=206b3ccf6eb842fc8e1d86592c6500b7)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:34.967973 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e5425-e0d6-4fb5-a050-1d47cc8cd25d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:34.984009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 473/949] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:51:34 2022] PUT /identity/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/8683be51c17341f7b0e85d366a74869f/roles/206b3ccf6eb842fc8e1d86592c6500b7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:51:35.104606 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3d687-2016-4f06-ba40-f3a6e4ce2add None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:35.105139 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3d687-2016-4f06-ba40-f3a6e4ce2add None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:35.105581 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3d687-2016-4f06-ba40-f3a6e4ce2add None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:35.107697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 477/950] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:35.114303 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aaf212-70fb-4c5b-95e3-9a8b6210d0e2 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:35.114792 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aaf212-70fb-4c5b-95e3-9a8b6210d0e2 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:35.115178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aaf212-70fb-4c5b-95e3-9a8b6210d0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:35.136671 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-46aaf212-70fb-4c5b-95e3-9a8b6210d0e2 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:35.138000 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-46aaf212-70fb-4c5b-95e3-9a8b6210d0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:35.213838 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 474/951] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:35.239515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3d687-2016-4f06-ba40-f3a6e4ce2add None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:35.309212 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d2265-c7c0-40f1-85c2-10f1d02fca45 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:35.309522 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d2265-c7c0-40f1-85c2-10f1d02fca45 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:35.309770 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d2265-c7c0-40f1-85c2-10f1d02fca45 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:35.311608 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 475/952] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:35.316793 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decbc64-fbe7-460d-bce7-139dfa81b59f None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:35.317079 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decbc64-fbe7-460d-bce7-139dfa81b59f None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:35.317358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decbc64-fbe7-460d-bce7-139dfa81b59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:35.338346 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-5decbc64-fbe7-460d-bce7-139dfa81b59f None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:35.339867 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-5decbc64-fbe7-460d-bce7-139dfa81b59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:35.353732 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3d687-2016-4f06-ba40-f3a6e4ce2add None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:35.389255 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b739d10-cabc-4a30-bab8-d831cd989f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:35.390081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b739d10-cabc-4a30-bab8-d831cd989f79 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:35.390392 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b739d10-cabc-4a30-bab8-d831cd989f79 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:35.390664 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b739d10-cabc-4a30-bab8-d831cd989f79 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:35.391327 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b739d10-cabc-4a30-bab8-d831cd989f79 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:35.392861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b739d10-cabc-4a30-bab8-d831cd989f79 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-6b739d10-cabc-4a30-bab8-d831cd989f79 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:35.409174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:35.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:35.410971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:35.410971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:35.410971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:35.410971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:35.410971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:35.410971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:35.410971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:35.410971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:35.410971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:35.410971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:35.410971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:35.410971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:35.410971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:51:35.410971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:35.410971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 478/953] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:51:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:35.417608 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b739d10-cabc-4a30-bab8-d831cd989f79 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:35.426314 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 476/954] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:35.448697 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5decbc64-fbe7-460d-bce7-139dfa81b59f None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:35.456897 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:35.483130 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:35.530667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b739d10-cabc-4a30-bab8-d831cd989f79 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:35.565494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c633f2-29d1-4dab-bb0d-730d9439b5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:35.566168 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c633f2-29d1-4dab-bb0d-730d9439b5ed None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:35.566496 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c633f2-29d1-4dab-bb0d-730d9439b5ed None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:35.566775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c633f2-29d1-4dab-bb0d-730d9439b5ed None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:35.567497 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c633f2-29d1-4dab-bb0d-730d9439b5ed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:35.567725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5decbc64-fbe7-460d-bce7-139dfa81b59f None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:35.567925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c633f2-29d1-4dab-bb0d-730d9439b5ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:35.569166 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c633f2-29d1-4dab-bb0d-730d9439b5ed None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:35.574596 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 479/955] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:51:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:35.580991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c633f2-29d1-4dab-bb0d-730d9439b5ed None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:35.614104 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5103fa0-5f68-44e8-b9ab-13d3a0e54664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:35.614950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5103fa0-5f68-44e8-b9ab-13d3a0e54664 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:35.615238 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5103fa0-5f68-44e8-b9ab-13d3a0e54664 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:35.615655 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5103fa0-5f68-44e8-b9ab-13d3a0e54664 None admin] PATH_INFO: `/v3/services/compute` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:35.616524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5103fa0-5f68-44e8-b9ab-13d3a0e54664 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:35.618142 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5103fa0-5f68-44e8-b9ab-13d3a0e54664 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-c5103fa0-5f68-44e8-b9ab-13d3a0e54664 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:35.633928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:35.634882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:35.635642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:35.635642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:35.635642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:35.635642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:35.635642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:35.635642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:35.635642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:35.635642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:35.635642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:35.635642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:35.635642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:35.635642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:35.635642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:51:35.635642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:35.635642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 477/956] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:35.643312 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5103fa0-5f68-44e8-b9ab-13d3a0e54664 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:35.719134 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c633f2-29d1-4dab-bb0d-730d9439b5ed None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:35.738944 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:35.741371 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:35.759017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d3c3d-d00a-4e86-8b8d-979b7304183f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:35.760502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d3c3d-d00a-4e86-8b8d-979b7304183f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:35.760976 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d3c3d-d00a-4e86-8b8d-979b7304183f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:35.761488 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d3c3d-d00a-4e86-8b8d-979b7304183f None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:35.762535 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d3c3d-d00a-4e86-8b8d-979b7304183f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:35.763048 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d3c3d-d00a-4e86-8b8d-979b7304183f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:35.764736 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d3c3d-d00a-4e86-8b8d-979b7304183f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:35.770811 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 480/957] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:51:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:35.776723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d3c3d-d00a-4e86-8b8d-979b7304183f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:35.786805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5103fa0-5f68-44e8-b9ab-13d3a0e54664 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:35.824894 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c7ab1-eb42-412b-a5b7-7e2f8ed47407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:35.825548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c7ab1-eb42-412b-a5b7-7e2f8ed47407 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:35.826138 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c7ab1-eb42-412b-a5b7-7e2f8ed47407 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:35.826138 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c7ab1-eb42-412b-a5b7-7e2f8ed47407 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:35.826902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c7ab1-eb42-412b-a5b7-7e2f8ed47407 None admin] RBAC: Adding query filter params (name=compute) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:35.827209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c7ab1-eb42-412b-a5b7-7e2f8ed47407 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:35.828753 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c7ab1-eb42-412b-a5b7-7e2f8ed47407 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:35.840199 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 478/958] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:51:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:35.850496 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c7ab1-eb42-412b-a5b7-7e2f8ed47407 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:35.900997 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d3c3d-d00a-4e86-8b8d-979b7304183f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:35.937348 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950d8a0-f697-441f-9f84-f024f72a77ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:35.938013 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950d8a0-f697-441f-9f84-f024f72a77ba None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:35.938312 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950d8a0-f697-441f-9f84-f024f72a77ba None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:35.938589 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950d8a0-f697-441f-9f84-f024f72a77ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:35.939340 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2950d8a0-f697-441f-9f84-f024f72a77ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:35.941037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2950d8a0-f697-441f-9f84-f024f72a77ba None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:35.970290 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 481/959] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:51:35 2022] POST /identity/v3/endpoints => generated 349 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:51:35.978519 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c7ab1-eb42-412b-a5b7-7e2f8ed47407 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:36.020475 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:36.028256 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d29bcc-a477-4747-8558-8091106b7a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:36.028987 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d29bcc-a477-4747-8558-8091106b7a26 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:36.029267 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d29bcc-a477-4747-8558-8091106b7a26 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:36.029539 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d29bcc-a477-4747-8558-8091106b7a26 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:36.030382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d29bcc-a477-4747-8558-8091106b7a26 None admin] RBAC: Adding query filter params (type=compute) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:36.030693 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d29bcc-a477-4747-8558-8091106b7a26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:36.032331 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d29bcc-a477-4747-8558-8091106b7a26 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:36.038006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 479/960] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:51:35 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:36.299557 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58ea63-093f-40fc-a5f9-41b25c0d0289 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:36.299842 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58ea63-093f-40fc-a5f9-41b25c0d0289 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:36.300029 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58ea63-093f-40fc-a5f9-41b25c0d0289 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:36.301570 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 482/961] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:36.307324 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1f3dd-841d-42b4-a3ba-c25a25160c72 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:36.307892 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1f3dd-841d-42b4-a3ba-c25a25160c72 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:36.308367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1f3dd-841d-42b4-a3ba-c25a25160c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:36.327627 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-50b1f3dd-841d-42b4-a3ba-c25a25160c72 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:36.329149 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-50b1f3dd-841d-42b4-a3ba-c25a25160c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:36.406363 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 480/962] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:36.424159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58ea63-093f-40fc-a5f9-41b25c0d0289 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:36.470042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:36.536643 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58ea63-093f-40fc-a5f9-41b25c0d0289 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:36.572214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ae5e9-652b-48d9-b972-1eb239b99a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:36.573217 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ae5e9-652b-48d9-b972-1eb239b99a9d None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:36.573687 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ae5e9-652b-48d9-b972-1eb239b99a9d None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:36.574165 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ae5e9-652b-48d9-b972-1eb239b99a9d None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:36.575093 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ae5e9-652b-48d9-b972-1eb239b99a9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:36.576873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ae5e9-652b-48d9-b972-1eb239b99a9d None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:36.591794 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.sql.core [None req-1f6ae5e9-652b-48d9-b972-1eb239b99a9d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 19:51:36.591794 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 19:51:36.591794 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [parameters: {'user_id': '2b61b31363594be196f48a2cb33a0454', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 19:51:36.591794 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=150548) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 19:51:36.600588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-1f6ae5e9-652b-48d9-b972-1eb239b99a9d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 19:51:36.600588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:36.600588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:51:36.600588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:51:36.600588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:51:36.600588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:51:36.600588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:51:36.600588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:51:36.600588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:51:36.600588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application conn.query(q) May 19 19:51:36.600588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:51:36.600588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:51:36.600588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:51:36.600588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application result.read() May 19 19:51:36.600588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:51:36.601363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application next(self.gen) May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application yield resource May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application next(self.gen) May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self.session.rollback() May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self.force_reraise() May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise self.value May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application session.commit() May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 19:51:36.602245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._prepare_impl() May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self.session.flush() May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._flush(objects) May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application compat.raise_( May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application flush_context.execute() May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rec.execute(self) May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 19:51:36.603543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:51:36.604355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application conn.query(q) May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application result.read() May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:51:36.605113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:51:36.605870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:51:36.605870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:51:36.605870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:51:36.605870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:51:36.605870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 19:51:36.605870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 19:51:36.605870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [parameters: {'user_id': '2b61b31363594be196f48a2cb33a0454', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 19:51:36.605870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 19:51:36.605870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:36.605870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:36.605870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:36.605870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:36.605870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:36.605870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:36.605870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 19:51:36.607925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 19:51:36.610012 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 19:51:36.610012 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 19:51:36.610012 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 19:51:36.610012 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 19:51:36.610012 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 19:51:36.610012 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:36.612689 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 483/963] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 19:51:36 2022] POST /identity/v3/users => generated 160 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:36.620395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1f3dd-841d-42b4-a3ba-c25a25160c72 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:36.732519 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1f3dd-841d-42b4-a3ba-c25a25160c72 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:36.758281 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:36.769437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b16a1c-8414-413f-9070-d4c05d72f497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:36.770695 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b16a1c-8414-413f-9070-d4c05d72f497 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:36.771210 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b16a1c-8414-413f-9070-d4c05d72f497 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:36.771765 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b16a1c-8414-413f-9070-d4c05d72f497 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:36.780815 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b16a1c-8414-413f-9070-d4c05d72f497 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:36.782297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b16a1c-8414-413f-9070-d4c05d72f497 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-63b16a1c-8414-413f-9070-d4c05d72f497 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:36.798097 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:36.799038 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:36.799871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:36.799871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:36.799871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:36.799871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:36.799871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:36.799871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:36.799871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:36.799871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:36.799871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:51:36.799871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:36.799871 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 481/964] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:51:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:36.805654 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ae5e9-652b-48d9-b972-1eb239b99a9d None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:36.920974 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ae5e9-652b-48d9-b972-1eb239b99a9d None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:36.923003 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:36.965597 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70eab7-4cad-4de5-b998-2dce6a8e85f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:36.966324 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70eab7-4cad-4de5-b998-2dce6a8e85f8 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:36.966775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70eab7-4cad-4de5-b998-2dce6a8e85f8 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:36.967059 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70eab7-4cad-4de5-b998-2dce6a8e85f8 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:36.967895 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70eab7-4cad-4de5-b998-2dce6a8e85f8 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:36.968202 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70eab7-4cad-4de5-b998-2dce6a8e85f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:36.969872 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70eab7-4cad-4de5-b998-2dce6a8e85f8 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:37.010113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 484/965] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 19:51:36 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:37.377312 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2b13b-aa3b-4c1b-888c-246cdc24aaf1 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:37.377854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2b13b-aa3b-4c1b-888c-246cdc24aaf1 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:37.377854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2b13b-aa3b-4c1b-888c-246cdc24aaf1 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:37.379480 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 482/966] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:37.385106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732490e1-428c-4530-b3f6-a5844edb5bdc None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:37.385403 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732490e1-428c-4530-b3f6-a5844edb5bdc None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:37.385685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732490e1-428c-4530-b3f6-a5844edb5bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:37.404316 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-732490e1-428c-4530-b3f6-a5844edb5bdc None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:37.406008 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-732490e1-428c-4530-b3f6-a5844edb5bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:37.485888 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 485/967] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:37.507150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2b13b-aa3b-4c1b-888c-246cdc24aaf1 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:37.614484 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2b13b-aa3b-4c1b-888c-246cdc24aaf1 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:37.648995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887acf7-a34f-48bb-be69-1d05eb76e2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:37.649745 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887acf7-a34f-48bb-be69-1d05eb76e2da None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:37.650026 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887acf7-a34f-48bb-be69-1d05eb76e2da None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:37.650461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887acf7-a34f-48bb-be69-1d05eb76e2da None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:37.661239 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887acf7-a34f-48bb-be69-1d05eb76e2da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-c887acf7-a34f-48bb-be69-1d05eb76e2da None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:37.669489 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:37.670582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:37.671361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:37.671361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:37.671361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:37.671361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:37.671361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:37.671361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:37.671361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:37.671361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:37.671361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:37.671361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:37.671361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:51:37.671361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:37.674644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 483/968] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 19:51:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:37.676812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732490e1-428c-4530-b3f6-a5844edb5bdc None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:37.772754 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:37.783804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732490e1-428c-4530-b3f6-a5844edb5bdc None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:37.819981 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ff472-2d9a-43f6-9679-68e3e3588460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:37.820746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ff472-2d9a-43f6-9679-68e3e3588460 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:37.820995 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ff472-2d9a-43f6-9679-68e3e3588460 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:37.821273 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ff472-2d9a-43f6-9679-68e3e3588460 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:37.822032 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ff472-2d9a-43f6-9679-68e3e3588460 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:37.822378 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ff472-2d9a-43f6-9679-68e3e3588460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:37.823746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ff472-2d9a-43f6-9679-68e3e3588460 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:37.833067 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 486/969] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 19:51:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:37.838739 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887acf7-a34f-48bb-be69-1d05eb76e2da None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:37.888251 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:37.946371 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887acf7-a34f-48bb-be69-1d05eb76e2da None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:37.980905 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262aa899-897d-4b30-9523-a773a53f4c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:37.981696 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262aa899-897d-4b30-9523-a773a53f4c6b None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:37.981994 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262aa899-897d-4b30-9523-a773a53f4c6b None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:37.982333 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262aa899-897d-4b30-9523-a773a53f4c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:37.993260 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262aa899-897d-4b30-9523-a773a53f4c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:37.994744 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262aa899-897d-4b30-9523-a773a53f4c6b None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:38.011945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-262aa899-897d-4b30-9523-a773a53f4c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:38.011945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:38.011945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:38.011945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:38.011945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:38.011945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:38.011945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:38.011945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:38.011945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:38.011945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:38.011945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:38.011945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:38.011945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:38.011945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:38.011945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:38.012779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:38.013715 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:38.013715 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:38.013715 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:38.013715 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:38.017915 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 484/970] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:38.019236 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ff472-2d9a-43f6-9679-68e3e3588460 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:38.134873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ff472-2d9a-43f6-9679-68e3e3588460 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:38.169704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8652615-c1e9-47de-b2e4-c68eccdecf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:38.170421 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8652615-c1e9-47de-b2e4-c68eccdecf15 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:38.170705 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8652615-c1e9-47de-b2e4-c68eccdecf15 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:38.170983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8652615-c1e9-47de-b2e4-c68eccdecf15 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:38.171721 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8652615-c1e9-47de-b2e4-c68eccdecf15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:38.172029 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8652615-c1e9-47de-b2e4-c68eccdecf15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:38.173404 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8652615-c1e9-47de-b2e4-c68eccdecf15 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:38.185962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 487/971] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:38.191576 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262aa899-897d-4b30-9523-a773a53f4c6b None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:38.255285 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdc08d-ce32-4f3a-b5f2-092666a477c0 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:38.255580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdc08d-ce32-4f3a-b5f2-092666a477c0 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:38.255768 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdc08d-ce32-4f3a-b5f2-092666a477c0 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:38.257294 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 488/972] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:38.262118 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258b0fd-313f-471a-8fa8-9fa53de2fdb5 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:38.262408 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258b0fd-313f-471a-8fa8-9fa53de2fdb5 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:38.262727 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258b0fd-313f-471a-8fa8-9fa53de2fdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:38.279481 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-b258b0fd-313f-471a-8fa8-9fa53de2fdb5 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:38.280902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-b258b0fd-313f-471a-8fa8-9fa53de2fdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:38.289697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:38.301268 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:38.308470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262aa899-897d-4b30-9523-a773a53f4c6b None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:38.344263 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e8f2f-01b4-4f22-a182-67402da6e281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:38.345092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e8f2f-01b4-4f22-a182-67402da6e281 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:38.345373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e8f2f-01b4-4f22-a182-67402da6e281 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:38.345646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e8f2f-01b4-4f22-a182-67402da6e281 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:38.354793 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e8f2f-01b4-4f22-a182-67402da6e281 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:38.356278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e8f2f-01b4-4f22-a182-67402da6e281 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:38.359326 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 489/973] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-454e8f2f-01b4-4f22-a182-67402da6e281 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:38.371433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:38.372909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:38.374230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:38.374230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:38.374230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:38.374230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:38.374230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:38.374230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:38.374230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:38.374230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:38.374230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:51:38.374230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:38.375047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 485/974] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:51:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:38.378477 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258b0fd-313f-471a-8fa8-9fa53de2fdb5 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:38.384609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e8f2f-01b4-4f22-a182-67402da6e281 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:38.479144 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258b0fd-313f-471a-8fa8-9fa53de2fdb5 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:38.485736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e8f2f-01b4-4f22-a182-67402da6e281 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:38.513022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f65fb9-bdf4-4b97-b36a-76b58f42262c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:38.513680 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f65fb9-bdf4-4b97-b36a-76b58f42262c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:38.513962 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f65fb9-bdf4-4b97-b36a-76b58f42262c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:38.514233 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f65fb9-bdf4-4b97-b36a-76b58f42262c None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:38.515137 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f65fb9-bdf4-4b97-b36a-76b58f42262c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:38.515447 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f65fb9-bdf4-4b97-b36a-76b58f42262c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:38.516910 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f65fb9-bdf4-4b97-b36a-76b58f42262c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:38.518826 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7f26d-e7a2-4220-ad3a-3b2256e9554b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:38.519625 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7f26d-e7a2-4220-ad3a-3b2256e9554b None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:38.519901 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7f26d-e7a2-4220-ad3a-3b2256e9554b None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:38.520176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7f26d-e7a2-4220-ad3a-3b2256e9554b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:38.520840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7f26d-e7a2-4220-ad3a-3b2256e9554b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:38.522189 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7f26d-e7a2-4220-ad3a-3b2256e9554b None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:38.531492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-e3f7f26d-e7a2-4220-ad3a-3b2256e9554b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:38.534639 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:38.535604 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:38.536375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:38.536375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:38.536375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:38.536375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:38.536375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:38.536375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:38.536375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:38.536375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:38.536375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:38.536375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:38.536375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:38.536375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:38.536375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:51:38.536375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:38.536375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 486/975] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:51:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:38.537104 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b4531-1ab2-4b84-957f-af45105a6c9e None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:38.537645 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b4531-1ab2-4b84-957f-af45105a6c9e None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:38.537645 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b4531-1ab2-4b84-957f-af45105a6c9e None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:38.538006 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 490/976] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 19:51:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:38.539029 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 487/977] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:38.540805 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f65fb9-bdf4-4b97-b36a-76b58f42262c None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:38.543209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b4531-1ab2-4b84-957f-af45105a6c9e None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:38.567325 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:38.642160 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f65fb9-bdf4-4b97-b36a-76b58f42262c None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:38.646794 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b4531-1ab2-4b84-957f-af45105a6c9e None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:38.678582 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a2e8a-504d-47be-b324-921ff97dc749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:38.679242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a2e8a-504d-47be-b324-921ff97dc749 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:38.679528 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a2e8a-504d-47be-b324-921ff97dc749 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:38.679806 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a2e8a-504d-47be-b324-921ff97dc749 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:38.681200 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4d5b6-e951-4e09-bc0b-7e0a3c176626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:38.681862 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4d5b6-e951-4e09-bc0b-7e0a3c176626 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:38.682143 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4d5b6-e951-4e09-bc0b-7e0a3c176626 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:38.682452 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4d5b6-e951-4e09-bc0b-7e0a3c176626 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:38.683164 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4d5b6-e951-4e09-bc0b-7e0a3c176626 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:38.683473 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4d5b6-e951-4e09-bc0b-7e0a3c176626 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:38.684848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4d5b6-e951-4e09-bc0b-7e0a3c176626 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:38.690539 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 488/978] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:51:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:38.691816 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d267f5-0b04-4179-baaa-30228834e79e None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:38.692185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d267f5-0b04-4179-baaa-30228834e79e None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:38.692445 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d267f5-0b04-4179-baaa-30228834e79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:38.711838 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-46d267f5-0b04-4179-baaa-30228834e79e None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:38.713014 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-46d267f5-0b04-4179-baaa-30228834e79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:38.753156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:38.755967 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a2e8a-504d-47be-b324-921ff97dc749 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:38.757952 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a2e8a-504d-47be-b324-921ff97dc749 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:38.781620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 489/979] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:38.782499 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d267f5-0b04-4179-baaa-30228834e79e None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:38.808123 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:38.858774 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 491/980] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:38 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:51:38.859716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a2e8a-504d-47be-b324-921ff97dc749 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:38.894383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d267f5-0b04-4179-baaa-30228834e79e None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:38.930187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa275a4-2d31-4ef0-bace-f3957630deaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:38.930896 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa275a4-2d31-4ef0-bace-f3957630deaf None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:38.931181 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa275a4-2d31-4ef0-bace-f3957630deaf None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:38.931460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa275a4-2d31-4ef0-bace-f3957630deaf None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:38.932183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa275a4-2d31-4ef0-bace-f3957630deaf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:38.932500 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa275a4-2d31-4ef0-bace-f3957630deaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:38.933901 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa275a4-2d31-4ef0-bace-f3957630deaf None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:38.939007 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 490/981] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:51:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:38.939792 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa275a4-2d31-4ef0-bace-f3957630deaf None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:38.955121 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:38.957891 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a2e8a-504d-47be-b324-921ff97dc749 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:38.992911 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf3b3f-948d-4a83-9401-fb517ddfae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:38.993580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf3b3f-948d-4a83-9401-fb517ddfae67 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:38.993863 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf3b3f-948d-4a83-9401-fb517ddfae67 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:38.994136 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf3b3f-948d-4a83-9401-fb517ddfae67 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:38.994902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf3b3f-948d-4a83-9401-fb517ddfae67 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:38.996277 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf3b3f-948d-4a83-9401-fb517ddfae67 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:39.010397 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 492/982] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:51:38 2022] POST /identity/v3/services => generated 242 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:51:39.011408 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf3b3f-948d-4a83-9401-fb517ddfae67 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:39.025536 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:39.053163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa275a4-2d31-4ef0-bace-f3957630deaf None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:39.102788 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29f6c1-4877-4b9c-b1d3-a45a660b2bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:39.103836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29f6c1-4877-4b9c-b1d3-a45a660b2bd7 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:39.104121 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29f6c1-4877-4b9c-b1d3-a45a660b2bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:39.104397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29f6c1-4877-4b9c-b1d3-a45a660b2bd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:39.116539 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29f6c1-4877-4b9c-b1d3-a45a660b2bd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:39.118181 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29f6c1-4877-4b9c-b1d3-a45a660b2bd7 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:39.126838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf3b3f-948d-4a83-9401-fb517ddfae67 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:39.138115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-3e29f6c1-4877-4b9c-b1d3-a45a660b2bd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:39.138115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:39.138115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:39.138115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:39.138115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:39.138115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:39.138115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:39.138115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:39.138115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:39.138115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:39.138115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:39.138115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:39.138115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:39.138115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:39.138115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:39.139424 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:39.140371 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:39.140371 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:39.140371 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:39.140371 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:39.142774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 491/983] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:39.145247 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29f6c1-4877-4b9c-b1d3-a45a660b2bd7 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:39.171197 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807049f0-228f-456e-a4c3-10a56a8fa30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:39.171543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807049f0-228f-456e-a4c3-10a56a8fa30f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:39.171843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807049f0-228f-456e-a4c3-10a56a8fa30f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:39.172128 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807049f0-228f-456e-a4c3-10a56a8fa30f None admin] PATH_INFO: `/v3/endpoints` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:39.173051 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807049f0-228f-456e-a4c3-10a56a8fa30f None admin] RBAC: Adding query filter params (interface=admin, service_id=f4f472fecdae4fa088176a6249dbb33a, region_id=RegionOne) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:39.173359 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807049f0-228f-456e-a4c3-10a56a8fa30f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:39.174782 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807049f0-228f-456e-a4c3-10a56a8fa30f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:39.185027 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 493/984] 10.222.0.84 () {56 vars in 1249 bytes} [Thu May 19 19:51:39 2022] GET /identity/v3/endpoints?service_id=f4f472fecdae4fa088176a6249dbb33a&interface=admin®ion_id=RegionOne => generated 193 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:39.186381 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807049f0-228f-456e-a4c3-10a56a8fa30f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:39.218822 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:39.262390 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29f6c1-4877-4b9c-b1d3-a45a660b2bd7 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:39.290935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:39.301917 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807049f0-228f-456e-a4c3-10a56a8fa30f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:39.303045 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e671b-86e8-4a2b-b0ec-a5d87b109ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:39.303885 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e671b-86e8-4a2b-b0ec-a5d87b109ec6 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:39.304178 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e671b-86e8-4a2b-b0ec-a5d87b109ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:39.304459 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e671b-86e8-4a2b-b0ec-a5d87b109ec6 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:39.305302 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e671b-86e8-4a2b-b0ec-a5d87b109ec6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:39.305616 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e671b-86e8-4a2b-b0ec-a5d87b109ec6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:39.307030 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e671b-86e8-4a2b-b0ec-a5d87b109ec6 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:39.319320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 492/985] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:39.324741 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e671b-86e8-4a2b-b0ec-a5d87b109ec6 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:39.343559 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d6392-0e8a-4cab-bc05-07fdd14c55c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:39.344228 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d6392-0e8a-4cab-bc05-07fdd14c55c2 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:39.344508 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d6392-0e8a-4cab-bc05-07fdd14c55c2 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:39.344775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d6392-0e8a-4cab-bc05-07fdd14c55c2 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:39.345461 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d6392-0e8a-4cab-bc05-07fdd14c55c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:39.347195 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d6392-0e8a-4cab-bc05-07fdd14c55c2 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:39.362299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 494/986] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:51:39 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:51:39.449876 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e671b-86e8-4a2b-b0ec-a5d87b109ec6 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:39.488158 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e60bcf-584b-40ef-852c-cb144b3c9731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:39.488986 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e60bcf-584b-40ef-852c-cb144b3c9731 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:39.489274 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e60bcf-584b-40ef-852c-cb144b3c9731 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:39.489548 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e60bcf-584b-40ef-852c-cb144b3c9731 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:39.497479 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e60bcf-584b-40ef-852c-cb144b3c9731 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:39.499165 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e60bcf-584b-40ef-852c-cb144b3c9731 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:39.515127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-b2e60bcf-584b-40ef-852c-cb144b3c9731 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:39.515127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:39.515127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:39.515127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:39.515127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:39.515127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:39.515127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:39.515127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:39.515127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:39.515127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:39.515127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:39.515127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:39.515127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:39.515127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:39.515127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:39.516031 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:39.517009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:39.517009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:39.517009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:39.517009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:39.517009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:39.517009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:39.517009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:39.517009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:39.517009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 493/987] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:39.522423 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d6392-0e8a-4cab-bc05-07fdd14c55c2 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:39.617919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:39.630856 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d6392-0e8a-4cab-bc05-07fdd14c55c2 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:39.664641 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42669b-3fea-4dbf-b1c7-ab866daf6c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:39.665306 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42669b-3fea-4dbf-b1c7-ab866daf6c8c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:39.665591 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42669b-3fea-4dbf-b1c7-ab866daf6c8c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:39.665868 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42669b-3fea-4dbf-b1c7-ab866daf6c8c None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:39.666709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42669b-3fea-4dbf-b1c7-ab866daf6c8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:39.667020 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42669b-3fea-4dbf-b1c7-ab866daf6c8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:39.668444 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42669b-3fea-4dbf-b1c7-ab866daf6c8c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:39.680856 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 495/988] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:39.686786 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e60bcf-584b-40ef-852c-cb144b3c9731 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:39.747448 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:39.804048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e60bcf-584b-40ef-852c-cb144b3c9731 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:39.838647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bccd61-6735-483c-828f-a434a41c91e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:39.839312 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bccd61-6735-483c-828f-a434a41c91e0 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:39.839589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bccd61-6735-483c-828f-a434a41c91e0 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:39.839865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bccd61-6735-483c-828f-a434a41c91e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:39.840710 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bccd61-6735-483c-828f-a434a41c91e0 None admin] RBAC: Adding query filter params (role.id=206b3ccf6eb842fc8e1d86592c6500b7, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=8683be51c17341f7b0e85d366a74869f) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:39.841022 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bccd61-6735-483c-828f-a434a41c91e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:39.842700 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bccd61-6735-483c-828f-a434a41c91e0 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:39.849264 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 494/989] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:39 2022] GET /identity/v3/role_assignments?user.id=8683be51c17341f7b0e85d366a74869f&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=206b3ccf6eb842fc8e1d86592c6500b7 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:41.291925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ae27a-ff60-4293-af98-b157a88593f2 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:41.292440 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ae27a-ff60-4293-af98-b157a88593f2 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:41.292514 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ae27a-ff60-4293-af98-b157a88593f2 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:41.293955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 496/990] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:41.299569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6787b-24a4-4edc-8b80-e9a94bd63c78 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:41.300022 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6787b-24a4-4edc-8b80-e9a94bd63c78 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:41.300096 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6787b-24a4-4edc-8b80-e9a94bd63c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:41.317877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-a9f6787b-24a4-4edc-8b80-e9a94bd63c78 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:41.319125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-a9f6787b-24a4-4edc-8b80-e9a94bd63c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:41.391807 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 495/991] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:41.415380 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ae27a-ff60-4293-af98-b157a88593f2 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:41.526790 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ae27a-ff60-4293-af98-b157a88593f2 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:41.568852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2744b78-a022-4bab-8d20-c06ed648cb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:41.569561 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2744b78-a022-4bab-8d20-c06ed648cb1a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:41.569839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2744b78-a022-4bab-8d20-c06ed648cb1a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:41.570116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2744b78-a022-4bab-8d20-c06ed648cb1a None admin] PATH_INFO: `/v3/services/compute` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:41.570807 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2744b78-a022-4bab-8d20-c06ed648cb1a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:41.572169 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2744b78-a022-4bab-8d20-c06ed648cb1a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-b2744b78-a022-4bab-8d20-c06ed648cb1a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:41.583924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:41.584843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:41.585607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:41.585607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:41.585607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:41.585607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:41.585607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:41.585607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:41.585607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:41.585607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:41.585607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:41.585607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:41.585607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:41.585607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:41.585607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:51:41.585607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:41.585607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 497/992] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:41.588546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9753b-318e-450e-9951-e2f900c5b414 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:41.588915 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9753b-318e-450e-9951-e2f900c5b414 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:41.589264 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9753b-318e-450e-9951-e2f900c5b414 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:41.590969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 496/993] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:41.591292 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2744b78-a022-4bab-8d20-c06ed648cb1a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:41.597463 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e66e7c-1057-401f-9b10-1ae6ef1da4d7 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:41.597757 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e66e7c-1057-401f-9b10-1ae6ef1da4d7 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:41.598040 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e66e7c-1057-401f-9b10-1ae6ef1da4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:41.608850 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:41.617085 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-36e66e7c-1057-401f-9b10-1ae6ef1da4d7 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:41.618378 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-36e66e7c-1057-401f-9b10-1ae6ef1da4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:41.659850 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:41.696182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 497/994] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:41.709179 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2744b78-a022-4bab-8d20-c06ed648cb1a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:41.720599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e66e7c-1057-401f-9b10-1ae6ef1da4d7 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:41.750878 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564e27a-b8a9-4155-9b02-3ebd14b82347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:41.752156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564e27a-b8a9-4155-9b02-3ebd14b82347 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:41.753018 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564e27a-b8a9-4155-9b02-3ebd14b82347 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:41.753720 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564e27a-b8a9-4155-9b02-3ebd14b82347 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:41.754638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564e27a-b8a9-4155-9b02-3ebd14b82347 None admin] RBAC: Adding query filter params (name=compute) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:41.755115 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564e27a-b8a9-4155-9b02-3ebd14b82347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:41.757598 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564e27a-b8a9-4155-9b02-3ebd14b82347 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:41.763489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 498/995] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:51:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:41.770159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564e27a-b8a9-4155-9b02-3ebd14b82347 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:41.824127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e66e7c-1057-401f-9b10-1ae6ef1da4d7 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:41.857472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:41.863566 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680b4fd-ebda-458c-8e2f-78d7e7ef7a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:41.864406 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680b4fd-ebda-458c-8e2f-78d7e7ef7a66 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:41.864691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680b4fd-ebda-458c-8e2f-78d7e7ef7a66 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:41.864966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680b4fd-ebda-458c-8e2f-78d7e7ef7a66 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:41.865633 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680b4fd-ebda-458c-8e2f-78d7e7ef7a66 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:41.867052 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680b4fd-ebda-458c-8e2f-78d7e7ef7a66 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-d680b4fd-ebda-458c-8e2f-78d7e7ef7a66 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:41.880041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:41.880945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:41.881880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:41.881880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:41.881880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:41.881880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:41.881880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:41.881880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:41.881880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:41.881880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:41.881880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:41.881880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:41.881880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:41.881880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:41.881880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:51:41.881880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:41.881880 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 498/996] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 19:51:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:41.886685 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564e27a-b8a9-4155-9b02-3ebd14b82347 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:41.887351 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680b4fd-ebda-458c-8e2f-78d7e7ef7a66 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:41.922886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:41.931377 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7be7d-3f55-4105-916d-ca32b69c7fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:41.932071 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7be7d-3f55-4105-916d-ca32b69c7fea None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:41.932353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7be7d-3f55-4105-916d-ca32b69c7fea None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:41.932958 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7be7d-3f55-4105-916d-ca32b69c7fea None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:41.933591 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7be7d-3f55-4105-916d-ca32b69c7fea None admin] RBAC: Adding query filter params (type=compute) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:41.933905 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7be7d-3f55-4105-916d-ca32b69c7fea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:41.935754 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7be7d-3f55-4105-916d-ca32b69c7fea None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:41.946746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 499/997] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:51:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:41.948874 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7be7d-3f55-4105-916d-ca32b69c7fea None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:41.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680b4fd-ebda-458c-8e2f-78d7e7ef7a66 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:42.034990 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0560f53-a251-4bb9-b4a8-79658fe97c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:42.035894 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0560f53-a251-4bb9-b4a8-79658fe97c33 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:42.036315 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0560f53-a251-4bb9-b4a8-79658fe97c33 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:42.036726 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0560f53-a251-4bb9-b4a8-79658fe97c33 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:42.037724 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0560f53-a251-4bb9-b4a8-79658fe97c33 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:42.038220 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0560f53-a251-4bb9-b4a8-79658fe97c33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:42.039752 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0560f53-a251-4bb9-b4a8-79658fe97c33 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:42.049759 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 499/998] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:51:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:42.050843 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0560f53-a251-4bb9-b4a8-79658fe97c33 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:42.053101 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7be7d-3f55-4105-916d-ca32b69c7fea None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:42.091901 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe12b64-2afb-45f3-93bb-5e2b2cea136f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:42.092559 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe12b64-2afb-45f3-93bb-5e2b2cea136f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:42.092848 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe12b64-2afb-45f3-93bb-5e2b2cea136f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:42.093127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe12b64-2afb-45f3-93bb-5e2b2cea136f None admin] PATH_INFO: `/v3/endpoints` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:42.093927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe12b64-2afb-45f3-93bb-5e2b2cea136f None admin] RBAC: Adding query filter params (interface=public, service_id=ad994a9f81c74ec583cce618100baf06, region_id=RegionOne) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:42.094243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe12b64-2afb-45f3-93bb-5e2b2cea136f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:42.095681 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe12b64-2afb-45f3-93bb-5e2b2cea136f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:42.101598 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 500/999] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:51:41 2022] GET /identity/v3/endpoints?service_id=ad994a9f81c74ec583cce618100baf06&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:42.107640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4b733-b161-49c8-9ac8-4029b1bc0022 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:42.108016 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4b733-b161-49c8-9ac8-4029b1bc0022 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:42.108301 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4b733-b161-49c8-9ac8-4029b1bc0022 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:42.109607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 501/1000] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:51:42.110358 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4b733-b161-49c8-9ac8-4029b1bc0022 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:42.114126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:42.164508 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:42.168564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0560f53-a251-4bb9-b4a8-79658fe97c33 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:42.203112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b98f0d-7ea0-4cd4-b553-85ce00d1aede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:42.204009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b98f0d-7ea0-4cd4-b553-85ce00d1aede None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:42.204287 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b98f0d-7ea0-4cd4-b553-85ce00d1aede None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:42.204559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b98f0d-7ea0-4cd4-b553-85ce00d1aede None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:42.205280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b98f0d-7ea0-4cd4-b553-85ce00d1aede None admin] RBAC: Adding query filter params (type=object-store) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:42.205585 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b98f0d-7ea0-4cd4-b553-85ce00d1aede None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:42.206985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b98f0d-7ea0-4cd4-b553-85ce00d1aede None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:42.212322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 500/1001] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 19:51:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:42.213344 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b89f41-ae0f-49d1-a78e-f76b9f92c288 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:42.213626 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b89f41-ae0f-49d1-a78e-f76b9f92c288 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:42.213716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4b733-b161-49c8-9ac8-4029b1bc0022 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:42.213898 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b89f41-ae0f-49d1-a78e-f76b9f92c288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:42.230470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-62b89f41-ae0f-49d1-a78e-f76b9f92c288 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:42.231682 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-62b89f41-ae0f-49d1-a78e-f76b9f92c288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:42.248675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142605d-8a8e-4445-a03b-e33926ff3ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:42.249344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142605d-8a8e-4445-a03b-e33926ff3ea5 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:42.249627 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142605d-8a8e-4445-a03b-e33926ff3ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:42.249906 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142605d-8a8e-4445-a03b-e33926ff3ea5 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:42.250614 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142605d-8a8e-4445-a03b-e33926ff3ea5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:42.251975 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142605d-8a8e-4445-a03b-e33926ff3ea5 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:42.258559 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 502/1002] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:51:42 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:51:42.259162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142605d-8a8e-4445-a03b-e33926ff3ea5 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:42.303276 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 501/1003] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:42.325828 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b89f41-ae0f-49d1-a78e-f76b9f92c288 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:42.351813 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:42.392202 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142605d-8a8e-4445-a03b-e33926ff3ea5 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:42.433243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d771ae-5cfe-42f5-ac1e-f14fc90705cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:42.433455 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:42.434368 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d771ae-5cfe-42f5-ac1e-f14fc90705cd None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:42.434826 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d771ae-5cfe-42f5-ac1e-f14fc90705cd None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:42.435870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d771ae-5cfe-42f5-ac1e-f14fc90705cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:42.436800 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d771ae-5cfe-42f5-ac1e-f14fc90705cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:42.438166 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d771ae-5cfe-42f5-ac1e-f14fc90705cd None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:42.443735 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b89f41-ae0f-49d1-a78e-f76b9f92c288 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:42.457965 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 503/1004] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:51:42 2022] POST /identity/v3/endpoints => generated 325 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:51:42.493884 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73d5b9-71ff-4327-8dca-c7eb01259a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:42.494619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73d5b9-71ff-4327-8dca-c7eb01259a44 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:42.494903 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73d5b9-71ff-4327-8dca-c7eb01259a44 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:42.495176 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73d5b9-71ff-4327-8dca-c7eb01259a44 None admin] PATH_INFO: `/v3/services/image` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:42.495875 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73d5b9-71ff-4327-8dca-c7eb01259a44 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:42.497329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73d5b9-71ff-4327-8dca-c7eb01259a44 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-5c73d5b9-71ff-4327-8dca-c7eb01259a44 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:42.510323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:42.511323 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:42.512106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:42.512106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:42.512106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:42.512106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:42.512106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:42.512106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:42.512106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:42.512106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:42.512106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:42.512106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:42.512106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:42.512106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:42.512106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:51:42.512106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:42.512106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 502/1005] 10.222.0.84 () {56 vars in 1105 bytes} [Thu May 19 19:51:42 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:42.518605 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d771ae-5cfe-42f5-ac1e-f14fc90705cd None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:42.631424 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d771ae-5cfe-42f5-ac1e-f14fc90705cd None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:42.662094 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:42.673875 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bc255-56ec-45dc-b33f-f6dd1b3d2e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:42.674622 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bc255-56ec-45dc-b33f-f6dd1b3d2e11 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:42.674907 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bc255-56ec-45dc-b33f-f6dd1b3d2e11 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:42.675170 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bc255-56ec-45dc-b33f-f6dd1b3d2e11 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:42.675897 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bc255-56ec-45dc-b33f-f6dd1b3d2e11 None admin] RBAC: Adding query filter params (name=image) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:42.676198 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bc255-56ec-45dc-b33f-f6dd1b3d2e11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:42.677605 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bc255-56ec-45dc-b33f-f6dd1b3d2e11 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:42.682708 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 504/1006] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:42.689307 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73d5b9-71ff-4327-8dca-c7eb01259a44 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:42.798593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73d5b9-71ff-4327-8dca-c7eb01259a44 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:42.834745 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87745555-1c20-40d3-b126-2e24bf6b2a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:42.835413 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87745555-1c20-40d3-b126-2e24bf6b2a22 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:42.835689 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87745555-1c20-40d3-b126-2e24bf6b2a22 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:42.835972 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87745555-1c20-40d3-b126-2e24bf6b2a22 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:42.836681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87745555-1c20-40d3-b126-2e24bf6b2a22 None admin] RBAC: Adding query filter params (type=image) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:42.836985 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87745555-1c20-40d3-b126-2e24bf6b2a22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:42.838383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87745555-1c20-40d3-b126-2e24bf6b2a22 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:42.843486 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 503/1007] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:42 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:44.591887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c75f9-a70a-4c69-8b90-c16359b49d06 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:44.592618 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c75f9-a70a-4c69-8b90-c16359b49d06 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:44.592618 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c75f9-a70a-4c69-8b90-c16359b49d06 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:44.594008 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 505/1008] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:44.599754 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d52fe4-077e-4a48-9e89-3ca528bb23c7 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:44.600188 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d52fe4-077e-4a48-9e89-3ca528bb23c7 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:44.600260 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d52fe4-077e-4a48-9e89-3ca528bb23c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:44.618240 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-90d52fe4-077e-4a48-9e89-3ca528bb23c7 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:44.619467 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-90d52fe4-077e-4a48-9e89-3ca528bb23c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:44.631153 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:44.693713 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 504/1009] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:44 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:44.708895 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac230a0b-e32d-4be1-ae5f-77e486bdcd13 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:44.709177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac230a0b-e32d-4be1-ae5f-77e486bdcd13 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:44.709460 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac230a0b-e32d-4be1-ae5f-77e486bdcd13 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:44.714326 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 506/1010] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:44.718301 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d52fe4-077e-4a48-9e89-3ca528bb23c7 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:44.720054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce138ed8-2760-43c0-8695-3152cacd8282 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:44.720336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce138ed8-2760-43c0-8695-3152cacd8282 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:44.720619 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce138ed8-2760-43c0-8695-3152cacd8282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:44.739200 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-ce138ed8-2760-43c0-8695-3152cacd8282 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:44.740409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-ce138ed8-2760-43c0-8695-3152cacd8282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:44.816863 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 507/1011] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:44 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:44.836881 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d52fe4-077e-4a48-9e89-3ca528bb23c7 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:44.839991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce138ed8-2760-43c0-8695-3152cacd8282 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:44.880596 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6676537-acd8-4ec2-8517-a2cbd1c5f302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:44.881412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6676537-acd8-4ec2-8517-a2cbd1c5f302 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:44.881606 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6676537-acd8-4ec2-8517-a2cbd1c5f302 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:44.881897 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6676537-acd8-4ec2-8517-a2cbd1c5f302 None admin] PATH_INFO: `/v3/services/compute` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:44.882632 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6676537-acd8-4ec2-8517-a2cbd1c5f302 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:44.884019 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6676537-acd8-4ec2-8517-a2cbd1c5f302 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-f6676537-acd8-4ec2-8517-a2cbd1c5f302 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:44.896699 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:44.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:44.898458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:44.898458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:44.898458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:44.898458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:44.898458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:44.898458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:44.898458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:44.898458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:44.898458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:44.898458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:44.898458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:44.898458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:44.898458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:51:44.898458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:44.898458 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 505/1012] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:44.904030 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6676537-acd8-4ec2-8517-a2cbd1c5f302 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:44.917044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:44.965388 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce138ed8-2760-43c0-8695-3152cacd8282 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:44.994557 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:45.007664 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f38f6-29c2-4f23-81ca-e83d4069e421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:45.008728 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f38f6-29c2-4f23-81ca-e83d4069e421 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:45.009192 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f38f6-29c2-4f23-81ca-e83d4069e421 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:45.009471 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f38f6-29c2-4f23-81ca-e83d4069e421 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:45.010732 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f38f6-29c2-4f23-81ca-e83d4069e421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:45.012173 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f38f6-29c2-4f23-81ca-e83d4069e421 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:45.013916 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6676537-acd8-4ec2-8517-a2cbd1c5f302 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:45.018428 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 508/1013] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:45.020000 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f38f6-29c2-4f23-81ca-e83d4069e421 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:45.050829 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80fb7e5-aea2-4676-b496-27cf31a02253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:45.051505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fb7e5-aea2-4676-b496-27cf31a02253 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:45.051787 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fb7e5-aea2-4676-b496-27cf31a02253 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:45.052069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fb7e5-aea2-4676-b496-27cf31a02253 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:45.052789 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80fb7e5-aea2-4676-b496-27cf31a02253 None admin] RBAC: Adding query filter params (name=compute) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:45.053095 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80fb7e5-aea2-4676-b496-27cf31a02253 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:45.054504 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80fb7e5-aea2-4676-b496-27cf31a02253 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:45.062320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 506/1014] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:51:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:45.062320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cdaf9-75fa-43ae-b2f0-67925476fab9 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:45.062320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cdaf9-75fa-43ae-b2f0-67925476fab9 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:45.062320 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cdaf9-75fa-43ae-b2f0-67925476fab9 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:45.062748 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 507/1015] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:45.065009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cdaf9-75fa-43ae-b2f0-67925476fab9 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:45.122434 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f38f6-29c2-4f23-81ca-e83d4069e421 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:45.140454 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:45.157578 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c8d07-fe5c-426f-87cf-52d8fb311bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:45.158241 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c8d07-fe5c-426f-87cf-52d8fb311bd3 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:45.158556 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c8d07-fe5c-426f-87cf-52d8fb311bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:45.158827 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c8d07-fe5c-426f-87cf-52d8fb311bd3 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:45.159541 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c8d07-fe5c-426f-87cf-52d8fb311bd3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:45.160988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c8d07-fe5c-426f-87cf-52d8fb311bd3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:45.166707 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cdaf9-75fa-43ae-b2f0-67925476fab9 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:45.187993 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 509/1016] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:51:45 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:51:45.189447 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453be439-fbf7-40cb-a2a0-73363d97533f None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:45.189734 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453be439-fbf7-40cb-a2a0-73363d97533f None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:45.190177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453be439-fbf7-40cb-a2a0-73363d97533f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:45.204776 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fca52-b10a-4fec-9684-9ce1a1cafee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:45.205476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fca52-b10a-4fec-9684-9ce1a1cafee5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:45.205760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fca52-b10a-4fec-9684-9ce1a1cafee5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:45.206034 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fca52-b10a-4fec-9684-9ce1a1cafee5 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:45.206819 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95fca52-b10a-4fec-9684-9ce1a1cafee5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:45.207131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95fca52-b10a-4fec-9684-9ce1a1cafee5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:45.208568 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95fca52-b10a-4fec-9684-9ce1a1cafee5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:45.208676 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-453be439-fbf7-40cb-a2a0-73363d97533f None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:45.210120 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-453be439-fbf7-40cb-a2a0-73363d97533f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:45.219744 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 508/1017] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 19:51:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:45.220779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fca52-b10a-4fec-9684-9ce1a1cafee5 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:45.254410 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:45.291521 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 510/1018] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:45 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:45.316511 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453be439-fbf7-40cb-a2a0-73363d97533f None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:45.338816 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fca52-b10a-4fec-9684-9ce1a1cafee5 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:45.388598 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a0aa7-7373-4c4f-9272-2a62688de0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:45.388598 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a0aa7-7373-4c4f-9272-2a62688de0ce None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:45.388852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a0aa7-7373-4c4f-9272-2a62688de0ce None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:45.389060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a0aa7-7373-4c4f-9272-2a62688de0ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:45.389717 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a0aa7-7373-4c4f-9272-2a62688de0ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:45.391181 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a0aa7-7373-4c4f-9272-2a62688de0ce None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:45.400192 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:45.417484 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 509/1019] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:51:45 2022] POST /identity/v3/endpoints => generated 334 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:51:45.431598 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453be439-fbf7-40cb-a2a0-73363d97533f None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:45.478901 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351e98a-af02-4589-ba7f-8eaf88051200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:45.479600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351e98a-af02-4589-ba7f-8eaf88051200 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:45.479940 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351e98a-af02-4589-ba7f-8eaf88051200 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:45.480224 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351e98a-af02-4589-ba7f-8eaf88051200 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:45.480987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351e98a-af02-4589-ba7f-8eaf88051200 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:45.482586 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351e98a-af02-4589-ba7f-8eaf88051200 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:45.501317 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 511/1020] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:51:45 2022] POST /identity/v3/services => generated 242 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:51:47.572081 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163638b-1b34-4d92-bc95-7f1206087891 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:47.572607 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163638b-1b34-4d92-bc95-7f1206087891 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:47.572682 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163638b-1b34-4d92-bc95-7f1206087891 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:47.574105 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 510/1021] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:47.580815 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be2b12-c78f-40ec-866f-b00f68b97da0 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:47.581325 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be2b12-c78f-40ec-866f-b00f68b97da0 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:47.581325 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be2b12-c78f-40ec-866f-b00f68b97da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:47.600785 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-73be2b12-c78f-40ec-866f-b00f68b97da0 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:47.602024 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-73be2b12-c78f-40ec-866f-b00f68b97da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:47.631189 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:47.687847 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 512/1022] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:47.714467 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163638b-1b34-4d92-bc95-7f1206087891 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:47.738799 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d83c9-6567-482c-97bb-bce4c6ae9138 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:47.739125 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d83c9-6567-482c-97bb-bce4c6ae9138 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:47.739373 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d83c9-6567-482c-97bb-bce4c6ae9138 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:47.741236 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 513/1023] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:47.746744 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df5131-6ecb-4a78-b975-e30042a7babf None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:47.747285 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df5131-6ecb-4a78-b975-e30042a7babf None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:47.748784 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df5131-6ecb-4a78-b975-e30042a7babf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:47.771552 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-40df5131-6ecb-4a78-b975-e30042a7babf None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:47.773359 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-40df5131-6ecb-4a78-b975-e30042a7babf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:47.827645 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163638b-1b34-4d92-bc95-7f1206087891 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:47.844805 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 514/1024] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:47.846142 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd27aad-c95a-495a-8968-fe3661f03268 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:47.846943 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd27aad-c95a-495a-8968-fe3661f03268 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:47.847048 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd27aad-c95a-495a-8968-fe3661f03268 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:47.850423 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 515/1025] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:47.855891 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cf535-9d9a-486c-92b9-bdf356e583c5 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:47.856173 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cf535-9d9a-486c-92b9-bdf356e583c5 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:47.856668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cf535-9d9a-486c-92b9-bdf356e583c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:47.863574 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd8373-70e4-4c6e-b32a-ca436d126268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:47.864383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd8373-70e4-4c6e-b32a-ca436d126268 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:47.864670 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd8373-70e4-4c6e-b32a-ca436d126268 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:47.864949 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd8373-70e4-4c6e-b32a-ca436d126268 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:47.866144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd8373-70e4-4c6e-b32a-ca436d126268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:47.867634 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd8373-70e4-4c6e-b32a-ca436d126268 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:47.869389 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 511/1026] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:47.870042 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd8373-70e4-4c6e-b32a-ca436d126268 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:47.874299 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-622cf535-9d9a-486c-92b9-bdf356e583c5 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:47.875428 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-622cf535-9d9a-486c-92b9-bdf356e583c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:47.932160 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:47.944250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 516/1027] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:47.945129 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cf535-9d9a-486c-92b9-bdf356e583c5 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:47.968388 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:47.975416 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd8373-70e4-4c6e-b32a-ca436d126268 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:48.009200 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156d706-9938-40ab-a6a6-bed36bb95609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:48.009996 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156d706-9938-40ab-a6a6-bed36bb95609 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:48.010305 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156d706-9938-40ab-a6a6-bed36bb95609 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:48.010587 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156d706-9938-40ab-a6a6-bed36bb95609 None admin] PATH_INFO: `/v3/services/image` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:48.011269 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156d706-9938-40ab-a6a6-bed36bb95609 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:48.012665 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156d706-9938-40ab-a6a6-bed36bb95609 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-c156d706-9938-40ab-a6a6-bed36bb95609 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:48.024546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:48.025914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:48.026723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:48.026723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:48.026723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:48.026723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:48.026723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:48.026723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:48.026723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:48.026723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:48.026723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:48.026723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:48.026723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:48.026723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:48.026723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:51:48.026723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:48.026723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 512/1028] 10.222.0.84 () {56 vars in 1105 bytes} [Thu May 19 19:51:47 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:48.027508 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156d706-9938-40ab-a6a6-bed36bb95609 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:48.058976 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cf535-9d9a-486c-92b9-bdf356e583c5 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:48.093112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11628799-e2d9-426b-99a4-f8e8cfffa43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:48.093720 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11628799-e2d9-426b-99a4-f8e8cfffa43a None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:48.094016 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11628799-e2d9-426b-99a4-f8e8cfffa43a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:48.094309 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11628799-e2d9-426b-99a4-f8e8cfffa43a None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:48.095024 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11628799-e2d9-426b-99a4-f8e8cfffa43a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:48.096781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11628799-e2d9-426b-99a4-f8e8cfffa43a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:48.118359 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 517/1029] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:51:47 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:51:48.119175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11628799-e2d9-426b-99a4-f8e8cfffa43a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:48.145262 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156d706-9938-40ab-a6a6-bed36bb95609 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:48.177592 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:48.189403 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42e860-71f8-49fb-b878-8160ade10b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:48.190254 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42e860-71f8-49fb-b878-8160ade10b1c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:48.190566 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42e860-71f8-49fb-b878-8160ade10b1c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:48.190846 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42e860-71f8-49fb-b878-8160ade10b1c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:48.197746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:48.204183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42e860-71f8-49fb-b878-8160ade10b1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-5b42e860-71f8-49fb-b878-8160ade10b1c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:48.212802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:48.213785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:48.214609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:48.214609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:48.214609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:48.214609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:48.214609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:48.214609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:48.214609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:48.214609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:48.214609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:48.214609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:48.214609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:51:48.214609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:48.214609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 513/1030] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 19:51:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:48.221078 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42e860-71f8-49fb-b878-8160ade10b1c None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:48.241623 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11628799-e2d9-426b-99a4-f8e8cfffa43a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:48.279942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae6f62-0fe9-4621-96f1-98f543b45318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:48.280934 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae6f62-0fe9-4621-96f1-98f543b45318 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:48.281221 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae6f62-0fe9-4621-96f1-98f543b45318 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:48.281507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae6f62-0fe9-4621-96f1-98f543b45318 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:48.282236 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae6f62-0fe9-4621-96f1-98f543b45318 None admin] RBAC: Adding query filter params (name=image) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:48.282583 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae6f62-0fe9-4621-96f1-98f543b45318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:48.283980 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae6f62-0fe9-4621-96f1-98f543b45318 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:48.294316 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 518/1031] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:48.295544 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae6f62-0fe9-4621-96f1-98f543b45318 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:48.348570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42e860-71f8-49fb-b878-8160ade10b1c None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:48.390930 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb3dad-fdc0-4fdd-bb59-9ccf66a4ab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:48.391595 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb3dad-fdc0-4fdd-bb59-9ccf66a4ab33 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:48.391909 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb3dad-fdc0-4fdd-bb59-9ccf66a4ab33 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:48.392192 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb3dad-fdc0-4fdd-bb59-9ccf66a4ab33 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:48.392929 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb3dad-fdc0-4fdd-bb59-9ccf66a4ab33 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:48.393243 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb3dad-fdc0-4fdd-bb59-9ccf66a4ab33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:48.394770 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb3dad-fdc0-4fdd-bb59-9ccf66a4ab33 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:48.404541 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 514/1032] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 19:51:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:48.409522 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb3dad-fdc0-4fdd-bb59-9ccf66a4ab33 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:48.421667 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae6f62-0fe9-4621-96f1-98f543b45318 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:48.458052 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cd01a-7781-47e0-bdcc-9c543d021c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:48.458905 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cd01a-7781-47e0-bdcc-9c543d021c39 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:48.459351 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cd01a-7781-47e0-bdcc-9c543d021c39 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:48.459570 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cd01a-7781-47e0-bdcc-9c543d021c39 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:48.460293 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cd01a-7781-47e0-bdcc-9c543d021c39 None admin] RBAC: Adding query filter params (type=image) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:48.460759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cd01a-7781-47e0-bdcc-9c543d021c39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:48.462194 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cd01a-7781-47e0-bdcc-9c543d021c39 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:48.463654 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:48.464081 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:48.470709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 519/1033] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:48.476650 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cd01a-7781-47e0-bdcc-9c543d021c39 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:48.521754 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb3dad-fdc0-4fdd-bb59-9ccf66a4ab33 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:48.556601 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd8815-2488-4da5-9df0-334bcb7f2da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:48.557416 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd8815-2488-4da5-9df0-334bcb7f2da1 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:48.557704 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd8815-2488-4da5-9df0-334bcb7f2da1 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:48.557986 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd8815-2488-4da5-9df0-334bcb7f2da1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:48.569232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd8815-2488-4da5-9df0-334bcb7f2da1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:48.570762 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd8815-2488-4da5-9df0-334bcb7f2da1 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:48.586168 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cd01a-7781-47e0-bdcc-9c543d021c39 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:48.588439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-b9dd8815-2488-4da5-9df0-334bcb7f2da1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:48.588439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:48.588439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:48.588439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:48.588439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:48.588439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:48.588439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:48.588439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:48.588439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:48.588439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:48.588439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:48.588439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:48.588439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:48.588439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:48.588439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:48.589533 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:48.595555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:48.595555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:48.595555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:48.595555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:48.595555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 515/1034] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:48.597788 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd8815-2488-4da5-9df0-334bcb7f2da1 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:48.623388 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44777a1-0610-48c7-80f1-a1ef77b0f0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:48.623985 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44777a1-0610-48c7-80f1-a1ef77b0f0cd None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:48.624269 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44777a1-0610-48c7-80f1-a1ef77b0f0cd None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:48.624549 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44777a1-0610-48c7-80f1-a1ef77b0f0cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:48.625356 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44777a1-0610-48c7-80f1-a1ef77b0f0cd None admin] RBAC: Adding query filter params (interface=public, service_id=93ff2c2faf0a4b17af12b342bea9ae3e, region_id=RegionOne) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:48.625682 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44777a1-0610-48c7-80f1-a1ef77b0f0cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:48.627097 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44777a1-0610-48c7-80f1-a1ef77b0f0cd None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:48.632320 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 520/1035] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:51:48 2022] GET /identity/v3/endpoints?service_id=93ff2c2faf0a4b17af12b342bea9ae3e&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:48.637751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44777a1-0610-48c7-80f1-a1ef77b0f0cd None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:48.703077 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:48.721891 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:48.724998 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd8815-2488-4da5-9df0-334bcb7f2da1 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:48.749944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44777a1-0610-48c7-80f1-a1ef77b0f0cd None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:48.763894 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6b8f3-db0e-44be-aaf0-3be19106f915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:48.764704 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6b8f3-db0e-44be-aaf0-3be19106f915 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:48.764992 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6b8f3-db0e-44be-aaf0-3be19106f915 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:48.765279 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6b8f3-db0e-44be-aaf0-3be19106f915 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:48.766515 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6b8f3-db0e-44be-aaf0-3be19106f915 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:48.766994 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6b8f3-db0e-44be-aaf0-3be19106f915 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:48.769133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6b8f3-db0e-44be-aaf0-3be19106f915 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:48.784525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 516/1036] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:48.786013 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d23e31-c022-4b35-8fc2-3262d9aeb2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:48.786727 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d23e31-c022-4b35-8fc2-3262d9aeb2bb None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:48.787010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d23e31-c022-4b35-8fc2-3262d9aeb2bb None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:48.787283 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d23e31-c022-4b35-8fc2-3262d9aeb2bb None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:48.787974 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d23e31-c022-4b35-8fc2-3262d9aeb2bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:48.789355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d23e31-c022-4b35-8fc2-3262d9aeb2bb None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:48.790018 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6b8f3-db0e-44be-aaf0-3be19106f915 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:48.806516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 521/1037] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:51:48 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 19:51:48.902022 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6b8f3-db0e-44be-aaf0-3be19106f915 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:48.940807 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825a071-cc19-4846-b0d4-bb7d51215d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:48.941628 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825a071-cc19-4846-b0d4-bb7d51215d37 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:48.941907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825a071-cc19-4846-b0d4-bb7d51215d37 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:48.942158 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825a071-cc19-4846-b0d4-bb7d51215d37 None admin] PATH_INFO: `/v3/users/nova` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:48.950761 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825a071-cc19-4846-b0d4-bb7d51215d37 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:48.952273 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825a071-cc19-4846-b0d4-bb7d51215d37 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-5825a071-cc19-4846-b0d4-bb7d51215d37 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:48.968569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:48.969545 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:48.970470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:48.970470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:48.970470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:48.970470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:48.970470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:48.970470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:48.970470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:48.970470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:48.970470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:51:48.970470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:48.974665 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 517/1038] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:51:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:48.976149 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d23e31-c022-4b35-8fc2-3262d9aeb2bb None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:49.082158 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d23e31-c022-4b35-8fc2-3262d9aeb2bb None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:49.108542 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:49.118959 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883f791-3667-4cf6-bb11-bba5f56f074e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:49.119646 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883f791-3667-4cf6-bb11-bba5f56f074e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:49.119928 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883f791-3667-4cf6-bb11-bba5f56f074e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:49.120204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883f791-3667-4cf6-bb11-bba5f56f074e None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:49.121058 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883f791-3667-4cf6-bb11-bba5f56f074e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:49.121370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883f791-3667-4cf6-bb11-bba5f56f074e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:49.123243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883f791-3667-4cf6-bb11-bba5f56f074e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:49.146027 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 522/1039] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:51:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:49.147658 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825a071-cc19-4846-b0d4-bb7d51215d37 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:49.172834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:49.257192 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825a071-cc19-4846-b0d4-bb7d51215d37 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:49.290920 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76861bc-69e4-46ac-b002-e8a72b18cc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:49.291573 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76861bc-69e4-46ac-b002-e8a72b18cc4e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:49.291852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76861bc-69e4-46ac-b002-e8a72b18cc4e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:49.292129 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76861bc-69e4-46ac-b002-e8a72b18cc4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:49.362291 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76861bc-69e4-46ac-b002-e8a72b18cc4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:49.363784 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76861bc-69e4-46ac-b002-e8a72b18cc4e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:49.384761 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:49.463939 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 518/1040] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:49 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:51:49.469968 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883f791-3667-4cf6-bb11-bba5f56f074e None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:49.576674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883f791-3667-4cf6-bb11-bba5f56f074e None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:49.613444 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0337f-b265-4575-8434-8cbef6c760fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:49.614203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0337f-b265-4575-8434-8cbef6c760fd None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:49.614544 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0337f-b265-4575-8434-8cbef6c760fd None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:49.614787 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0337f-b265-4575-8434-8cbef6c760fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:49.625513 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0337f-b265-4575-8434-8cbef6c760fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:49.627031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0337f-b265-4575-8434-8cbef6c760fd None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:49.644745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-8bf0337f-b265-4575-8434-8cbef6c760fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:49.644745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:49.644745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:49.644745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:49.644745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:49.644745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:49.644745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:49.644745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:49.644745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:49.644745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:49.644745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:49.644745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:49.644745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:49.644745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:49.644745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:49.645649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:49.651154 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:49.651154 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:49.651154 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:49.651154 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:49.651154 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 523/1041] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:49.653356 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76861bc-69e4-46ac-b002-e8a72b18cc4e None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:49.762561 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76861bc-69e4-46ac-b002-e8a72b18cc4e None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:49.783855 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:49.799161 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccc3d1-9e96-41a5-a87d-79912fb9a765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:49.799830 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccc3d1-9e96-41a5-a87d-79912fb9a765 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:49.800110 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccc3d1-9e96-41a5-a87d-79912fb9a765 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:49.800384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccc3d1-9e96-41a5-a87d-79912fb9a765 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:49.801109 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccc3d1-9e96-41a5-a87d-79912fb9a765 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:49.801417 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccc3d1-9e96-41a5-a87d-79912fb9a765 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:49.802816 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccc3d1-9e96-41a5-a87d-79912fb9a765 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:49.815183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 519/1042] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:49.820911 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0337f-b265-4575-8434-8cbef6c760fd None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:49.859008 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:49.929476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0337f-b265-4575-8434-8cbef6c760fd None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:49.966390 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33847ea9-7328-4348-bb5f-fcbfd3e08714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:49.967138 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33847ea9-7328-4348-bb5f-fcbfd3e08714 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:49.967410 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33847ea9-7328-4348-bb5f-fcbfd3e08714 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:49.967690 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33847ea9-7328-4348-bb5f-fcbfd3e08714 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:49.975247 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33847ea9-7328-4348-bb5f-fcbfd3e08714 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:49.976679 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33847ea9-7328-4348-bb5f-fcbfd3e08714 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:49.990852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-33847ea9-7328-4348-bb5f-fcbfd3e08714 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:49.990852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:49.990852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:49.990852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:49.990852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:49.990852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:49.990852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:49.990852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:49.990852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:49.990852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:49.990852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:49.990852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:49.990852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:49.990852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:49.990852 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:49.991791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:49.992654 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:49.992654 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:49.992654 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:49.992654 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:49.992654 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:49.992654 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:49.992654 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:49.992654 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:49.997543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 524/1043] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:49.998945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccc3d1-9e96-41a5-a87d-79912fb9a765 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:50.101895 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccc3d1-9e96-41a5-a87d-79912fb9a765 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:50.135751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239d452-99f4-43cb-9656-b010d54e99a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:50.136415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239d452-99f4-43cb-9656-b010d54e99a7 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:50.136694 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239d452-99f4-43cb-9656-b010d54e99a7 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:50.136967 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239d452-99f4-43cb-9656-b010d54e99a7 None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:50.137805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3239d452-99f4-43cb-9656-b010d54e99a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:50.138366 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3239d452-99f4-43cb-9656-b010d54e99a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:50.139904 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3239d452-99f4-43cb-9656-b010d54e99a7 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:50.152554 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 520/1044] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:50.158789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33847ea9-7328-4348-bb5f-fcbfd3e08714 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:50.264891 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:50.272576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33847ea9-7328-4348-bb5f-fcbfd3e08714 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:50.307246 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331ea82-94c0-4b3d-8a2c-1775afa4f1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:50.307957 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331ea82-94c0-4b3d-8a2c-1775afa4f1f8 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:50.308236 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331ea82-94c0-4b3d-8a2c-1775afa4f1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:50.308506 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331ea82-94c0-4b3d-8a2c-1775afa4f1f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:50.309389 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331ea82-94c0-4b3d-8a2c-1775afa4f1f8 None admin] RBAC: Adding query filter params (role.id=206b3ccf6eb842fc8e1d86592c6500b7, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=0da7f490d9604107b974e9e461ebc38b) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:50.309781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331ea82-94c0-4b3d-8a2c-1775afa4f1f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:50.311264 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331ea82-94c0-4b3d-8a2c-1775afa4f1f8 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:50.317168 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 525/1045] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:50 2022] GET /identity/v3/role_assignments?user.id=0da7f490d9604107b974e9e461ebc38b&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=206b3ccf6eb842fc8e1d86592c6500b7 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:50.537851 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970523f0-4591-47ec-85f9-59021089ed33 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:50.538153 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970523f0-4591-47ec-85f9-59021089ed33 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:50.538464 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970523f0-4591-47ec-85f9-59021089ed33 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:50.539991 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 521/1046] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:50.545815 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669a0e1-3b92-47d5-84ea-18f518b4cae1 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:50.546115 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669a0e1-3b92-47d5-84ea-18f518b4cae1 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:50.546441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669a0e1-3b92-47d5-84ea-18f518b4cae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:50.564255 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-a669a0e1-3b92-47d5-84ea-18f518b4cae1 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:50.565466 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-a669a0e1-3b92-47d5-84ea-18f518b4cae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:50.643502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 526/1047] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:50.668071 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970523f0-4591-47ec-85f9-59021089ed33 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:50.723197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:50.787749 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970523f0-4591-47ec-85f9-59021089ed33 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:50.824459 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f52a1-ca73-4271-87b6-eb1898503acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:50.825243 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f52a1-ca73-4271-87b6-eb1898503acd None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:50.825582 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f52a1-ca73-4271-87b6-eb1898503acd None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:50.825858 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f52a1-ca73-4271-87b6-eb1898503acd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:50.836409 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f52a1-ca73-4271-87b6-eb1898503acd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-412f52a1-ca73-4271-87b6-eb1898503acd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:50.844017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:50.844889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:50.845725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:50.845725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:50.845725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:50.845725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:50.845725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:50.845725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:50.845725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:50.845725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:50.845725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:50.845725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:50.845725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:50.845725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:50.845725 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 522/1048] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:51:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:50.852946 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669a0e1-3b92-47d5-84ea-18f518b4cae1 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:50.957031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:50.968072 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669a0e1-3b92-47d5-84ea-18f518b4cae1 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:51.004564 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095955c-914a-4852-82e1-cb5a9e1e7d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:51.005237 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095955c-914a-4852-82e1-cb5a9e1e7d83 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:51.005519 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095955c-914a-4852-82e1-cb5a9e1e7d83 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:51.005797 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095955c-914a-4852-82e1-cb5a9e1e7d83 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:51.006583 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095955c-914a-4852-82e1-cb5a9e1e7d83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:51.006890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095955c-914a-4852-82e1-cb5a9e1e7d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:51.008263 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095955c-914a-4852-82e1-cb5a9e1e7d83 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:51.017241 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 527/1049] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:51:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:51.022995 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f52a1-ca73-4271-87b6-eb1898503acd None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:51.064034 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe192625-ea92-4bb1-9819-828c47a5eece None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:51.064370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe192625-ea92-4bb1-9819-828c47a5eece None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:51.064613 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe192625-ea92-4bb1-9819-828c47a5eece None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:51.066156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 528/1050] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:51:51.071118 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c7a8e-f124-43b8-973d-bf61b5ea1413 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:51.071429 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c7a8e-f124-43b8-973d-bf61b5ea1413 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:51.071711 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c7a8e-f124-43b8-973d-bf61b5ea1413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:51.092024 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-f02c7a8e-f124-43b8-973d-bf61b5ea1413 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:51.094127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-f02c7a8e-f124-43b8-973d-bf61b5ea1413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:51.133058 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:51.136266 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f52a1-ca73-4271-87b6-eb1898503acd None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:51.167145 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 529/1051] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:51.172131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4eeaa0-a4f3-4567-95e6-98e924aff202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:51.173030 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4eeaa0-a4f3-4567-95e6-98e924aff202 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:51.173313 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4eeaa0-a4f3-4567-95e6-98e924aff202 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:51.173587 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4eeaa0-a4f3-4567-95e6-98e924aff202 None admin] PATH_INFO: `/v3/users/28661e7300ca425182a14c7321793c87` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:51.187828 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4eeaa0-a4f3-4567-95e6-98e924aff202 None admin] RBAC: Authorizing `identity:get_user(user_id=28661e7300ca425182a14c7321793c87)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:51.189313 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4eeaa0-a4f3-4567-95e6-98e924aff202 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:51.189779 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c7a8e-f124-43b8-973d-bf61b5ea1413 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:51.195758 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 523/1052] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:51:51 2022] GET /identity/v3/users/28661e7300ca425182a14c7321793c87 => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:51.196496 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4eeaa0-a4f3-4567-95e6-98e924aff202 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:51.282252 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:51.298844 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4eeaa0-a4f3-4567-95e6-98e924aff202 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:51.307708 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c7a8e-f124-43b8-973d-bf61b5ea1413 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:51.334886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c8a49-fd65-4291-811c-f2fb9b1381c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:51.335591 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c8a49-fd65-4291-811c-f2fb9b1381c4 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:51.335875 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c8a49-fd65-4291-811c-f2fb9b1381c4 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:51.336153 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c8a49-fd65-4291-811c-f2fb9b1381c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:51.351856 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0771d8-d12b-4723-b32f-6e6daa2ba40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:51.352588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0771d8-d12b-4723-b32f-6e6daa2ba40e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:51.352871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0771d8-d12b-4723-b32f-6e6daa2ba40e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:51.353146 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0771d8-d12b-4723-b32f-6e6daa2ba40e None admin] PATH_INFO: `/v3/services/image` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:51.353806 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0771d8-d12b-4723-b32f-6e6daa2ba40e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:51.355214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0771d8-d12b-4723-b32f-6e6daa2ba40e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:51.364657 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-8c0771d8-d12b-4723-b32f-6e6daa2ba40e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:51.367409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:51.368562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:51.369452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:51.369452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:51.369452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:51.369452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:51.369452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:51.369452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:51.369452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:51.369452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:51.369452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:51.369452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:51.369452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:51:51.369452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:51:51.369452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:51:51.369452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:51.369452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 530/1053] 10.222.0.84 () {56 vars in 1105 bytes} [Thu May 19 19:51:51 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:51.375069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0771d8-d12b-4723-b32f-6e6daa2ba40e None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:51.415187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c8a49-fd65-4291-811c-f2fb9b1381c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:51.416650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c8a49-fd65-4291-811c-f2fb9b1381c4 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:51.478241 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0771d8-d12b-4723-b32f-6e6daa2ba40e None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:51.514711 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafc9b1-ab62-45fd-9525-d904bcd21c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:51.515382 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafc9b1-ab62-45fd-9525-d904bcd21c43 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:51.515659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafc9b1-ab62-45fd-9525-d904bcd21c43 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:51.515944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafc9b1-ab62-45fd-9525-d904bcd21c43 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:51.516663 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafc9b1-ab62-45fd-9525-d904bcd21c43 None admin] RBAC: Adding query filter params (name=image) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:51.516974 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafc9b1-ab62-45fd-9525-d904bcd21c43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:51.518391 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafc9b1-ab62-45fd-9525-d904bcd21c43 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:51.523668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 531/1054] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:51.529473 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafc9b1-ab62-45fd-9525-d904bcd21c43 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:51.534433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 524/1055] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:51 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:51:51.540422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c8a49-fd65-4291-811c-f2fb9b1381c4 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:51.542406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:51.601379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:51.641126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafc9b1-ab62-45fd-9525-d904bcd21c43 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:51.649630 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c8a49-fd65-4291-811c-f2fb9b1381c4 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:51.677440 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94127d5-22de-4e4d-995d-d646ab1cce44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:51.678199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94127d5-22de-4e4d-995d-d646ab1cce44 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:51.678422 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94127d5-22de-4e4d-995d-d646ab1cce44 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:51.678716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94127d5-22de-4e4d-995d-d646ab1cce44 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:51.679411 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94127d5-22de-4e4d-995d-d646ab1cce44 None admin] RBAC: Adding query filter params (type=image) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:51.679737 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94127d5-22de-4e4d-995d-d646ab1cce44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:51.681121 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94127d5-22de-4e4d-995d-d646ab1cce44 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:51.692331 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107bdbe7-8260-4081-a479-dc1298f07b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:51.692822 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 532/1056] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:51.692902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bdbe7-8260-4081-a479-dc1298f07b8b None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:51.692902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bdbe7-8260-4081-a479-dc1298f07b8b None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:51.692902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bdbe7-8260-4081-a479-dc1298f07b8b None admin] PATH_INFO: `/v3/projects/75f52549678b4c4fac4ef9fb82a31f23` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:51.692902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107bdbe7-8260-4081-a479-dc1298f07b8b None admin] RBAC: Authorizing `identity:get_project(project_id=75f52549678b4c4fac4ef9fb82a31f23)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:51.692902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107bdbe7-8260-4081-a479-dc1298f07b8b None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:51.693381 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94127d5-22de-4e4d-995d-d646ab1cce44 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:51.698476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 525/1057] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:51:51 2022] GET /identity/v3/projects/75f52549678b4c4fac4ef9fb82a31f23 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:51.699270 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107bdbe7-8260-4081-a479-dc1298f07b8b None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:51.774340 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:51.800057 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94127d5-22de-4e4d-995d-d646ab1cce44 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:51.816376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107bdbe7-8260-4081-a479-dc1298f07b8b None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:51.835089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b9adb-2dfd-468e-822c-ad0f4a1f16b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:51.835940 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b9adb-2dfd-468e-822c-ad0f4a1f16b9 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:51.836356 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b9adb-2dfd-468e-822c-ad0f4a1f16b9 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:51.836864 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b9adb-2dfd-468e-822c-ad0f4a1f16b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:51.837606 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b9adb-2dfd-468e-822c-ad0f4a1f16b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:51.839068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b9adb-2dfd-468e-822c-ad0f4a1f16b9 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:51.843620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:51.853737 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c241994-15eb-4f4b-b0b6-4b9fd8d35884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:51.854637 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c241994-15eb-4f4b-b0b6-4b9fd8d35884 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:51.855085 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c241994-15eb-4f4b-b0b6-4b9fd8d35884 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:51.855379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c241994-15eb-4f4b-b0b6-4b9fd8d35884 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:51.856217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c241994-15eb-4f4b-b0b6-4b9fd8d35884 None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=75f52549678b4c4fac4ef9fb82a31f23, user.id=28661e7300ca425182a14c7321793c87) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:51.856730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c241994-15eb-4f4b-b0b6-4b9fd8d35884 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:51.858676 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c241994-15eb-4f4b-b0b6-4b9fd8d35884 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:51.864300 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 533/1058] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:51:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:51:51.874380 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 526/1059] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:51 2022] GET /identity/v3/role_assignments?user.id=28661e7300ca425182a14c7321793c87&scope.project.id=75f52549678b4c4fac4ef9fb82a31f23&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:52.565616 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93c5c9-e2ba-4f56-bf21-c905f29fa4e8 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:52.566057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93c5c9-e2ba-4f56-bf21-c905f29fa4e8 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:52.566166 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93c5c9-e2ba-4f56-bf21-c905f29fa4e8 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:52.567768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 527/1060] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:52.573109 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb33311-90c8-48fd-b891-a188ff298438 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:52.573403 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb33311-90c8-48fd-b891-a188ff298438 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:52.573684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb33311-90c8-48fd-b891-a188ff298438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:52.591423 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-8bb33311-90c8-48fd-b891-a188ff298438 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:52.592886 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-8bb33311-90c8-48fd-b891-a188ff298438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:52.673054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 534/1061] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:52.696062 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93c5c9-e2ba-4f56-bf21-c905f29fa4e8 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:52.808526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93c5c9-e2ba-4f56-bf21-c905f29fa4e8 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:52.843836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703365c-7fd3-41d1-ab04-b35142703cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:52.844579 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703365c-7fd3-41d1-ab04-b35142703cd5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:52.844862 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703365c-7fd3-41d1-ab04-b35142703cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:52.845141 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703365c-7fd3-41d1-ab04-b35142703cd5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:52.855816 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703365c-7fd3-41d1-ab04-b35142703cd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-e703365c-7fd3-41d1-ab04-b35142703cd5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:52.864182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:52.865069 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:52.865969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:52.865969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:52.865969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:52.865969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:52.865969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:52.865969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:52.865969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:52.865969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:52.865969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:52.865969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:52.865969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:51:52.865969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:52.870485 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 528/1062] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 19:51:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:52.871801 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb33311-90c8-48fd-b891-a188ff298438 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:52.940849 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:52.988492 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb33311-90c8-48fd-b891-a188ff298438 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:53.027115 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34e37c-6d10-447f-881d-22af8a4aa3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:53.027805 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34e37c-6d10-447f-881d-22af8a4aa3b9 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:53.028087 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34e37c-6d10-447f-881d-22af8a4aa3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:53.028448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34e37c-6d10-447f-881d-22af8a4aa3b9 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:53.029497 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34e37c-6d10-447f-881d-22af8a4aa3b9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:53.029806 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34e37c-6d10-447f-881d-22af8a4aa3b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:53.031226 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34e37c-6d10-447f-881d-22af8a4aa3b9 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:53.042443 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 535/1063] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 19:51:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:53.046242 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703365c-7fd3-41d1-ab04-b35142703cd5 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:53.100879 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:53.154958 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703365c-7fd3-41d1-ab04-b35142703cd5 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:53.189546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b9c88-71ac-4106-b552-da5b6b49275f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:53.190708 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b9c88-71ac-4106-b552-da5b6b49275f None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:53.191172 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b9c88-71ac-4106-b552-da5b6b49275f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:53.191621 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b9c88-71ac-4106-b552-da5b6b49275f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:53.202850 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b9c88-71ac-4106-b552-da5b6b49275f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:53.204611 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b9c88-71ac-4106-b552-da5b6b49275f None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:53.222437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-f90b9c88-71ac-4106-b552-da5b6b49275f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:53.222437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:53.222437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:53.222437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:53.222437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:53.222437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:53.222437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:53.222437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:53.222437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:53.222437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:53.222437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:53.222437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:53.222437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:53.222437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:53.222437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:53.223296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:53.224143 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:53.224143 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:53.224143 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:53.224143 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:53.230481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 529/1064] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:53.232124 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34e37c-6d10-447f-881d-22af8a4aa3b9 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:53.341694 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34e37c-6d10-447f-881d-22af8a4aa3b9 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:53.360163 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:53.378060 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476960d-3132-45d7-97d0-763530e94552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:53.378969 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476960d-3132-45d7-97d0-763530e94552 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:53.379432 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476960d-3132-45d7-97d0-763530e94552 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:53.379884 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476960d-3132-45d7-97d0-763530e94552 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:53.380819 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476960d-3132-45d7-97d0-763530e94552 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:53.381303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476960d-3132-45d7-97d0-763530e94552 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:53.382877 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476960d-3132-45d7-97d0-763530e94552 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:53.399840 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 536/1065] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:53.401492 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b9c88-71ac-4106-b552-da5b6b49275f None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:53.516877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b9c88-71ac-4106-b552-da5b6b49275f None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:53.518482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:53.554188 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3f69b-fef5-4047-af78-6351e852e409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:53.555026 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3f69b-fef5-4047-af78-6351e852e409 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:53.555310 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3f69b-fef5-4047-af78-6351e852e409 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:53.555576 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3f69b-fef5-4047-af78-6351e852e409 None admin] PATH_INFO: `/v3/users/nova` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:53.564164 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3f69b-fef5-4047-af78-6351e852e409 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:53.565630 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3f69b-fef5-4047-af78-6351e852e409 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-2cf3f69b-fef5-4047-af78-6351e852e409 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:53.580482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:53.581438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:53.587235 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:53.587235 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:53.587235 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:53.587235 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:53.587235 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:53.587235 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:53.587235 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:53.587235 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:53.587235 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:51:53.587235 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:53.587235 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 530/1066] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:51:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:53.589473 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476960d-3132-45d7-97d0-763530e94552 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:53.699775 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476960d-3132-45d7-97d0-763530e94552 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:53.743445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03831c-f4b7-4001-8a2d-e097291968d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:53.744397 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03831c-f4b7-4001-8a2d-e097291968d8 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:53.744829 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03831c-f4b7-4001-8a2d-e097291968d8 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:53.745252 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03831c-f4b7-4001-8a2d-e097291968d8 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:53.746246 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03831c-f4b7-4001-8a2d-e097291968d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:53.746740 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03831c-f4b7-4001-8a2d-e097291968d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:53.748349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03831c-f4b7-4001-8a2d-e097291968d8 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:53.771638 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 537/1067] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:51:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:53.773202 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3f69b-fef5-4047-af78-6351e852e409 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:53.885007 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3f69b-fef5-4047-af78-6351e852e409 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:53.921163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e93c3c-660b-4ca1-a6ba-4610f4ce143f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:53.921998 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e93c3c-660b-4ca1-a6ba-4610f4ce143f None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:53.922437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e93c3c-660b-4ca1-a6ba-4610f4ce143f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:53.922629 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e93c3c-660b-4ca1-a6ba-4610f4ce143f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:53.935342 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e93c3c-660b-4ca1-a6ba-4610f4ce143f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:53.937072 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e93c3c-660b-4ca1-a6ba-4610f4ce143f None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:53.956722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-62e93c3c-660b-4ca1-a6ba-4610f4ce143f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:53.956722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:53.956722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:53.956722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:53.956722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:53.956722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:53.956722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:53.956722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:53.956722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:53.956722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:53.956722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:53.956722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:53.956722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:53.956722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:53.956722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:53.957653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:53.963506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:53.963506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:53.963506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:53.963506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:53.963506 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 531/1068] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:53.965461 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03831c-f4b7-4001-8a2d-e097291968d8 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:54.011461 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:54.080718 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03831c-f4b7-4001-8a2d-e097291968d8 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:54.116990 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25905796-e38a-4843-84e8-1f58f394d434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:54.117790 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25905796-e38a-4843-84e8-1f58f394d434 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:54.118039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25905796-e38a-4843-84e8-1f58f394d434 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:54.118333 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25905796-e38a-4843-84e8-1f58f394d434 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:54.119071 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25905796-e38a-4843-84e8-1f58f394d434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:54.119346 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04a954-e440-4c9f-9856-6635d4d37059 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:54.119455 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25905796-e38a-4843-84e8-1f58f394d434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:54.119614 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04a954-e440-4c9f-9856-6635d4d37059 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:54.120033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04a954-e440-4c9f-9856-6635d4d37059 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:54.120737 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25905796-e38a-4843-84e8-1f58f394d434 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:54.121622 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 532/1069] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:51:54.127497 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26883ad-3449-488a-949a-4801cc91b6ae None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:54.127783 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26883ad-3449-488a-949a-4801cc91b6ae None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:54.128089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26883ad-3449-488a-949a-4801cc91b6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:54.138818 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 538/1070] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:54.140277 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25905796-e38a-4843-84e8-1f58f394d434 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:54.150250 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-a26883ad-3449-488a-949a-4801cc91b6ae None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:54.151448 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-a26883ad-3449-488a-949a-4801cc91b6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:54.162047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:54.220648 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 533/1071] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:54.222224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add35703-d6d2-4357-989d-e95177c3248f None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:54.222544 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add35703-d6d2-4357-989d-e95177c3248f None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:54.222832 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add35703-d6d2-4357-989d-e95177c3248f None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:54.224077 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 534/1072] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:54.229364 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa2ef0-e727-4ff2-8c03-937353db692d None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:54.229650 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa2ef0-e727-4ff2-8c03-937353db692d None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:54.229927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa2ef0-e727-4ff2-8c03-937353db692d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:54.239988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:54.246643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-79fa2ef0-e727-4ff2-8c03-937353db692d None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:54.247901 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-79fa2ef0-e727-4ff2-8c03-937353db692d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:54.248008 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25905796-e38a-4843-84e8-1f58f394d434 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:54.283523 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d86963-54cc-4d14-bffa-ea0e358da4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:54.284317 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d86963-54cc-4d14-bffa-ea0e358da4a2 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:54.284594 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d86963-54cc-4d14-bffa-ea0e358da4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:54.284862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d86963-54cc-4d14-bffa-ea0e358da4a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:54.291604 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d86963-54cc-4d14-bffa-ea0e358da4a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:54.293069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d86963-54cc-4d14-bffa-ea0e358da4a2 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:54.307830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-45d86963-54cc-4d14-bffa-ea0e358da4a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:54.307830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:54.307830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:54.307830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:54.307830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:54.307830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:54.307830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:54.307830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:54.307830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:54.307830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:54.307830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:54.307830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:54.307830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:54.307830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:54.307830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:54.308777 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:54.309763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:54.309763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:54.309763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:54.309763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:54.309763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:54.309763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:54.309763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:54.309763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:54.309763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 539/1073] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:54.312772 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d86963-54cc-4d14-bffa-ea0e358da4a2 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:54.317615 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 535/1074] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:54.318505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa2ef0-e727-4ff2-8c03-937353db692d None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:54.395426 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:54.416106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d86963-54cc-4d14-bffa-ea0e358da4a2 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:54.424575 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa2ef0-e727-4ff2-8c03-937353db692d None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:54.450185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba9ab9-9ced-4f56-bcc8-dcfee08232e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:54.451091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba9ab9-9ced-4f56-bcc8-dcfee08232e6 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:54.451370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba9ab9-9ced-4f56-bcc8-dcfee08232e6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:54.451651 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba9ab9-9ced-4f56-bcc8-dcfee08232e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:54.461839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2424e-f66f-43d8-aac6-1aa1c73f7b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:54.462086 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba9ab9-9ced-4f56-bcc8-dcfee08232e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:54.462823 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2424e-f66f-43d8-aac6-1aa1c73f7b3c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:54.463229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2424e-f66f-43d8-aac6-1aa1c73f7b3c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:54.463657 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2424e-f66f-43d8-aac6-1aa1c73f7b3c None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:54.464897 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2424e-f66f-43d8-aac6-1aa1c73f7b3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:54.465369 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2424e-f66f-43d8-aac6-1aa1c73f7b3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:54.467700 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2424e-f66f-43d8-aac6-1aa1c73f7b3c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-c5ba9ab9-9ced-4f56-bcc8-dcfee08232e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:54.469854 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:54.470831 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:54.472257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:54.472257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:54.472257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:54.472257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:54.472257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:54.472257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:54.472257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:54.472257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:54.472257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:54.472257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:54.472257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:54.472257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:54.472257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 540/1075] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:51:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:54.472257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba9ab9-9ced-4f56-bcc8-dcfee08232e6 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:54.481577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 536/1076] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:54.483116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2424e-f66f-43d8-aac6-1aa1c73f7b3c None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:54.497322 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:54.581573 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba9ab9-9ced-4f56-bcc8-dcfee08232e6 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:54.591308 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2424e-f66f-43d8-aac6-1aa1c73f7b3c None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:54.616159 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97472afb-3f98-4e6c-8309-420895fd851d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:54.616785 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97472afb-3f98-4e6c-8309-420895fd851d None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:54.617085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97472afb-3f98-4e6c-8309-420895fd851d None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:54.617354 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97472afb-3f98-4e6c-8309-420895fd851d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:54.625706 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c01de0-e031-4b5c-83e1-a1e19478b834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:54.626399 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c01de0-e031-4b5c-83e1-a1e19478b834 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:54.626683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c01de0-e031-4b5c-83e1-a1e19478b834 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:54.626962 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c01de0-e031-4b5c-83e1-a1e19478b834 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:54.627732 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c01de0-e031-4b5c-83e1-a1e19478b834 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:54.628047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c01de0-e031-4b5c-83e1-a1e19478b834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:54.629425 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c01de0-e031-4b5c-83e1-a1e19478b834 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:54.630860 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:54.639199 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 537/1077] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:51:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:54.640189 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c01de0-e031-4b5c-83e1-a1e19478b834 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:54.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97472afb-3f98-4e6c-8309-420895fd851d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:54.686964 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97472afb-3f98-4e6c-8309-420895fd851d None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:54.704247 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:54.745454 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c01de0-e031-4b5c-83e1-a1e19478b834 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:54.779567 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661ad2c-e822-4d06-be62-9c2bb712e003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:54.780462 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661ad2c-e822-4d06-be62-9c2bb712e003 None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:54.780741 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661ad2c-e822-4d06-be62-9c2bb712e003 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:54.781015 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661ad2c-e822-4d06-be62-9c2bb712e003 None admin] PATH_INFO: `/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/0da7f490d9604107b974e9e461ebc38b/roles/206b3ccf6eb842fc8e1d86592c6500b7` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:54.783373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661ad2c-e822-4d06-be62-9c2bb712e003 None admin] RBAC: Authorizing `identity:create_grant(project_id=d638a29a793d41e7bfa686c0fcc29149, user_id=0da7f490d9604107b974e9e461ebc38b, role_id=206b3ccf6eb842fc8e1d86592c6500b7)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:54.784840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661ad2c-e822-4d06-be62-9c2bb712e003 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:54.790449 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 541/1078] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:51:54.791358 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97472afb-3f98-4e6c-8309-420895fd851d None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:54.814338 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 538/1079] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:51:54 2022] PUT /identity/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/0da7f490d9604107b974e9e461ebc38b/roles/206b3ccf6eb842fc8e1d86592c6500b7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:51:54.814338 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661ad2c-e822-4d06-be62-9c2bb712e003 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:54.882737 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:54.907249 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97472afb-3f98-4e6c-8309-420895fd851d None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:54.928069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:54.938315 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661ad2c-e822-4d06-be62-9c2bb712e003 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:54.951500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9fe8a-0770-4895-99d8-bd31e8c6a4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:54.952416 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9fe8a-0770-4895-99d8-bd31e8c6a4d6 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:54.952703 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9fe8a-0770-4895-99d8-bd31e8c6a4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:54.952979 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9fe8a-0770-4895-99d8-bd31e8c6a4d6 None admin] PATH_INFO: `/v3/users/28661e7300ca425182a14c7321793c87` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:54.954319 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9fe8a-0770-4895-99d8-bd31e8c6a4d6 None admin] RBAC: Authorizing `identity:get_user(user_id=28661e7300ca425182a14c7321793c87)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:54.955797 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9fe8a-0770-4895-99d8-bd31e8c6a4d6 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:54.957551 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 542/1080] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:51:54 2022] GET /identity/v3/users/28661e7300ca425182a14c7321793c87 => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:54.963478 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9fe8a-0770-4895-99d8-bd31e8c6a4d6 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:54.983400 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ad362-4bc5-4531-b037-fe78af8419da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:54.984222 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ad362-4bc5-4531-b037-fe78af8419da None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:54.984509 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ad362-4bc5-4531-b037-fe78af8419da None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:54.984789 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ad362-4bc5-4531-b037-fe78af8419da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:54.998785 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ad362-4bc5-4531-b037-fe78af8419da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:55.000179 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ad362-4bc5-4531-b037-fe78af8419da None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:55.020951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-919ad362-4bc5-4531-b037-fe78af8419da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:55.020951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:55.020951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:55.020951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:55.020951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:55.020951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:55.020951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:55.020951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:55.020951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:55.020951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:55.020951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:55.020951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:55.020951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:55.020951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:55.020951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:55.021877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:55.022852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:55.022852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:55.022852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:55.022852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:55.022852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 539/1081] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:55.029879 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ad362-4bc5-4531-b037-fe78af8419da None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:55.081723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9fe8a-0770-4895-99d8-bd31e8c6a4d6 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:55.117741 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c17f35-973a-470e-90f1-f90d7086a4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:55.118674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c17f35-973a-470e-90f1-f90d7086a4a3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:55.118957 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c17f35-973a-470e-90f1-f90d7086a4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:55.119231 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c17f35-973a-470e-90f1-f90d7086a4a3 None admin] PATH_INFO: `/v3/projects/75f52549678b4c4fac4ef9fb82a31f23` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:55.120445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c17f35-973a-470e-90f1-f90d7086a4a3 None admin] RBAC: Authorizing `identity:get_project(project_id=75f52549678b4c4fac4ef9fb82a31f23)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:55.121887 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c17f35-973a-470e-90f1-f90d7086a4a3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:55.124342 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 543/1082] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:51:54 2022] GET /identity/v3/projects/75f52549678b4c4fac4ef9fb82a31f23 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:55.130138 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c17f35-973a-470e-90f1-f90d7086a4a3 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:55.138416 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ad362-4bc5-4531-b037-fe78af8419da None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:55.148899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:55.181709 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fc840-88d2-4bc4-bfec-73b569f117dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:55.182515 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fc840-88d2-4bc4-bfec-73b569f117dd None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:55.182727 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fc840-88d2-4bc4-bfec-73b569f117dd None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:55.183005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fc840-88d2-4bc4-bfec-73b569f117dd None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:55.183935 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fc840-88d2-4bc4-bfec-73b569f117dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:55.184057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fc840-88d2-4bc4-bfec-73b569f117dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:55.185501 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fc840-88d2-4bc4-bfec-73b569f117dd None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:55.187413 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:55.199403 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 540/1083] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:55.244322 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c17f35-973a-470e-90f1-f90d7086a4a3 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:55.285019 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295a756-3181-4b2d-9fc0-d1c9ad909c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:55.285944 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295a756-3181-4b2d-9fc0-d1c9ad909c6a None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:55.286226 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295a756-3181-4b2d-9fc0-d1c9ad909c6a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:55.286535 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295a756-3181-4b2d-9fc0-d1c9ad909c6a None admin] PATH_INFO: `/v3/projects/75f52549678b4c4fac4ef9fb82a31f23/users/28661e7300ca425182a14c7321793c87/roles/2cd6d3dd613c4824a26cacf53fb6c07f` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:55.288651 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1295a756-3181-4b2d-9fc0-d1c9ad909c6a None admin] RBAC: Authorizing `identity:create_grant(project_id=75f52549678b4c4fac4ef9fb82a31f23, user_id=28661e7300ca425182a14c7321793c87, role_id=2cd6d3dd613c4824a26cacf53fb6c07f)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:55.290115 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1295a756-3181-4b2d-9fc0-d1c9ad909c6a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:55.307356 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 544/1084] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:51:55 2022] PUT /identity/v3/projects/75f52549678b4c4fac4ef9fb82a31f23/users/28661e7300ca425182a14c7321793c87/roles/2cd6d3dd613c4824a26cacf53fb6c07f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:51:57.012281 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9684ccd-f215-43ef-b173-92445cbbfc1d None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:57.012947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9684ccd-f215-43ef-b173-92445cbbfc1d None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:57.012947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9684ccd-f215-43ef-b173-92445cbbfc1d None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:57.014215 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 541/1085] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:51:57.019753 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f452c-cc8b-44c0-9d07-5c67ed5d4a54 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:57.020194 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f452c-cc8b-44c0-9d07-5c67ed5d4a54 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:57.020386 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f452c-cc8b-44c0-9d07-5c67ed5d4a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:57.038781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-b95f452c-cc8b-44c0-9d07-5c67ed5d4a54 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:57.039990 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-b95f452c-cc8b-44c0-9d07-5c67ed5d4a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:57.111745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 545/1086] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:51:57.135250 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9684ccd-f215-43ef-b173-92445cbbfc1d None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:57.242115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9684ccd-f215-43ef-b173-92445cbbfc1d None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:57.276655 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12afccf-af69-4368-85f2-b812259d037f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:57.277661 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12afccf-af69-4368-85f2-b812259d037f None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:57.277939 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12afccf-af69-4368-85f2-b812259d037f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:57.278434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12afccf-af69-4368-85f2-b812259d037f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:57.289569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12afccf-af69-4368-85f2-b812259d037f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-b12afccf-af69-4368-85f2-b812259d037f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:57.297863 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:57.298923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:57.299887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:57.299887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:57.299887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:57.299887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:57.299887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:57.299887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:57.299887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:57.299887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:57.299887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:57.299887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:57.299887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:51:57.299887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:57.302685 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 542/1087] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 19:51:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:57.305214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f452c-cc8b-44c0-9d07-5c67ed5d4a54 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:57.352780 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:57.412037 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f452c-cc8b-44c0-9d07-5c67ed5d4a54 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:57.447198 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d59e1-bc50-4a1f-8c01-642838f287c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:57.447868 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d59e1-bc50-4a1f-8c01-642838f287c8 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:57.448150 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d59e1-bc50-4a1f-8c01-642838f287c8 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:57.448422 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d59e1-bc50-4a1f-8c01-642838f287c8 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:57.449172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d59e1-bc50-4a1f-8c01-642838f287c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:57.449476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d59e1-bc50-4a1f-8c01-642838f287c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:57.450933 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d59e1-bc50-4a1f-8c01-642838f287c8 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:57.459169 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 546/1088] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 19:51:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:57.465181 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12afccf-af69-4368-85f2-b812259d037f None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:57.495709 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:57.521653 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f7394-c182-43ea-93bf-6c0c78db45d2 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:57.521970 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f7394-c182-43ea-93bf-6c0c78db45d2 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:57.522239 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f7394-c182-43ea-93bf-6c0c78db45d2 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:57.523806 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 547/1089] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:51:57.528867 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e70403-bc16-478a-af12-79cf68e9b262 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:57.529069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e70403-bc16-478a-af12-79cf68e9b262 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:57.529370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e70403-bc16-478a-af12-79cf68e9b262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:57.546117 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-72e70403-bc16-478a-af12-79cf68e9b262 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:51:57.547339 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-72e70403-bc16-478a-af12-79cf68e9b262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:51:57.577476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12afccf-af69-4368-85f2-b812259d037f None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:57.612693 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e5e87-4eaf-4c0e-bc5b-482ac0f31a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:57.613495 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e5e87-4eaf-4c0e-bc5b-482ac0f31a16 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:57.613781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e5e87-4eaf-4c0e-bc5b-482ac0f31a16 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:57.614057 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e5e87-4eaf-4c0e-bc5b-482ac0f31a16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:57.616308 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 548/1090] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:51:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:51:57.624616 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e5e87-4eaf-4c0e-bc5b-482ac0f31a16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:57.626107 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e5e87-4eaf-4c0e-bc5b-482ac0f31a16 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:57.639990 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e70403-bc16-478a-af12-79cf68e9b262 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:57.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-673e5e87-4eaf-4c0e-bc5b-482ac0f31a16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:57.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:57.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:57.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:57.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:57.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:57.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:57.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:57.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:57.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:57.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:57.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:57.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:57.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:57.644147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:57.645116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:57.646146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:57.646146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:57.646146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:57.646146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:57.646146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 543/1091] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:57.651033 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e5e87-4eaf-4c0e-bc5b-482ac0f31a16 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:57.675558 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:57.735229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:57.745768 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e70403-bc16-478a-af12-79cf68e9b262 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:57.755315 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e5e87-4eaf-4c0e-bc5b-482ac0f31a16 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:57.783350 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e65b6-8c5b-46eb-90b0-c284f540869b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:57.784164 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e65b6-8c5b-46eb-90b0-c284f540869b None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:57.784448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e65b6-8c5b-46eb-90b0-c284f540869b None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:57.784721 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e65b6-8c5b-46eb-90b0-c284f540869b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:57.788767 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d6249-1133-4fe8-b8e2-7a3994236fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:57.789422 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d6249-1133-4fe8-b8e2-7a3994236fc8 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:57.789705 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d6249-1133-4fe8-b8e2-7a3994236fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:57.789981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d6249-1133-4fe8-b8e2-7a3994236fc8 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:57.790742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d6249-1133-4fe8-b8e2-7a3994236fc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:57.791054 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d6249-1133-4fe8-b8e2-7a3994236fc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:57.792437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d6249-1133-4fe8-b8e2-7a3994236fc8 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:57.796235 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e65b6-8c5b-46eb-90b0-c284f540869b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-881e65b6-8c5b-46eb-90b0-c284f540869b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:51:57.803919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:57.804862 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:57.805692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:57.805692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:57.805692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:57.805692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:57.805692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:51:57.805692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:51:57.805692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:51:57.805692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:51:57.805692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:51:57.805692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:51:57.805692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:51:57.805692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:57.805692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 549/1092] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:51:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:51:57.808971 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 544/1093] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:57.810467 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e65b6-8c5b-46eb-90b0-c284f540869b None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:57.810929 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d6249-1133-4fe8-b8e2-7a3994236fc8 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:57.905541 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:57.911499 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d6249-1133-4fe8-b8e2-7a3994236fc8 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:57.912769 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e65b6-8c5b-46eb-90b0-c284f540869b None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:57.945008 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df1f3b-d789-4d6e-872d-e0900f789fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:57.945668 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df1f3b-d789-4d6e-872d-e0900f789fbc None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:57.945947 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df1f3b-d789-4d6e-872d-e0900f789fbc None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:57.946054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7649b-a893-4905-bc17-9d4c50df9f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:57.946205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df1f3b-d789-4d6e-872d-e0900f789fbc None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:57.946836 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7649b-a893-4905-bc17-9d4c50df9f1e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:57.946989 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df1f3b-d789-4d6e-872d-e0900f789fbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:57.947123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7649b-a893-4905-bc17-9d4c50df9f1e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:57.947292 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df1f3b-d789-4d6e-872d-e0900f789fbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:57.947388 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7649b-a893-4905-bc17-9d4c50df9f1e None admin] PATH_INFO: `/v3/users/nova` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:57.948659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df1f3b-d789-4d6e-872d-e0900f789fbc None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:57.954924 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7649b-a893-4905-bc17-9d4c50df9f1e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:57.956367 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7649b-a893-4905-bc17-9d4c50df9f1e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:57.957013 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 545/1094] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:51:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:57.962895 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df1f3b-d789-4d6e-872d-e0900f789fbc None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:57.965520 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-e8a7649b-a893-4905-bc17-9d4c50df9f1e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:51:57.972328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:57.973364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:57.974250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:57.974250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:57.974250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:51:57.974250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:51:57.974250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:51:57.974250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:51:57.974250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:51:57.974250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:51:57.974250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:51:57.974250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:57.974250 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 550/1095] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 19:51:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:57.979285 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7649b-a893-4905-bc17-9d4c50df9f1e None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:58.065385 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df1f3b-d789-4d6e-872d-e0900f789fbc None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:58.085586 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7649b-a893-4905-bc17-9d4c50df9f1e None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:58.100983 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87ccec-37e2-46f9-b509-2137be2ea947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:58.101869 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87ccec-37e2-46f9-b509-2137be2ea947 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:58.102154 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87ccec-37e2-46f9-b509-2137be2ea947 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:58.102461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87ccec-37e2-46f9-b509-2137be2ea947 None admin] PATH_INFO: `/v3/users/28661e7300ca425182a14c7321793c87` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:58.103736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87ccec-37e2-46f9-b509-2137be2ea947 None admin] RBAC: Authorizing `identity:get_user(user_id=28661e7300ca425182a14c7321793c87)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:58.105187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87ccec-37e2-46f9-b509-2137be2ea947 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:58.106939 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 546/1096] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:51:57 2022] GET /identity/v3/users/28661e7300ca425182a14c7321793c87 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:58.114030 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87ccec-37e2-46f9-b509-2137be2ea947 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:58.122295 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796bf8c-1494-49a5-b72b-da1c41a39c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:58.122977 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796bf8c-1494-49a5-b72b-da1c41a39c96 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:58.123245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796bf8c-1494-49a5-b72b-da1c41a39c96 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:58.123519 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796bf8c-1494-49a5-b72b-da1c41a39c96 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:58.124363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796bf8c-1494-49a5-b72b-da1c41a39c96 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:58.124672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796bf8c-1494-49a5-b72b-da1c41a39c96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:58.126104 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796bf8c-1494-49a5-b72b-da1c41a39c96 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:58.143918 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 551/1097] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 19:51:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:58.150335 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796bf8c-1494-49a5-b72b-da1c41a39c96 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:58.164797 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:58.184461 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:58.215869 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87ccec-37e2-46f9-b509-2137be2ea947 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:58.250123 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796bf8c-1494-49a5-b72b-da1c41a39c96 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:58.250353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43736c-e4bf-4508-ad18-b7322f27625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:58.250837 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43736c-e4bf-4508-ad18-b7322f27625c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:58.251117 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43736c-e4bf-4508-ad18-b7322f27625c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:58.251400 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43736c-e4bf-4508-ad18-b7322f27625c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:58.284015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d7ace-be3f-40e9-84af-3aeff4fac2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:58.284726 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d7ace-be3f-40e9-84af-3aeff4fac2af None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:58.285015 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d7ace-be3f-40e9-84af-3aeff4fac2af None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:58.285288 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d7ace-be3f-40e9-84af-3aeff4fac2af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:58.320006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43736c-e4bf-4508-ad18-b7322f27625c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:58.321385 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43736c-e4bf-4508-ad18-b7322f27625c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:58.352320 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d7ace-be3f-40e9-84af-3aeff4fac2af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:58.353773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d7ace-be3f-40e9-84af-3aeff4fac2af None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:58.367502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:58.388395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:58.421399 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 547/1098] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:51:58.427578 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43736c-e4bf-4508-ad18-b7322f27625c None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:58.453269 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 552/1099] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:51:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:51:58.459140 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d7ace-be3f-40e9-84af-3aeff4fac2af None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:58.531222 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43736c-e4bf-4508-ad18-b7322f27625c None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:58.562073 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d7ace-be3f-40e9-84af-3aeff4fac2af None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:58.565637 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd953d7-7f94-4eef-8e59-629d615304fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:58.566543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd953d7-7f94-4eef-8e59-629d615304fa None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:58.566827 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd953d7-7f94-4eef-8e59-629d615304fa None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:58.567100 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd953d7-7f94-4eef-8e59-629d615304fa None admin] PATH_INFO: `/v3/projects/75f52549678b4c4fac4ef9fb82a31f23` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:58.568390 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd953d7-7f94-4eef-8e59-629d615304fa None admin] RBAC: Authorizing `identity:get_project(project_id=75f52549678b4c4fac4ef9fb82a31f23)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:58.569847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd953d7-7f94-4eef-8e59-629d615304fa None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:58.571765 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 548/1100] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:51:58 2022] GET /identity/v3/projects/75f52549678b4c4fac4ef9fb82a31f23 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:58.575402 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:58.578160 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd953d7-7f94-4eef-8e59-629d615304fa None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:58.597412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679e1fe-b313-40c8-8b21-81f16a085bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:58.598206 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679e1fe-b313-40c8-8b21-81f16a085bd5 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:58.598517 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679e1fe-b313-40c8-8b21-81f16a085bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:58.598789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679e1fe-b313-40c8-8b21-81f16a085bd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:58.609043 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679e1fe-b313-40c8-8b21-81f16a085bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:58.610519 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679e1fe-b313-40c8-8b21-81f16a085bd5 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:58.618439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:58.627984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-1679e1fe-b313-40c8-8b21-81f16a085bd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:58.627984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:58.627984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:58.627984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:58.627984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:58.627984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:58.627984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:58.627984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:58.627984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:58.627984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:58.627984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:58.627984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:58.627984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:51:58.627984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:51:58.627984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:58.628987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:58.630036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:51:58.630036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:51:58.630036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:51:58.630036 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:51:58.634042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 553/1101] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:58.634827 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679e1fe-b313-40c8-8b21-81f16a085bd5 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:58.683032 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd953d7-7f94-4eef-8e59-629d615304fa None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:58.722662 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82bcbe-7934-446f-9b7e-d678fb7f68e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:58.723335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82bcbe-7934-446f-9b7e-d678fb7f68e7 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:58.723619 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82bcbe-7934-446f-9b7e-d678fb7f68e7 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:58.723897 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82bcbe-7934-446f-9b7e-d678fb7f68e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:58.724747 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82bcbe-7934-446f-9b7e-d678fb7f68e7 None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=75f52549678b4c4fac4ef9fb82a31f23, user.id=28661e7300ca425182a14c7321793c87) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:58.725066 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82bcbe-7934-446f-9b7e-d678fb7f68e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:58.726553 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82bcbe-7934-446f-9b7e-d678fb7f68e7 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:58.732730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 549/1102] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:58 2022] GET /identity/v3/role_assignments?user.id=28661e7300ca425182a14c7321793c87&scope.project.id=75f52549678b4c4fac4ef9fb82a31f23&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:58.736269 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679e1fe-b313-40c8-8b21-81f16a085bd5 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:58.775801 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6981530-aa50-4e74-8aef-c3508165cb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:58.776482 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6981530-aa50-4e74-8aef-c3508165cb91 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:58.776833 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6981530-aa50-4e74-8aef-c3508165cb91 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:58.777112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6981530-aa50-4e74-8aef-c3508165cb91 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:58.777851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6981530-aa50-4e74-8aef-c3508165cb91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:58.778224 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6981530-aa50-4e74-8aef-c3508165cb91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:58.779971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6981530-aa50-4e74-8aef-c3508165cb91 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:58.792934 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 554/1103] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:51:58.799107 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82bcbe-7934-446f-9b7e-d678fb7f68e7 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:58.925948 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82bcbe-7934-446f-9b7e-d678fb7f68e7 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:58.927679 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:58.963862 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db464b82-f6b4-4b77-b69d-a8394b39dad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:58.964670 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db464b82-f6b4-4b77-b69d-a8394b39dad5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:58.964950 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db464b82-f6b4-4b77-b69d-a8394b39dad5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:58.965224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db464b82-f6b4-4b77-b69d-a8394b39dad5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:58.973017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db464b82-f6b4-4b77-b69d-a8394b39dad5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:58.974502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db464b82-f6b4-4b77-b69d-a8394b39dad5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:58.988646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-db464b82-f6b4-4b77-b69d-a8394b39dad5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:58.988646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:51:58.988646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:51:58.988646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:51:58.988646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:51:58.988646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:51:58.988646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:51:58.988646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:51:58.988646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:51:58.988646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:51:58.988646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:51:58.988646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:51:58.988646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:51:58.988646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:51:58.988646 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:51:58.989547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:51:58.990555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:51:58.990555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:51:58.990555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:51:58.990555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:51:58.990555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:51:58.990555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:51:58.990555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:51:58.990555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:51:58.990555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 550/1104] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:51:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:51:58.995773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6981530-aa50-4e74-8aef-c3508165cb91 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:59.022490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:59.098817 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6981530-aa50-4e74-8aef-c3508165cb91 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:59.134243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766655a7-1db5-4a39-98e0-7529f554feb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:59.134931 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766655a7-1db5-4a39-98e0-7529f554feb2 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:59.135214 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766655a7-1db5-4a39-98e0-7529f554feb2 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:59.135494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766655a7-1db5-4a39-98e0-7529f554feb2 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:59.136291 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766655a7-1db5-4a39-98e0-7529f554feb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:59.136602 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766655a7-1db5-4a39-98e0-7529f554feb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:59.138113 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766655a7-1db5-4a39-98e0-7529f554feb2 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:59.150734 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 555/1105] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:51:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:51:59.156942 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db464b82-f6b4-4b77-b69d-a8394b39dad5 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:51:59.259328 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db464b82-f6b4-4b77-b69d-a8394b39dad5 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:51:59.293949 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce6db2-9eae-4c00-92af-33eceefb634f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:51:59.294630 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce6db2-9eae-4c00-92af-33eceefb634f None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:51:59.294910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce6db2-9eae-4c00-92af-33eceefb634f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:51:59.295182 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce6db2-9eae-4c00-92af-33eceefb634f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:51:59.296015 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce6db2-9eae-4c00-92af-33eceefb634f None admin] RBAC: Adding query filter params (role.id=206b3ccf6eb842fc8e1d86592c6500b7, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=0da7f490d9604107b974e9e461ebc38b) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:51:59.296324 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce6db2-9eae-4c00-92af-33eceefb634f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:51:59.297751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce6db2-9eae-4c00-92af-33eceefb634f None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:51:59.303608 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 551/1106] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:51:59 2022] GET /identity/v3/role_assignments?user.id=0da7f490d9604107b974e9e461ebc38b&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=206b3ccf6eb842fc8e1d86592c6500b7 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:00.976090 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160758ed-0cbb-4303-8b2e-41c1f89d79e0 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:00.976618 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160758ed-0cbb-4303-8b2e-41c1f89d79e0 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:00.976618 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160758ed-0cbb-4303-8b2e-41c1f89d79e0 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:00.977921 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 556/1107] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:00.983648 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628df29-c96b-42ad-a4b6-84a1d163329e None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:00.984063 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628df29-c96b-42ad-a4b6-84a1d163329e None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:00.984198 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628df29-c96b-42ad-a4b6-84a1d163329e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:01.006337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-c628df29-c96b-42ad-a4b6-84a1d163329e None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:01.007555 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-c628df29-c96b-42ad-a4b6-84a1d163329e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:01.028643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:01.078112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 552/1108] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:52:01.109569 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160758ed-0cbb-4303-8b2e-41c1f89d79e0 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:01.206125 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:01.233166 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160758ed-0cbb-4303-8b2e-41c1f89d79e0 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:01.270362 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6dae4b-110d-4d36-89ac-b0f7876d3bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:01.271153 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6dae4b-110d-4d36-89ac-b0f7876d3bce None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:01.271432 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6dae4b-110d-4d36-89ac-b0f7876d3bce None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:01.271705 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6dae4b-110d-4d36-89ac-b0f7876d3bce None admin] PATH_INFO: `/v3/domains/default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:01.272935 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6dae4b-110d-4d36-89ac-b0f7876d3bce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:01.274411 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6dae4b-110d-4d36-89ac-b0f7876d3bce None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:01.276031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 557/1109] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:52:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:01.282702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628df29-c96b-42ad-a4b6-84a1d163329e None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:01.398736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628df29-c96b-42ad-a4b6-84a1d163329e None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:01.435818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecf6a7-e42d-4b78-b099-367602252bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:01.436493 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecf6a7-e42d-4b78-b099-367602252bca None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:01.436778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecf6a7-e42d-4b78-b099-367602252bca None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:01.437059 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecf6a7-e42d-4b78-b099-367602252bca None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:01.437778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecf6a7-e42d-4b78-b099-367602252bca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:01.439246 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecf6a7-e42d-4b78-b099-367602252bca None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:01.466702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 553/1110] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:52:01 2022] POST /identity/v3/users => generated 286 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:52:03.828803 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef068584-d318-48b2-a6f4-53af10b2be9b None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:03.829356 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef068584-d318-48b2-a6f4-53af10b2be9b None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:03.829356 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef068584-d318-48b2-a6f4-53af10b2be9b None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:03.830810 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 558/1111] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:03.836658 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d5c26-e691-4bc2-8282-93a38110dbe8 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:03.837136 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d5c26-e691-4bc2-8282-93a38110dbe8 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:03.837226 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d5c26-e691-4bc2-8282-93a38110dbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:03.869294 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-375d5c26-e691-4bc2-8282-93a38110dbe8 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:03.870751 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-375d5c26-e691-4bc2-8282-93a38110dbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:03.879317 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:03.949904 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 554/1112] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:52:03.980429 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef068584-d318-48b2-a6f4-53af10b2be9b None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:04.086402 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef068584-d318-48b2-a6f4-53af10b2be9b None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:04.122110 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a952d9a-de29-4cc6-9757-2404b3f7fdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:04.123005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a952d9a-de29-4cc6-9757-2404b3f7fdc1 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:04.123283 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a952d9a-de29-4cc6-9757-2404b3f7fdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:04.123553 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a952d9a-de29-4cc6-9757-2404b3f7fdc1 None admin] PATH_INFO: `/v3/roles/964e02e925f147e69a9e02e02a69612b` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:04.125111 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a952d9a-de29-4cc6-9757-2404b3f7fdc1 None admin] RBAC: Authorizing `identity:get_role(role_id=964e02e925f147e69a9e02e02a69612b)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:04.126538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a952d9a-de29-4cc6-9757-2404b3f7fdc1 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:04.128014 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 559/1113] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:52:03 2022] GET /identity/v3/roles/964e02e925f147e69a9e02e02a69612b => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:04.134359 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d5c26-e691-4bc2-8282-93a38110dbe8 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:04.241001 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d5c26-e691-4bc2-8282-93a38110dbe8 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:04.277103 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c2c14-93a3-47b2-a427-35a17fb012f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:04.278000 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c2c14-93a3-47b2-a427-35a17fb012f3 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:04.278291 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c2c14-93a3-47b2-a427-35a17fb012f3 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:04.278590 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c2c14-93a3-47b2-a427-35a17fb012f3 None admin] PATH_INFO: `/v3/users/7dd248c996594d6b9f1b91f91216fa86` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:04.289888 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:04.293735 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c2c14-93a3-47b2-a427-35a17fb012f3 None admin] RBAC: Authorizing `identity:get_user(user_id=7dd248c996594d6b9f1b91f91216fa86)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:04.296206 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c2c14-93a3-47b2-a427-35a17fb012f3 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:04.299374 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 555/1114] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:52:04 2022] GET /identity/v3/users/7dd248c996594d6b9f1b91f91216fa86 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:04.308966 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a952d9a-de29-4cc6-9757-2404b3f7fdc1 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:04.341077 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:04.436571 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a952d9a-de29-4cc6-9757-2404b3f7fdc1 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:04.485453 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07331a2-e341-40c3-bb7c-3aece8d08c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:04.486111 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07331a2-e341-40c3-bb7c-3aece8d08c8a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:04.486424 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07331a2-e341-40c3-bb7c-3aece8d08c8a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:04.486699 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07331a2-e341-40c3-bb7c-3aece8d08c8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:04.558678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07331a2-e341-40c3-bb7c-3aece8d08c8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:04.560171 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07331a2-e341-40c3-bb7c-3aece8d08c8a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:04.582452 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:04.669103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 560/1115] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:52:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:52:04.676029 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c2c14-93a3-47b2-a427-35a17fb012f3 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:04.786208 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c2c14-93a3-47b2-a427-35a17fb012f3 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:04.832645 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6dc21-f6fa-4dc7-a206-a45b314faf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:04.833443 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6dc21-f6fa-4dc7-a206-a45b314faf74 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:04.833728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6dc21-f6fa-4dc7-a206-a45b314faf74 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:04.834009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6dc21-f6fa-4dc7-a206-a45b314faf74 None admin] PATH_INFO: `/v3/projects/75f52549678b4c4fac4ef9fb82a31f23` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:04.835377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6dc21-f6fa-4dc7-a206-a45b314faf74 None admin] RBAC: Authorizing `identity:get_project(project_id=75f52549678b4c4fac4ef9fb82a31f23)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:04.836829 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6dc21-f6fa-4dc7-a206-a45b314faf74 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:04.838588 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 556/1116] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:52:04 2022] GET /identity/v3/projects/75f52549678b4c4fac4ef9fb82a31f23 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:04.845140 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07331a2-e341-40c3-bb7c-3aece8d08c8a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:04.950898 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07331a2-e341-40c3-bb7c-3aece8d08c8a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:04.973918 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:04.989965 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2457b-884a-43f6-b3a8-c5efe8c9e5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:04.990754 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2457b-884a-43f6-b3a8-c5efe8c9e5ec None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:04.991035 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2457b-884a-43f6-b3a8-c5efe8c9e5ec None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:04.991311 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2457b-884a-43f6-b3a8-c5efe8c9e5ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:04.992156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2457b-884a-43f6-b3a8-c5efe8c9e5ec None admin] RBAC: Adding query filter params (role.id=964e02e925f147e69a9e02e02a69612b, scope.project.id=75f52549678b4c4fac4ef9fb82a31f23, user.id=7dd248c996594d6b9f1b91f91216fa86) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:52:04.992470 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2457b-884a-43f6-b3a8-c5efe8c9e5ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:04.993911 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2457b-884a-43f6-b3a8-c5efe8c9e5ec None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:04.999741 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 561/1117] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:52:04 2022] GET /identity/v3/role_assignments?user.id=7dd248c996594d6b9f1b91f91216fa86&scope.project.id=75f52549678b4c4fac4ef9fb82a31f23&role.id=964e02e925f147e69a9e02e02a69612b => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:07.158026 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822c029-85b6-45e5-83bc-899cc16fa64d None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:07.158637 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822c029-85b6-45e5-83bc-899cc16fa64d None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:07.158637 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822c029-85b6-45e5-83bc-899cc16fa64d None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:07.159999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 557/1118] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:07.165389 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e170b1-2a99-420a-b9af-d7ff8babf095 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:07.165821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e170b1-2a99-420a-b9af-d7ff8babf095 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:07.165930 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e170b1-2a99-420a-b9af-d7ff8babf095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:07.184780 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-10e170b1-2a99-420a-b9af-d7ff8babf095 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:07.185976 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-10e170b1-2a99-420a-b9af-d7ff8babf095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:07.257199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 562/1119] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:52:07.279891 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822c029-85b6-45e5-83bc-899cc16fa64d None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:07.358195 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:07.394408 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822c029-85b6-45e5-83bc-899cc16fa64d None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:07.430006 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c308f3-94e3-4624-ada0-c46c4b7eca85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:07.430838 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c308f3-94e3-4624-ada0-c46c4b7eca85 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:07.431123 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c308f3-94e3-4624-ada0-c46c4b7eca85 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:07.431396 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c308f3-94e3-4624-ada0-c46c4b7eca85 None admin] PATH_INFO: `/v3/roles/964e02e925f147e69a9e02e02a69612b` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:07.433004 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c308f3-94e3-4624-ada0-c46c4b7eca85 None admin] RBAC: Authorizing `identity:get_role(role_id=964e02e925f147e69a9e02e02a69612b)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:07.434451 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c308f3-94e3-4624-ada0-c46c4b7eca85 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:07.435941 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 558/1120] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:52:07 2022] GET /identity/v3/roles/964e02e925f147e69a9e02e02a69612b => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:07.445709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e170b1-2a99-420a-b9af-d7ff8babf095 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:07.558510 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e170b1-2a99-420a-b9af-d7ff8babf095 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:07.572939 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:07.595795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2389340-913c-4a93-96f8-9eccc8b47260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:07.596995 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2389340-913c-4a93-96f8-9eccc8b47260 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:07.597278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2389340-913c-4a93-96f8-9eccc8b47260 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:07.597557 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2389340-913c-4a93-96f8-9eccc8b47260 None admin] PATH_INFO: `/v3/users/7dd248c996594d6b9f1b91f91216fa86` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:07.598809 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2389340-913c-4a93-96f8-9eccc8b47260 None admin] RBAC: Authorizing `identity:get_user(user_id=7dd248c996594d6b9f1b91f91216fa86)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:07.600260 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2389340-913c-4a93-96f8-9eccc8b47260 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:07.601917 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 563/1121] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:52:07 2022] GET /identity/v3/users/7dd248c996594d6b9f1b91f91216fa86 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:07.608291 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c308f3-94e3-4624-ada0-c46c4b7eca85 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:07.717397 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c308f3-94e3-4624-ada0-c46c4b7eca85 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:07.752516 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4977762-4d27-4285-b8fb-cc87df0da5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:07.753392 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4977762-4d27-4285-b8fb-cc87df0da5c4 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:07.753680 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4977762-4d27-4285-b8fb-cc87df0da5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:07.753958 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4977762-4d27-4285-b8fb-cc87df0da5c4 None admin] PATH_INFO: `/v3/projects/75f52549678b4c4fac4ef9fb82a31f23` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:07.755319 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4977762-4d27-4285-b8fb-cc87df0da5c4 None admin] RBAC: Authorizing `identity:get_project(project_id=75f52549678b4c4fac4ef9fb82a31f23)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:07.756774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4977762-4d27-4285-b8fb-cc87df0da5c4 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:07.758524 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 559/1122] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:52:07 2022] GET /identity/v3/projects/75f52549678b4c4fac4ef9fb82a31f23 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:07.765542 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2389340-913c-4a93-96f8-9eccc8b47260 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:07.878688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2389340-913c-4a93-96f8-9eccc8b47260 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:07.914453 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114f366-c963-46dc-b17b-67553d350727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:07.915340 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114f366-c963-46dc-b17b-67553d350727 None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:07.915619 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114f366-c963-46dc-b17b-67553d350727 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:07.915893 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114f366-c963-46dc-b17b-67553d350727 None admin] PATH_INFO: `/v3/projects/75f52549678b4c4fac4ef9fb82a31f23/users/7dd248c996594d6b9f1b91f91216fa86/roles/964e02e925f147e69a9e02e02a69612b` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:07.918304 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114f366-c963-46dc-b17b-67553d350727 None admin] RBAC: Authorizing `identity:create_grant(project_id=75f52549678b4c4fac4ef9fb82a31f23, user_id=7dd248c996594d6b9f1b91f91216fa86, role_id=964e02e925f147e69a9e02e02a69612b)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:07.919857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114f366-c963-46dc-b17b-67553d350727 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:07.942789 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 564/1123] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:52:07 2022] PUT /identity/v3/projects/75f52549678b4c4fac4ef9fb82a31f23/users/7dd248c996594d6b9f1b91f91216fa86/roles/964e02e925f147e69a9e02e02a69612b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:52:10.132825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26861e06-1a60-4fbd-8c1d-4e6fc6f7dbc4 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:10.133403 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26861e06-1a60-4fbd-8c1d-4e6fc6f7dbc4 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:10.133403 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26861e06-1a60-4fbd-8c1d-4e6fc6f7dbc4 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:10.134804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 560/1124] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:10.141373 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc7551-7e41-40df-8656-fcc53c24c2fd None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:10.141802 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc7551-7e41-40df-8656-fcc53c24c2fd None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:10.141898 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc7551-7e41-40df-8656-fcc53c24c2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:10.169589 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-8fdc7551-7e41-40df-8656-fcc53c24c2fd None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:10.170840 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-8fdc7551-7e41-40df-8656-fcc53c24c2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:10.209472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:10.242065 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 565/1125] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:52:10.265728 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26861e06-1a60-4fbd-8c1d-4e6fc6f7dbc4 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:10.270119 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:10.376347 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26861e06-1a60-4fbd-8c1d-4e6fc6f7dbc4 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:10.412879 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02903e04-1144-41b8-8c2e-16d1eedc04ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:10.413674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02903e04-1144-41b8-8c2e-16d1eedc04ff None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:10.413952 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02903e04-1144-41b8-8c2e-16d1eedc04ff None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:10.414221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02903e04-1144-41b8-8c2e-16d1eedc04ff None admin] PATH_INFO: `/v3/roles/964e02e925f147e69a9e02e02a69612b` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:10.415870 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02903e04-1144-41b8-8c2e-16d1eedc04ff None admin] RBAC: Authorizing `identity:get_role(role_id=964e02e925f147e69a9e02e02a69612b)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:10.417254 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02903e04-1144-41b8-8c2e-16d1eedc04ff None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:10.418764 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 561/1126] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:52:10 2022] GET /identity/v3/roles/964e02e925f147e69a9e02e02a69612b => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:10.425520 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc7551-7e41-40df-8656-fcc53c24c2fd None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:10.548560 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc7551-7e41-40df-8656-fcc53c24c2fd None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:10.584562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729645f-9990-4c0b-be7d-968496cdd53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:10.585426 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729645f-9990-4c0b-be7d-968496cdd53f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:10.585704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729645f-9990-4c0b-be7d-968496cdd53f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:10.585976 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729645f-9990-4c0b-be7d-968496cdd53f None admin] PATH_INFO: `/v3/users/7dd248c996594d6b9f1b91f91216fa86` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:10.587240 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729645f-9990-4c0b-be7d-968496cdd53f None admin] RBAC: Authorizing `identity:get_user(user_id=7dd248c996594d6b9f1b91f91216fa86)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:10.588677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729645f-9990-4c0b-be7d-968496cdd53f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:10.590456 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 566/1127] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:52:10 2022] GET /identity/v3/users/7dd248c996594d6b9f1b91f91216fa86 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:10.600014 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02903e04-1144-41b8-8c2e-16d1eedc04ff None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:10.679929 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:10.719575 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02903e04-1144-41b8-8c2e-16d1eedc04ff None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:10.755217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7b526-4595-4caa-890b-1598cd6acdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:10.755869 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7b526-4595-4caa-890b-1598cd6acdfb None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:10.756155 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7b526-4595-4caa-890b-1598cd6acdfb None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:10.756431 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7b526-4595-4caa-890b-1598cd6acdfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:10.841002 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7b526-4595-4caa-890b-1598cd6acdfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:10.842491 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7b526-4595-4caa-890b-1598cd6acdfb None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:10.912197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:10.943702 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 562/1128] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:52:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:52:10.950515 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729645f-9990-4c0b-be7d-968496cdd53f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:11.021830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:11.093719 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729645f-9990-4c0b-be7d-968496cdd53f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:11.128678 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2c05e-2b0c-4b7c-803d-bcb09700a7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:11.129532 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2c05e-2b0c-4b7c-803d-bcb09700a7ee None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:11.129814 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2c05e-2b0c-4b7c-803d-bcb09700a7ee None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:11.130085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2c05e-2b0c-4b7c-803d-bcb09700a7ee None admin] PATH_INFO: `/v3/projects/75f52549678b4c4fac4ef9fb82a31f23` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:11.131344 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2c05e-2b0c-4b7c-803d-bcb09700a7ee None admin] RBAC: Authorizing `identity:get_project(project_id=75f52549678b4c4fac4ef9fb82a31f23)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:11.132785 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2c05e-2b0c-4b7c-803d-bcb09700a7ee None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:11.134517 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 567/1129] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:52:10 2022] GET /identity/v3/projects/75f52549678b4c4fac4ef9fb82a31f23 => generated 311 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:11.141112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7b526-4595-4caa-890b-1598cd6acdfb None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:11.247157 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7b526-4595-4caa-890b-1598cd6acdfb None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:11.282868 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccaef1c-9b25-4fd4-bbe8-8ca848a2cbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:11.283508 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccaef1c-9b25-4fd4-bbe8-8ca848a2cbb8 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:11.283786 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccaef1c-9b25-4fd4-bbe8-8ca848a2cbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:11.284055 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccaef1c-9b25-4fd4-bbe8-8ca848a2cbb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:11.284891 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccaef1c-9b25-4fd4-bbe8-8ca848a2cbb8 None admin] RBAC: Adding query filter params (role.id=964e02e925f147e69a9e02e02a69612b, scope.project.id=75f52549678b4c4fac4ef9fb82a31f23, user.id=7dd248c996594d6b9f1b91f91216fa86) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:52:11.285202 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccaef1c-9b25-4fd4-bbe8-8ca848a2cbb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:11.286661 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccaef1c-9b25-4fd4-bbe8-8ca848a2cbb8 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:11.292609 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 563/1130] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:52:11 2022] GET /identity/v3/role_assignments?user.id=7dd248c996594d6b9f1b91f91216fa86&scope.project.id=75f52549678b4c4fac4ef9fb82a31f23&role.id=964e02e925f147e69a9e02e02a69612b => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:13.523759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c05d2-300f-4b5d-a12c-82a4d5615641 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:13.524675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c05d2-300f-4b5d-a12c-82a4d5615641 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:13.524675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c05d2-300f-4b5d-a12c-82a4d5615641 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:13.526998 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 568/1131] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:13.532577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fe931-d4dc-4d52-84ff-6b24eb2329b3 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:13.533008 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fe931-d4dc-4d52-84ff-6b24eb2329b3 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:13.533112 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fe931-d4dc-4d52-84ff-6b24eb2329b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:13.551720 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-650fe931-d4dc-4d52-84ff-6b24eb2329b3 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:13.552904 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-650fe931-d4dc-4d52-84ff-6b24eb2329b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:13.583535 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:13.626089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 564/1132] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:52:13.649882 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c05d2-300f-4b5d-a12c-82a4d5615641 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:13.763934 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c05d2-300f-4b5d-a12c-82a4d5615641 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:13.772881 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:13.813969 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c845133-89f3-4ede-b965-b1ee6109bf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:13.815084 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c845133-89f3-4ede-b965-b1ee6109bf66 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:13.815376 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c845133-89f3-4ede-b965-b1ee6109bf66 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:13.815658 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c845133-89f3-4ede-b965-b1ee6109bf66 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:13.816909 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c845133-89f3-4ede-b965-b1ee6109bf66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:13.818396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c845133-89f3-4ede-b965-b1ee6109bf66 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:13.820062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 569/1133] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:52:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:13.828973 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fe931-d4dc-4d52-84ff-6b24eb2329b3 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:13.932620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fe931-d4dc-4d52-84ff-6b24eb2329b3 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:13.967494 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351acb5-03e0-40b5-8371-ea7e0b178863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:13.968149 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351acb5-03e0-40b5-8371-ea7e0b178863 None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:13.968431 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351acb5-03e0-40b5-8371-ea7e0b178863 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:13.968708 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351acb5-03e0-40b5-8371-ea7e0b178863 None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:13.969423 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351acb5-03e0-40b5-8371-ea7e0b178863 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:13.971232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351acb5-03e0-40b5-8371-ea7e0b178863 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:14.000713 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 565/1134] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:52:13 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:52:16.303763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a00b4-96a1-415f-a7be-6c6b19277d0c None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:16.304648 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a00b4-96a1-415f-a7be-6c6b19277d0c None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:16.304648 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a00b4-96a1-415f-a7be-6c6b19277d0c None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:16.306906 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 570/1135] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:16.315547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ae9e2-a8f8-4d9f-86c0-799e1bfbcb9a None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:16.316000 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ae9e2-a8f8-4d9f-86c0-799e1bfbcb9a None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:16.316096 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ae9e2-a8f8-4d9f-86c0-799e1bfbcb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:16.335531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-280ae9e2-a8f8-4d9f-86c0-799e1bfbcb9a None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:16.336716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-280ae9e2-a8f8-4d9f-86c0-799e1bfbcb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:16.352483 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:16.425223 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 566/1136] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:52:16.460850 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a00b4-96a1-415f-a7be-6c6b19277d0c None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:16.586567 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a00b4-96a1-415f-a7be-6c6b19277d0c None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:16.625763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb4cd2-660b-4b8a-8631-1aa2f2a3ceda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:16.626592 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb4cd2-660b-4b8a-8631-1aa2f2a3ceda None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:16.626872 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb4cd2-660b-4b8a-8631-1aa2f2a3ceda None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:16.627143 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb4cd2-660b-4b8a-8631-1aa2f2a3ceda None admin] PATH_INFO: `/v3/domains/default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:16.628480 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb4cd2-660b-4b8a-8631-1aa2f2a3ceda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:16.629945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb4cd2-660b-4b8a-8631-1aa2f2a3ceda None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:16.631640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 571/1137] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:52:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:16.638902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ae9e2-a8f8-4d9f-86c0-799e1bfbcb9a None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:16.753095 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ae9e2-a8f8-4d9f-86c0-799e1bfbcb9a None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:16.792978 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4230d45-63f6-44e5-9539-11f9d481381e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:16.793987 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4230d45-63f6-44e5-9539-11f9d481381e None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:16.794308 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4230d45-63f6-44e5-9539-11f9d481381e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:16.794592 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4230d45-63f6-44e5-9539-11f9d481381e None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:16.795314 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4230d45-63f6-44e5-9539-11f9d481381e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:16.796782 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4230d45-63f6-44e5-9539-11f9d481381e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:16.823318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 567/1138] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:52:16 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:52:19.190522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8fbde-6173-4c1b-96af-e5d3400e088a None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:19.191396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8fbde-6173-4c1b-96af-e5d3400e088a None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:19.191396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8fbde-6173-4c1b-96af-e5d3400e088a None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:19.193584 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 572/1139] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:19.201792 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b367c-0d05-4308-b5db-96c356654f34 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:19.202193 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b367c-0d05-4308-b5db-96c356654f34 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:19.202375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b367c-0d05-4308-b5db-96c356654f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:19.209818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:19.223497 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-b87b367c-0d05-4308-b5db-96c356654f34 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:19.225133 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-b87b367c-0d05-4308-b5db-96c356654f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:19.296056 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 568/1140] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:52:19.332462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8fbde-6173-4c1b-96af-e5d3400e088a None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:19.385867 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:19.448825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8fbde-6173-4c1b-96af-e5d3400e088a None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:19.489764 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecf6c4-a852-47df-849b-6bb52adf8f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:19.490968 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecf6c4-a852-47df-849b-6bb52adf8f5a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:19.491324 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecf6c4-a852-47df-849b-6bb52adf8f5a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:19.491664 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecf6c4-a852-47df-849b-6bb52adf8f5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:19.511204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ecf6c4-a852-47df-849b-6bb52adf8f5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-c0ecf6c4-a852-47df-849b-6bb52adf8f5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:52:19.519116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:52:19.520127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:52:19.521021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:52:19.521021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:52:19.521021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:52:19.521021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:52:19.521021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:52:19.521021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:52:19.521021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:52:19.521021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:52:19.521021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:52:19.521021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:52:19.521021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:52:19.521021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:52:19.521021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 573/1141] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:52:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:52:19.528689 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b367c-0d05-4308-b5db-96c356654f34 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:19.649500 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b367c-0d05-4308-b5db-96c356654f34 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:19.685945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6139a0-8eb2-4805-83c2-235cdc216419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:19.686643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6139a0-8eb2-4805-83c2-235cdc216419 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:19.686929 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6139a0-8eb2-4805-83c2-235cdc216419 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:19.687207 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6139a0-8eb2-4805-83c2-235cdc216419 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:19.687971 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6139a0-8eb2-4805-83c2-235cdc216419 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:52:19.688284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6139a0-8eb2-4805-83c2-235cdc216419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:19.689678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6139a0-8eb2-4805-83c2-235cdc216419 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:19.691082 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:19.699292 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 569/1142] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:52:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:19.705422 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecf6c4-a852-47df-849b-6bb52adf8f5a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:19.813798 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecf6c4-a852-47df-849b-6bb52adf8f5a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:19.816232 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:19.865291 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dd7ec-b8a0-4b64-b500-2c044a830cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:19.866591 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dd7ec-b8a0-4b64-b500-2c044a830cde None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:19.866898 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dd7ec-b8a0-4b64-b500-2c044a830cde None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:19.867197 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dd7ec-b8a0-4b64-b500-2c044a830cde None admin] PATH_INFO: `/v3/users/c829da0ff81f4f6d99d94afeb018caf8` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:19.884016 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658dd7ec-b8a0-4b64-b500-2c044a830cde None admin] RBAC: Authorizing `identity:get_user(user_id=c829da0ff81f4f6d99d94afeb018caf8)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:19.885516 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658dd7ec-b8a0-4b64-b500-2c044a830cde None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:19.887345 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 574/1143] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:52:19 2022] GET /identity/v3/users/c829da0ff81f4f6d99d94afeb018caf8 => generated 286 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:19.893700 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6139a0-8eb2-4805-83c2-235cdc216419 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:20.001465 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6139a0-8eb2-4805-83c2-235cdc216419 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:20.037041 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db360c-024d-48c4-8e56-adba28da1b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:20.037686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db360c-024d-48c4-8e56-adba28da1b01 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:20.037972 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db360c-024d-48c4-8e56-adba28da1b01 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:20.038247 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db360c-024d-48c4-8e56-adba28da1b01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:20.107624 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:20.115765 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db360c-024d-48c4-8e56-adba28da1b01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:20.117230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db360c-024d-48c4-8e56-adba28da1b01 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:20.226466 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 570/1144] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:52:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:52:20.232607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dd7ec-b8a0-4b64-b500-2c044a830cde None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:20.354544 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dd7ec-b8a0-4b64-b500-2c044a830cde None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:20.391247 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6b82c-4ce1-434a-a201-c9b14c0f1cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:20.392111 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6b82c-4ce1-434a-a201-c9b14c0f1cd4 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:20.392394 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6b82c-4ce1-434a-a201-c9b14c0f1cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:20.392672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6b82c-4ce1-434a-a201-c9b14c0f1cd4 None admin] PATH_INFO: `/v3/projects/da4cd687b35d4dac81871c6d26213346` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:20.393918 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6b82c-4ce1-434a-a201-c9b14c0f1cd4 None admin] RBAC: Authorizing `identity:get_project(project_id=da4cd687b35d4dac81871c6d26213346)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:20.395396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6b82c-4ce1-434a-a201-c9b14c0f1cd4 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:20.397116 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 575/1145] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:52:20 2022] GET /identity/v3/projects/da4cd687b35d4dac81871c6d26213346 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:20.403936 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db360c-024d-48c4-8e56-adba28da1b01 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:20.505370 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:20.513110 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db360c-024d-48c4-8e56-adba28da1b01 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:20.549117 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa3940-f07f-4746-afe6-83e1236f894d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:20.549771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa3940-f07f-4746-afe6-83e1236f894d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:20.550051 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa3940-f07f-4746-afe6-83e1236f894d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:20.550351 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa3940-f07f-4746-afe6-83e1236f894d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:20.551199 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa3940-f07f-4746-afe6-83e1236f894d None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=da4cd687b35d4dac81871c6d26213346, user.id=c829da0ff81f4f6d99d94afeb018caf8) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:52:20.551501 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa3940-f07f-4746-afe6-83e1236f894d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:20.552957 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa3940-f07f-4746-afe6-83e1236f894d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:20.559042 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 571/1146] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:52:20 2022] GET /identity/v3/role_assignments?user.id=c829da0ff81f4f6d99d94afeb018caf8&scope.project.id=da4cd687b35d4dac81871c6d26213346&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:22.796900 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c414ca7-7d5d-465f-840f-7caf41a0fe97 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:22.797708 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c414ca7-7d5d-465f-840f-7caf41a0fe97 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:22.797823 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c414ca7-7d5d-465f-840f-7caf41a0fe97 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:22.800114 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 576/1147] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:22.805600 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc8387-96df-443b-8543-5b26ead2a663 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:22.806042 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc8387-96df-443b-8543-5b26ead2a663 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:22.806131 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc8387-96df-443b-8543-5b26ead2a663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:22.824977 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-92bc8387-96df-443b-8543-5b26ead2a663 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:22.826181 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-92bc8387-96df-443b-8543-5b26ead2a663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:22.897925 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 572/1148] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:52:22.921193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c414ca7-7d5d-465f-840f-7caf41a0fe97 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:22.953506 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:23.045511 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c414ca7-7d5d-465f-840f-7caf41a0fe97 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:23.082594 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874b7be-d875-48c8-91d9-6d0b8f4444bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:23.083390 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874b7be-d875-48c8-91d9-6d0b8f4444bd None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:23.083666 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874b7be-d875-48c8-91d9-6d0b8f4444bd None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:23.083933 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874b7be-d875-48c8-91d9-6d0b8f4444bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:23.095054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874b7be-d875-48c8-91d9-6d0b8f4444bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-7874b7be-d875-48c8-91d9-6d0b8f4444bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:52:23.103022 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:52:23.104103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:52:23.104927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:52:23.104927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:52:23.104927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:52:23.104927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:52:23.104927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:52:23.104927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:52:23.104927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:52:23.104927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:52:23.104927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:52:23.104927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:52:23.104927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:52:23.104927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:52:23.104927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 577/1149] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:52:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:52:23.112056 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc8387-96df-443b-8543-5b26ead2a663 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:23.223981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:23.238983 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc8387-96df-443b-8543-5b26ead2a663 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:23.279492 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809bc72-e5ac-40df-82bb-332e2c515c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:23.280158 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809bc72-e5ac-40df-82bb-332e2c515c72 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:23.280437 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809bc72-e5ac-40df-82bb-332e2c515c72 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:23.280713 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809bc72-e5ac-40df-82bb-332e2c515c72 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:23.281471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809bc72-e5ac-40df-82bb-332e2c515c72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:52:23.281779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809bc72-e5ac-40df-82bb-332e2c515c72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:23.283190 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809bc72-e5ac-40df-82bb-332e2c515c72 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:23.291366 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 573/1150] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:52:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:52:23.300713 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874b7be-d875-48c8-91d9-6d0b8f4444bd None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:23.398910 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:23.409828 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874b7be-d875-48c8-91d9-6d0b8f4444bd None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:23.444585 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961555d4-7882-468b-a4dc-c8bab566eceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:23.445445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961555d4-7882-468b-a4dc-c8bab566eceb None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:23.445720 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961555d4-7882-468b-a4dc-c8bab566eceb None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:23.445991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961555d4-7882-468b-a4dc-c8bab566eceb None admin] PATH_INFO: `/v3/users/c829da0ff81f4f6d99d94afeb018caf8` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:23.447255 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961555d4-7882-468b-a4dc-c8bab566eceb None admin] RBAC: Authorizing `identity:get_user(user_id=c829da0ff81f4f6d99d94afeb018caf8)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:23.448693 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961555d4-7882-468b-a4dc-c8bab566eceb None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:23.450458 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 578/1151] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:52:23 2022] GET /identity/v3/users/c829da0ff81f4f6d99d94afeb018caf8 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:23.457068 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809bc72-e5ac-40df-82bb-332e2c515c72 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:23.563982 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809bc72-e5ac-40df-82bb-332e2c515c72 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:23.599603 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49208451-5ff8-4096-9f86-f494f03b0859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:23.600460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49208451-5ff8-4096-9f86-f494f03b0859 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:23.600736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49208451-5ff8-4096-9f86-f494f03b0859 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:23.601010 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49208451-5ff8-4096-9f86-f494f03b0859 None admin] PATH_INFO: `/v3/projects/da4cd687b35d4dac81871c6d26213346` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:23.602327 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49208451-5ff8-4096-9f86-f494f03b0859 None admin] RBAC: Authorizing `identity:get_project(project_id=da4cd687b35d4dac81871c6d26213346)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:23.603800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49208451-5ff8-4096-9f86-f494f03b0859 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:23.605556 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 574/1152] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:52:23 2022] GET /identity/v3/projects/da4cd687b35d4dac81871c6d26213346 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:23.614891 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961555d4-7882-468b-a4dc-c8bab566eceb None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:23.735745 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961555d4-7882-468b-a4dc-c8bab566eceb None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:23.772028 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfbe58-94b6-449c-93b6-f40add9598a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:23.772922 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfbe58-94b6-449c-93b6-f40add9598a2 None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:23.773201 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfbe58-94b6-449c-93b6-f40add9598a2 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:23.773481 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfbe58-94b6-449c-93b6-f40add9598a2 None admin] PATH_INFO: `/v3/projects/da4cd687b35d4dac81871c6d26213346/users/c829da0ff81f4f6d99d94afeb018caf8/roles/2cd6d3dd613c4824a26cacf53fb6c07f` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:23.775776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfbe58-94b6-449c-93b6-f40add9598a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=da4cd687b35d4dac81871c6d26213346, user_id=c829da0ff81f4f6d99d94afeb018caf8, role_id=2cd6d3dd613c4824a26cacf53fb6c07f)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:23.777222 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfbe58-94b6-449c-93b6-f40add9598a2 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:23.794548 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 579/1153] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:52:23 2022] PUT /identity/v3/projects/da4cd687b35d4dac81871c6d26213346/users/c829da0ff81f4f6d99d94afeb018caf8/roles/2cd6d3dd613c4824a26cacf53fb6c07f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:52:26.058563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317225bf-d0c4-4f86-9fc1-5408fccf786f None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:26.059512 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317225bf-d0c4-4f86-9fc1-5408fccf786f None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:26.059512 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317225bf-d0c4-4f86-9fc1-5408fccf786f None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:26.061683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 575/1154] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:26.068151 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7859b27-2ef8-4a06-910f-1aaa1c3dc3c0 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:26.068919 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7859b27-2ef8-4a06-910f-1aaa1c3dc3c0 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:26.069020 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7859b27-2ef8-4a06-910f-1aaa1c3dc3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:26.089558 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-b7859b27-2ef8-4a06-910f-1aaa1c3dc3c0 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:26.090783 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-b7859b27-2ef8-4a06-910f-1aaa1c3dc3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:26.104843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:26.163030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 580/1155] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:52:26.187532 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317225bf-d0c4-4f86-9fc1-5408fccf786f None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:26.217354 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:26.309380 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317225bf-d0c4-4f86-9fc1-5408fccf786f None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:26.349012 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fe441-26e0-473e-88c4-086a3dbce719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:26.349813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fe441-26e0-473e-88c4-086a3dbce719 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:26.350096 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fe441-26e0-473e-88c4-086a3dbce719 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:26.350409 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fe441-26e0-473e-88c4-086a3dbce719 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:26.361272 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181fe441-26e0-473e-88c4-086a3dbce719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-181fe441-26e0-473e-88c4-086a3dbce719 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:52:26.369020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:52:26.370101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:52:26.371716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:52:26.371716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:52:26.371716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:52:26.371716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:52:26.371716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:52:26.371716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:52:26.371716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:52:26.371716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:52:26.371716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:52:26.371716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:52:26.371716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:52:26.371716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:52:26.371716 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 576/1156] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:52:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:52:26.379570 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7859b27-2ef8-4a06-910f-1aaa1c3dc3c0 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:26.494309 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7859b27-2ef8-4a06-910f-1aaa1c3dc3c0 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:26.529710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd6bad-3854-442c-ba89-138902b275a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:26.530396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd6bad-3854-442c-ba89-138902b275a7 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:26.530671 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd6bad-3854-442c-ba89-138902b275a7 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:26.530949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd6bad-3854-442c-ba89-138902b275a7 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:26.531700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd6bad-3854-442c-ba89-138902b275a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:52:26.532002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd6bad-3854-442c-ba89-138902b275a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:26.533383 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd6bad-3854-442c-ba89-138902b275a7 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:26.541554 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 581/1157] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:52:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:26.551226 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fe441-26e0-473e-88c4-086a3dbce719 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:26.655105 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:26.676333 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fe441-26e0-473e-88c4-086a3dbce719 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:26.713344 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408c748-cb7d-499d-a6cc-fde4510e6867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:26.714235 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408c748-cb7d-499d-a6cc-fde4510e6867 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:26.714591 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408c748-cb7d-499d-a6cc-fde4510e6867 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:26.714828 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408c748-cb7d-499d-a6cc-fde4510e6867 None admin] PATH_INFO: `/v3/users/c829da0ff81f4f6d99d94afeb018caf8` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:26.716530 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408c748-cb7d-499d-a6cc-fde4510e6867 None admin] RBAC: Authorizing `identity:get_user(user_id=c829da0ff81f4f6d99d94afeb018caf8)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:26.718933 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408c748-cb7d-499d-a6cc-fde4510e6867 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:26.721967 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 577/1158] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:52:26 2022] GET /identity/v3/users/c829da0ff81f4f6d99d94afeb018caf8 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:26.730658 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd6bad-3854-442c-ba89-138902b275a7 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:26.743928 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:26.839305 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd6bad-3854-442c-ba89-138902b275a7 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:26.875838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ee38c-ea6e-43bd-8a6c-ab918472c02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:26.876493 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ee38c-ea6e-43bd-8a6c-ab918472c02f None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:26.876781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ee38c-ea6e-43bd-8a6c-ab918472c02f None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:26.877060 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ee38c-ea6e-43bd-8a6c-ab918472c02f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:26.948778 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ee38c-ea6e-43bd-8a6c-ab918472c02f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:26.950812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ee38c-ea6e-43bd-8a6c-ab918472c02f None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:26.962154 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:27.076874 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 582/1159] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:52:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:52:27.086630 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408c748-cb7d-499d-a6cc-fde4510e6867 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:27.215185 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408c748-cb7d-499d-a6cc-fde4510e6867 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:27.255841 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fda58f-37a7-487d-a947-71dafab5ea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:27.256700 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fda58f-37a7-487d-a947-71dafab5ea2e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:27.256981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fda58f-37a7-487d-a947-71dafab5ea2e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:27.257260 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fda58f-37a7-487d-a947-71dafab5ea2e None admin] PATH_INFO: `/v3/projects/da4cd687b35d4dac81871c6d26213346` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:27.258539 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fda58f-37a7-487d-a947-71dafab5ea2e None admin] RBAC: Authorizing `identity:get_project(project_id=da4cd687b35d4dac81871c6d26213346)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:27.260009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fda58f-37a7-487d-a947-71dafab5ea2e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:27.261911 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 578/1160] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:52:27 2022] GET /identity/v3/projects/da4cd687b35d4dac81871c6d26213346 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:27.271712 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ee38c-ea6e-43bd-8a6c-ab918472c02f None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:27.382460 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ee38c-ea6e-43bd-8a6c-ab918472c02f None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:27.393585 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:27.422070 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb55a15-a6e3-4d7a-858e-2e681e3bc173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:27.422825 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb55a15-a6e3-4d7a-858e-2e681e3bc173 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:27.423105 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb55a15-a6e3-4d7a-858e-2e681e3bc173 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:27.423379 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb55a15-a6e3-4d7a-858e-2e681e3bc173 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:27.424220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb55a15-a6e3-4d7a-858e-2e681e3bc173 None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=da4cd687b35d4dac81871c6d26213346, user.id=c829da0ff81f4f6d99d94afeb018caf8) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:52:27.424533 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb55a15-a6e3-4d7a-858e-2e681e3bc173 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:27.425973 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb55a15-a6e3-4d7a-858e-2e681e3bc173 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:27.431987 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 583/1161] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:52:27 2022] GET /identity/v3/role_assignments?user.id=c829da0ff81f4f6d99d94afeb018caf8&scope.project.id=da4cd687b35d4dac81871c6d26213346&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:29.641782 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40ebcf-5be0-4a66-bff7-6d3db71330fb None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:29.642768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40ebcf-5be0-4a66-bff7-6d3db71330fb None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:29.642768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40ebcf-5be0-4a66-bff7-6d3db71330fb None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:29.644933 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 579/1162] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:29.653343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0c63b-b7be-47a4-8c36-8b8bcfe4447f None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:29.653813 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0c63b-b7be-47a4-8c36-8b8bcfe4447f None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:29.653910 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0c63b-b7be-47a4-8c36-8b8bcfe4447f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:29.672517 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-94c0c63b-b7be-47a4-8c36-8b8bcfe4447f None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:29.673700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-94c0c63b-b7be-47a4-8c36-8b8bcfe4447f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:29.753201 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 584/1163] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:52:29.779333 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40ebcf-5be0-4a66-bff7-6d3db71330fb None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:29.801044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:29.898090 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40ebcf-5be0-4a66-bff7-6d3db71330fb None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:29.933674 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73770e09-b37e-4b3c-ae45-28fbc4e8d6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:29.934308 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73770e09-b37e-4b3c-ae45-28fbc4e8d6e9 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:29.934627 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73770e09-b37e-4b3c-ae45-28fbc4e8d6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:29.934902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73770e09-b37e-4b3c-ae45-28fbc4e8d6e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:30.008833 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73770e09-b37e-4b3c-ae45-28fbc4e8d6e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:30.010311 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73770e09-b37e-4b3c-ae45-28fbc4e8d6e9 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:30.031633 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:30.122080 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 580/1164] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:52:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:52:30.128669 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c0c63b-b7be-47a4-8c36-8b8bcfe4447f None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:30.239645 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c0c63b-b7be-47a4-8c36-8b8bcfe4447f None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:30.241915 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:30.276503 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca770ff7-905a-49ca-9d29-e6173e750e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:30.277476 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca770ff7-905a-49ca-9d29-e6173e750e2c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:30.277756 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca770ff7-905a-49ca-9d29-e6173e750e2c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:30.278044 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca770ff7-905a-49ca-9d29-e6173e750e2c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:30.285326 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca770ff7-905a-49ca-9d29-e6173e750e2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:30.286826 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca770ff7-905a-49ca-9d29-e6173e750e2c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-ca770ff7-905a-49ca-9d29-e6173e750e2c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:52:30.301227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:52:30.302363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:52:30.303475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:52:30.303475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:52:30.303475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:52:30.303475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:52:30.303475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:52:30.303475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:52:30.303475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:52:30.303475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:52:30.303475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:52:30.303475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:52:30.303475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 19:52:30.303475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:52:30.303475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 19:52:30.303475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:52:30.303475 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 585/1165] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:52:30 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:52:30.310876 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73770e09-b37e-4b3c-ae45-28fbc4e8d6e9 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:30.416836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73770e09-b37e-4b3c-ae45-28fbc4e8d6e9 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:30.433852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:30.453228 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe86f7-cb01-4e80-88ae-fc0920694942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:30.453896 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe86f7-cb01-4e80-88ae-fc0920694942 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:30.454179 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe86f7-cb01-4e80-88ae-fc0920694942 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:30.454481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe86f7-cb01-4e80-88ae-fc0920694942 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:30.455224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe86f7-cb01-4e80-88ae-fc0920694942 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:52:30.455535 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe86f7-cb01-4e80-88ae-fc0920694942 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:30.456908 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe86f7-cb01-4e80-88ae-fc0920694942 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:30.464184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 581/1166] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 19:52:30 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:32.688942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc63524-4c3a-4c2c-a79a-e0038bcd1ba0 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:32.689820 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc63524-4c3a-4c2c-a79a-e0038bcd1ba0 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:32.689820 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc63524-4c3a-4c2c-a79a-e0038bcd1ba0 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:32.692097 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 586/1167] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:32.701261 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc145d-2492-42f0-a6a4-fadf99f95588 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:32.701682 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc145d-2492-42f0-a6a4-fadf99f95588 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:32.701838 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc145d-2492-42f0-a6a4-fadf99f95588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:32.724684 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-03fc145d-2492-42f0-a6a4-fadf99f95588 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:32.726621 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-03fc145d-2492-42f0-a6a4-fadf99f95588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:32.816179 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 582/1168] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:52:32.842724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc63524-4c3a-4c2c-a79a-e0038bcd1ba0 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:32.952903 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc63524-4c3a-4c2c-a79a-e0038bcd1ba0 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:32.992104 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fa0c6-7de5-4531-abeb-d3ba7ee1e219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:32.992792 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fa0c6-7de5-4531-abeb-d3ba7ee1e219 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:32.993073 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fa0c6-7de5-4531-abeb-d3ba7ee1e219 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:32.993355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fa0c6-7de5-4531-abeb-d3ba7ee1e219 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:32.994109 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fa0c6-7de5-4531-abeb-d3ba7ee1e219 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:32.995526 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fa0c6-7de5-4531-abeb-d3ba7ee1e219 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:33.017643 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 587/1169] 10.222.0.84 () {60 vars in 1138 bytes} [Thu May 19 19:52:32 2022] POST /identity/v3/domains => generated 267 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:52:35.242060 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b27b8-9481-4412-a587-738cd9e5e8dd None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:35.242957 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b27b8-9481-4412-a587-738cd9e5e8dd None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:35.243136 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b27b8-9481-4412-a587-738cd9e5e8dd None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:35.245254 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 583/1170] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:35.251784 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3cc27-d9ce-458a-afe5-3cd3d2b7bb02 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:35.252527 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3cc27-d9ce-458a-afe5-3cd3d2b7bb02 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:35.252679 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3cc27-d9ce-458a-afe5-3cd3d2b7bb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:35.271839 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-0dc3cc27-d9ce-458a-afe5-3cd3d2b7bb02 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:35.273061 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-0dc3cc27-d9ce-458a-afe5-3cd3d2b7bb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:35.293144 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:35.357003 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 588/1171] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:52:35.386824 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b27b8-9481-4412-a587-738cd9e5e8dd None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:35.509164 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b27b8-9481-4412-a587-738cd9e5e8dd None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:35.519843 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:35.546013 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6abf7e-6dfd-474b-8000-757acccae771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:35.546913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6abf7e-6dfd-474b-8000-757acccae771 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:35.547192 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6abf7e-6dfd-474b-8000-757acccae771 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:35.547464 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6abf7e-6dfd-474b-8000-757acccae771 None admin] PATH_INFO: `/v3/domains/d71916d0dc5146a296ae8c55c377ef4e` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:35.558623 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6abf7e-6dfd-474b-8000-757acccae771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d71916d0dc5146a296ae8c55c377ef4e)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:35.560109 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6abf7e-6dfd-474b-8000-757acccae771 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:35.561767 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 584/1172] 10.222.0.84 () {56 vars in 1183 bytes} [Thu May 19 19:52:35 2022] GET /identity/v3/domains/d71916d0dc5146a296ae8c55c377ef4e => generated 267 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:35.568096 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3cc27-d9ce-458a-afe5-3cd3d2b7bb02 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:35.678616 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3cc27-d9ce-458a-afe5-3cd3d2b7bb02 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:35.714469 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d84c0e-bc66-497a-8aee-a24668d781e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:35.715553 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d84c0e-bc66-497a-8aee-a24668d781e5 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:35.715553 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d84c0e-bc66-497a-8aee-a24668d781e5 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:35.715718 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d84c0e-bc66-497a-8aee-a24668d781e5 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:35.716390 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d84c0e-bc66-497a-8aee-a24668d781e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:35.717843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d84c0e-bc66-497a-8aee-a24668d781e5 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:35.747967 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 589/1173] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:52:35 2022] POST /identity/v3/projects => generated 361 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:52:37.980649 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568b288-6a2a-43b4-bb2f-570453050e94 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:37.981183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568b288-6a2a-43b4-bb2f-570453050e94 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:37.981183 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568b288-6a2a-43b4-bb2f-570453050e94 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:37.982614 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 585/1174] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:37.988886 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a682-fa47-4b70-a350-3734eaa93c92 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:37.989315 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a682-fa47-4b70-a350-3734eaa93c92 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:37.989445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a682-fa47-4b70-a350-3734eaa93c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:38.001584 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:38.011207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-4e50a682-fa47-4b70-a350-3734eaa93c92 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:38.013360 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-4e50a682-fa47-4b70-a350-3734eaa93c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:38.090133 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 590/1175] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:52:38.117259 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568b288-6a2a-43b4-bb2f-570453050e94 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:38.235971 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568b288-6a2a-43b4-bb2f-570453050e94 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:38.272741 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cd402-4164-4adf-b123-f934ba17b59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:38.273620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cd402-4164-4adf-b123-f934ba17b59d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:38.273905 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cd402-4164-4adf-b123-f934ba17b59d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:38.274180 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cd402-4164-4adf-b123-f934ba17b59d None admin] PATH_INFO: `/v3/domains/d71916d0dc5146a296ae8c55c377ef4e` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:38.275522 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cd402-4164-4adf-b123-f934ba17b59d None admin] RBAC: Authorizing `identity:get_domain(domain_id=d71916d0dc5146a296ae8c55c377ef4e)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:38.277386 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cd402-4164-4adf-b123-f934ba17b59d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:38.279084 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 586/1176] 10.222.0.84 () {56 vars in 1183 bytes} [Thu May 19 19:52:38 2022] GET /identity/v3/domains/d71916d0dc5146a296ae8c55c377ef4e => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:38.285902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a682-fa47-4b70-a350-3734eaa93c92 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:38.401580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a682-fa47-4b70-a350-3734eaa93c92 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:38.443071 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31dddb-d534-4348-b9df-af25274ba069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:38.443746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31dddb-d534-4348-b9df-af25274ba069 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:38.444034 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31dddb-d534-4348-b9df-af25274ba069 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:38.444312 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31dddb-d534-4348-b9df-af25274ba069 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:38.445030 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31dddb-d534-4348-b9df-af25274ba069 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:38.446502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31dddb-d534-4348-b9df-af25274ba069 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:38.454502 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:38.470248 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 591/1177] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:52:38 2022] POST /identity/v3/users => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:52:40.733700 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012abf5-17d9-422c-a7d8-07f2a7b57d5e None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:40.734184 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012abf5-17d9-422c-a7d8-07f2a7b57d5e None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:40.734307 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012abf5-17d9-422c-a7d8-07f2a7b57d5e None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:40.735932 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 587/1178] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:40.741082 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae41b04-3fa2-4d30-b727-879546262835 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:40.741497 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae41b04-3fa2-4d30-b727-879546262835 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:40.741645 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae41b04-3fa2-4d30-b727-879546262835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:40.761546 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-bae41b04-3fa2-4d30-b727-879546262835 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:40.762793 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-bae41b04-3fa2-4d30-b727-879546262835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:40.836677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 592/1179] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:52:40.875641 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012abf5-17d9-422c-a7d8-07f2a7b57d5e None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:40.925028 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:40.991577 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012abf5-17d9-422c-a7d8-07f2a7b57d5e None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:41.030061 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb4312-ab55-43dd-b5c8-cb0b17b31ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:41.030886 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb4312-ab55-43dd-b5c8-cb0b17b31ec4 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:41.031169 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb4312-ab55-43dd-b5c8-cb0b17b31ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:41.031444 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb4312-ab55-43dd-b5c8-cb0b17b31ec4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:41.044295 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb4312-ab55-43dd-b5c8-cb0b17b31ec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-27fb4312-ab55-43dd-b5c8-cb0b17b31ec4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:52:41.052278 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:52:41.053318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:52:41.054296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:52:41.054296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:52:41.054296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:52:41.054296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:52:41.054296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:52:41.054296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:52:41.054296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:52:41.054296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:52:41.054296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:52:41.054296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:52:41.054296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:52:41.054296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:52:41.054296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 588/1180] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:52:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:52:41.060691 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae41b04-3fa2-4d30-b727-879546262835 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:41.175056 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae41b04-3fa2-4d30-b727-879546262835 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:41.198510 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:41.219089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d22868-8e24-4241-beb0-bae5c52c96ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:41.220050 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d22868-8e24-4241-beb0-bae5c52c96ec None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:41.220439 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d22868-8e24-4241-beb0-bae5c52c96ec None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:41.221111 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d22868-8e24-4241-beb0-bae5c52c96ec None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:41.222312 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d22868-8e24-4241-beb0-bae5c52c96ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:52:41.222725 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d22868-8e24-4241-beb0-bae5c52c96ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:41.224863 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d22868-8e24-4241-beb0-bae5c52c96ec None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:41.235293 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 593/1181] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:52:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:41.241423 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb4312-ab55-43dd-b5c8-cb0b17b31ec4 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:41.342405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:41.346873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb4312-ab55-43dd-b5c8-cb0b17b31ec4 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:41.382241 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fcacb-dab0-4608-9e26-8815b06df5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:41.383158 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fcacb-dab0-4608-9e26-8815b06df5d5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:41.383442 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fcacb-dab0-4608-9e26-8815b06df5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:41.383720 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fcacb-dab0-4608-9e26-8815b06df5d5 None admin] PATH_INFO: `/v3/users/3f2e699a04e74349a35433dc898c1c4e` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:41.397148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3fcacb-dab0-4608-9e26-8815b06df5d5 None admin] RBAC: Authorizing `identity:get_user(user_id=3f2e699a04e74349a35433dc898c1c4e)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:41.398648 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3fcacb-dab0-4608-9e26-8815b06df5d5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:41.400309 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 589/1182] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:52:41 2022] GET /identity/v3/users/3f2e699a04e74349a35433dc898c1c4e => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:41.407225 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d22868-8e24-4241-beb0-bae5c52c96ec None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:41.528613 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d22868-8e24-4241-beb0-bae5c52c96ec None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:41.565280 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35583e7-2070-4211-9561-16935e5ebe4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:41.565937 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35583e7-2070-4211-9561-16935e5ebe4d None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:41.566199 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35583e7-2070-4211-9561-16935e5ebe4d None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:41.566586 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35583e7-2070-4211-9561-16935e5ebe4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:41.622971 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:41.645282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35583e7-2070-4211-9561-16935e5ebe4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:41.646784 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35583e7-2070-4211-9561-16935e5ebe4d None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:41.749457 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 594/1183] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:52:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:52:41.755583 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fcacb-dab0-4608-9e26-8815b06df5d5 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:41.860157 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fcacb-dab0-4608-9e26-8815b06df5d5 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:41.896568 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6e022-32df-4922-887a-6e107497edf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:41.897356 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6e022-32df-4922-887a-6e107497edf5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:41.897635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6e022-32df-4922-887a-6e107497edf5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:41.897908 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6e022-32df-4922-887a-6e107497edf5 None admin] PATH_INFO: `/v3/projects/df327a55d1624de48bfb026dac7b9978` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:41.899251 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6e022-32df-4922-887a-6e107497edf5 None admin] RBAC: Authorizing `identity:get_project(project_id=df327a55d1624de48bfb026dac7b9978)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:41.900692 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6e022-32df-4922-887a-6e107497edf5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:41.902454 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 590/1184] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:52:41 2022] GET /identity/v3/projects/df327a55d1624de48bfb026dac7b9978 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:41.908644 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35583e7-2070-4211-9561-16935e5ebe4d None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:41.997902 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:42.016580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35583e7-2070-4211-9561-16935e5ebe4d None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:42.053460 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c4aec-4328-44e4-97e2-1c4e29dd50a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:42.054105 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c4aec-4328-44e4-97e2-1c4e29dd50a2 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:42.054416 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c4aec-4328-44e4-97e2-1c4e29dd50a2 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:42.054694 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c4aec-4328-44e4-97e2-1c4e29dd50a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:42.055531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c4aec-4328-44e4-97e2-1c4e29dd50a2 None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=df327a55d1624de48bfb026dac7b9978, user.id=3f2e699a04e74349a35433dc898c1c4e) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:52:42.055844 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c4aec-4328-44e4-97e2-1c4e29dd50a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:42.057281 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c4aec-4328-44e4-97e2-1c4e29dd50a2 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:42.063292 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 595/1185] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:52:41 2022] GET /identity/v3/role_assignments?user.id=3f2e699a04e74349a35433dc898c1c4e&scope.project.id=df327a55d1624de48bfb026dac7b9978&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:44.236016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72255a4-a7b1-4334-ad23-66233d95651c None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:44.236603 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72255a4-a7b1-4334-ad23-66233d95651c None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:44.236603 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72255a4-a7b1-4334-ad23-66233d95651c None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:44.237945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 591/1186] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:44.244469 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4845d7-df9b-48d9-bc31-16f7a77ae9ef None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:44.245193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4845d7-df9b-48d9-bc31-16f7a77ae9ef None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:44.245397 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4845d7-df9b-48d9-bc31-16f7a77ae9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:44.270298 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-de4845d7-df9b-48d9-bc31-16f7a77ae9ef None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:44.271522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-de4845d7-df9b-48d9-bc31-16f7a77ae9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:44.354948 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 596/1187] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:52:44.378289 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72255a4-a7b1-4334-ad23-66233d95651c None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:44.407294 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:44.493899 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72255a4-a7b1-4334-ad23-66233d95651c None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:44.529593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932771a-058b-492d-8afd-71e8edf18ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:44.530415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932771a-058b-492d-8afd-71e8edf18ae8 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:44.530696 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932771a-058b-492d-8afd-71e8edf18ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:44.530974 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932771a-058b-492d-8afd-71e8edf18ae8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:44.541825 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932771a-058b-492d-8afd-71e8edf18ae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-4932771a-058b-492d-8afd-71e8edf18ae8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:52:44.551852 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:52:44.552912 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:52:44.553805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:52:44.553805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:52:44.553805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:52:44.553805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:52:44.553805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:52:44.553805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:52:44.553805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:52:44.553805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:52:44.553805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:52:44.553805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:52:44.553805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:52:44.553805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:52:44.553805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 592/1188] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:52:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:52:44.559670 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4845d7-df9b-48d9-bc31-16f7a77ae9ef None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:44.646184 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:44.671632 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4845d7-df9b-48d9-bc31-16f7a77ae9ef None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:44.705748 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55986bb0-ff8d-4ee4-abf2-a71a7475ba5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:44.706462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55986bb0-ff8d-4ee4-abf2-a71a7475ba5b None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:44.706746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55986bb0-ff8d-4ee4-abf2-a71a7475ba5b None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:44.707020 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55986bb0-ff8d-4ee4-abf2-a71a7475ba5b None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:44.707771 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55986bb0-ff8d-4ee4-abf2-a71a7475ba5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:52:44.708077 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55986bb0-ff8d-4ee4-abf2-a71a7475ba5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:44.709464 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55986bb0-ff8d-4ee4-abf2-a71a7475ba5b None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:44.718370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 597/1189] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:52:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:44.725349 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932771a-058b-492d-8afd-71e8edf18ae8 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:44.815403 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:44.831501 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932771a-058b-492d-8afd-71e8edf18ae8 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:44.865355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b222076-7943-4eba-a7e5-3c2a0cd38848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:44.866224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b222076-7943-4eba-a7e5-3c2a0cd38848 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:44.866549 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b222076-7943-4eba-a7e5-3c2a0cd38848 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:44.866813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b222076-7943-4eba-a7e5-3c2a0cd38848 None admin] PATH_INFO: `/v3/users/3f2e699a04e74349a35433dc898c1c4e` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:44.868150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b222076-7943-4eba-a7e5-3c2a0cd38848 None admin] RBAC: Authorizing `identity:get_user(user_id=3f2e699a04e74349a35433dc898c1c4e)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:44.869593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b222076-7943-4eba-a7e5-3c2a0cd38848 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:44.871344 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 593/1190] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:52:44 2022] GET /identity/v3/users/3f2e699a04e74349a35433dc898c1c4e => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:44.877512 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55986bb0-ff8d-4ee4-abf2-a71a7475ba5b None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:44.998098 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55986bb0-ff8d-4ee4-abf2-a71a7475ba5b None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:45.034715 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038f5db-eb4b-4432-94c0-dd5d66600a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:45.035577 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038f5db-eb4b-4432-94c0-dd5d66600a5c None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:45.035856 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038f5db-eb4b-4432-94c0-dd5d66600a5c None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:45.036137 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038f5db-eb4b-4432-94c0-dd5d66600a5c None admin] PATH_INFO: `/v3/projects/df327a55d1624de48bfb026dac7b9978` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:45.037460 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038f5db-eb4b-4432-94c0-dd5d66600a5c None admin] RBAC: Authorizing `identity:get_project(project_id=df327a55d1624de48bfb026dac7b9978)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:45.038955 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038f5db-eb4b-4432-94c0-dd5d66600a5c None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:45.040669 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 598/1191] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:52:44 2022] GET /identity/v3/projects/df327a55d1624de48bfb026dac7b9978 => generated 361 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:45.050137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b222076-7943-4eba-a7e5-3c2a0cd38848 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:45.155526 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b222076-7943-4eba-a7e5-3c2a0cd38848 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:45.190510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2473c80-73f3-4016-9ba0-9305784166fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:45.191393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2473c80-73f3-4016-9ba0-9305784166fc None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:45.191672 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2473c80-73f3-4016-9ba0-9305784166fc None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:45.191943 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2473c80-73f3-4016-9ba0-9305784166fc None admin] PATH_INFO: `/v3/projects/df327a55d1624de48bfb026dac7b9978/users/3f2e699a04e74349a35433dc898c1c4e/roles/2cd6d3dd613c4824a26cacf53fb6c07f` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:45.194155 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2473c80-73f3-4016-9ba0-9305784166fc None admin] RBAC: Authorizing `identity:create_grant(project_id=df327a55d1624de48bfb026dac7b9978, user_id=3f2e699a04e74349a35433dc898c1c4e, role_id=2cd6d3dd613c4824a26cacf53fb6c07f)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:45.195631 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2473c80-73f3-4016-9ba0-9305784166fc None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:45.211741 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 594/1192] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:52:45 2022] PUT /identity/v3/projects/df327a55d1624de48bfb026dac7b9978/users/3f2e699a04e74349a35433dc898c1c4e/roles/2cd6d3dd613c4824a26cacf53fb6c07f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:52:47.413558 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c478fce-6fbc-4073-9af0-6e4e853a0117 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:47.414151 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c478fce-6fbc-4073-9af0-6e4e853a0117 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:47.414151 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c478fce-6fbc-4073-9af0-6e4e853a0117 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:47.415534 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 599/1193] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:52:47.422051 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c9c69-c98c-489d-b29a-56dbad86856d None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:47.422535 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c9c69-c98c-489d-b29a-56dbad86856d None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:47.422773 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c9c69-c98c-489d-b29a-56dbad86856d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:47.445009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-8f2c9c69-c98c-489d-b29a-56dbad86856d None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:52:47.446205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-8f2c9c69-c98c-489d-b29a-56dbad86856d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:52:47.454189 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:47.522830 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 595/1194] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:52:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:52:47.548088 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c478fce-6fbc-4073-9af0-6e4e853a0117 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:47.563540 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:47.660910 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c478fce-6fbc-4073-9af0-6e4e853a0117 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:47.695457 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a5a3d-d16d-45ba-9739-3b8843ff2965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:47.696174 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a5a3d-d16d-45ba-9739-3b8843ff2965 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:47.696453 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a5a3d-d16d-45ba-9739-3b8843ff2965 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:47.696722 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a5a3d-d16d-45ba-9739-3b8843ff2965 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:47.709281 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a5a3d-d16d-45ba-9739-3b8843ff2965 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-179a5a3d-d16d-45ba-9739-3b8843ff2965 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:52:47.717175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:52:47.718211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:52:47.719085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:52:47.719085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:52:47.719085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:52:47.719085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:52:47.719085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:52:47.719085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:52:47.719085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:52:47.719085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:52:47.719085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:52:47.719085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:52:47.719085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:52:47.719085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:52:47.719085 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 600/1195] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:52:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:52:47.726830 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c9c69-c98c-489d-b29a-56dbad86856d None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:47.832709 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c9c69-c98c-489d-b29a-56dbad86856d None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:47.873002 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6acd3a-3ae4-42fc-8e70-5d679b18e89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:47.873667 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6acd3a-3ae4-42fc-8e70-5d679b18e89f None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:47.873945 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6acd3a-3ae4-42fc-8e70-5d679b18e89f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:47.874216 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6acd3a-3ae4-42fc-8e70-5d679b18e89f None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:47.875007 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6acd3a-3ae4-42fc-8e70-5d679b18e89f None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:52:47.875313 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6acd3a-3ae4-42fc-8e70-5d679b18e89f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:47.876690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6acd3a-3ae4-42fc-8e70-5d679b18e89f None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:47.885013 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 596/1196] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:52:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:47.893229 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a5a3d-d16d-45ba-9739-3b8843ff2965 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:47.962170 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:47.999877 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a5a3d-d16d-45ba-9739-3b8843ff2965 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:48.036072 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c3750-e682-4036-bb78-12daed61d853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:48.036934 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c3750-e682-4036-bb78-12daed61d853 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:48.037212 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c3750-e682-4036-bb78-12daed61d853 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:48.037484 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c3750-e682-4036-bb78-12daed61d853 None admin] PATH_INFO: `/v3/users/3f2e699a04e74349a35433dc898c1c4e` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:48.038705 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c3750-e682-4036-bb78-12daed61d853 None admin] RBAC: Authorizing `identity:get_user(user_id=3f2e699a04e74349a35433dc898c1c4e)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:48.040131 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c3750-e682-4036-bb78-12daed61d853 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:48.041774 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 601/1197] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 19:52:47 2022] GET /identity/v3/users/3f2e699a04e74349a35433dc898c1c4e => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:48.048074 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6acd3a-3ae4-42fc-8e70-5d679b18e89f None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:48.052471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:48.166421 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6acd3a-3ae4-42fc-8e70-5d679b18e89f None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:48.206440 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa2521-42bb-41d3-b0e5-dbfc3809859b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:48.207092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa2521-42bb-41d3-b0e5-dbfc3809859b None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:48.207372 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa2521-42bb-41d3-b0e5-dbfc3809859b None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:48.207643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa2521-42bb-41d3-b0e5-dbfc3809859b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:48.278865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa2521-42bb-41d3-b0e5-dbfc3809859b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:48.280232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa2521-42bb-41d3-b0e5-dbfc3809859b None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:48.282907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:48.402247 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 597/1198] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:52:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:52:48.411806 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c3750-e682-4036-bb78-12daed61d853 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:48.524615 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c3750-e682-4036-bb78-12daed61d853 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:48.561462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a40e0-a7b3-4597-9750-c0af3105abf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:48.562647 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a40e0-a7b3-4597-9750-c0af3105abf8 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:48.562932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a40e0-a7b3-4597-9750-c0af3105abf8 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:48.563207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a40e0-a7b3-4597-9750-c0af3105abf8 None admin] PATH_INFO: `/v3/projects/df327a55d1624de48bfb026dac7b9978` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:48.564494 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a40e0-a7b3-4597-9750-c0af3105abf8 None admin] RBAC: Authorizing `identity:get_project(project_id=df327a55d1624de48bfb026dac7b9978)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:48.565951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a40e0-a7b3-4597-9750-c0af3105abf8 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:48.567726 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 602/1199] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 19:52:48 2022] GET /identity/v3/projects/df327a55d1624de48bfb026dac7b9978 => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:48.574311 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa2521-42bb-41d3-b0e5-dbfc3809859b None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:52:48.679654 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa2521-42bb-41d3-b0e5-dbfc3809859b None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:52:48.682004 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:48.715678 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaab517-a7d8-4e4c-a3a6-058b6db16a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:52:48.716385 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaab517-a7d8-4e4c-a3a6-058b6db16a19 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:52:48.716666 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaab517-a7d8-4e4c-a3a6-058b6db16a19 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:52:48.716923 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaab517-a7d8-4e4c-a3a6-058b6db16a19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:52:48.717768 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaab517-a7d8-4e4c-a3a6-058b6db16a19 None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=df327a55d1624de48bfb026dac7b9978, user.id=3f2e699a04e74349a35433dc898c1c4e) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:52:48.718077 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaab517-a7d8-4e4c-a3a6-058b6db16a19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:52:48.719576 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaab517-a7d8-4e4c-a3a6-058b6db16a19 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:52:48.726206 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 598/1200] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:52:48 2022] GET /identity/v3/role_assignments?user.id=3f2e699a04e74349a35433dc898c1c4e&scope.project.id=df327a55d1624de48bfb026dac7b9978&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:52:58.555958 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=98 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:52:58.723347 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=16 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:47.520401 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4386036-e2cb-4522-8f11-76359b9a9107 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:47.521005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4386036-e2cb-4522-8f11-76359b9a9107 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:47.521005 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4386036-e2cb-4522-8f11-76359b9a9107 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:47.523058 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 603/1201] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:53:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:53:47.529276 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15ec87-9e4a-429f-a029-8ae3d2ccb44c None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:47.529844 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15ec87-9e4a-429f-a029-8ae3d2ccb44c None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:47.529844 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15ec87-9e4a-429f-a029-8ae3d2ccb44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:47.548693 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-1d15ec87-9e4a-429f-a029-8ae3d2ccb44c None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:53:47.549919 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-1d15ec87-9e4a-429f-a029-8ae3d2ccb44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:53:47.626169 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 599/1202] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:53:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:53:47.651423 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4386036-e2cb-4522-8f11-76359b9a9107 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:47.765576 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4386036-e2cb-4522-8f11-76359b9a9107 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:47.801710 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17b4ef-d80a-4a6a-9891-bdf0f4207c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:47.802560 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17b4ef-d80a-4a6a-9891-bdf0f4207c57 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:47.802837 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17b4ef-d80a-4a6a-9891-bdf0f4207c57 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:47.803094 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17b4ef-d80a-4a6a-9891-bdf0f4207c57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:47.813757 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17b4ef-d80a-4a6a-9891-bdf0f4207c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:47.815231 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17b4ef-d80a-4a6a-9891-bdf0f4207c57 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:47.833083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-af17b4ef-d80a-4a6a-9891-bdf0f4207c57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:53:47.833083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:53:47.833083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:53:47.833083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:53:47.833083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:53:47.833083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:53:47.833083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:53:47.833083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:53:47.833083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:53:47.833083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:53:47.833083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:53:47.833083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:53:47.833083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:53:47.833083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:53:47.833083 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:53:47.834126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:53:47.835112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:53:47.835112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:53:47.835112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:53:47.835112 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:53:47.836066 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 604/1203] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:53:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:53:47.841147 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15ec87-9e4a-429f-a029-8ae3d2ccb44c None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:47.951834 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15ec87-9e4a-429f-a029-8ae3d2ccb44c None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:47.983407 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:47.990606 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc308090-7a7c-49cb-b02e-e23daa3d0e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:47.991285 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc308090-7a7c-49cb-b02e-e23daa3d0e5c None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:47.991565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc308090-7a7c-49cb-b02e-e23daa3d0e5c None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:47.991843 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc308090-7a7c-49cb-b02e-e23daa3d0e5c None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:47.992563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc308090-7a7c-49cb-b02e-e23daa3d0e5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:53:47.992872 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc308090-7a7c-49cb-b02e-e23daa3d0e5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:47.994448 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc308090-7a7c-49cb-b02e-e23daa3d0e5c None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:48.008058 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 600/1204] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:53:48.014382 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17b4ef-d80a-4a6a-9891-bdf0f4207c57 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:48.071563 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:48.122692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17b4ef-d80a-4a6a-9891-bdf0f4207c57 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:48.159870 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf81d61-0c60-4b8c-a4b8-d9b5b16a56b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:48.160567 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf81d61-0c60-4b8c-a4b8-d9b5b16a56b7 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:48.160840 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf81d61-0c60-4b8c-a4b8-d9b5b16a56b7 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:48.161115 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf81d61-0c60-4b8c-a4b8-d9b5b16a56b7 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:48.161830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf81d61-0c60-4b8c-a4b8-d9b5b16a56b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:48.163371 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf81d61-0c60-4b8c-a4b8-d9b5b16a56b7 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:48.186340 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 605/1205] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 19:53:48 2022] POST /identity/v3/users => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:53:50.589981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534d3d0-1dc2-46d6-8bed-87c98fcf76af None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:50.590610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534d3d0-1dc2-46d6-8bed-87c98fcf76af None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:50.590610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534d3d0-1dc2-46d6-8bed-87c98fcf76af None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:50.592205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 601/1206] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:53:50.597927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51e384-6b80-414b-897e-203dffb2edfd None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:50.598411 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51e384-6b80-414b-897e-203dffb2edfd None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:50.598497 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51e384-6b80-414b-897e-203dffb2edfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:50.619358 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-be51e384-6b80-414b-897e-203dffb2edfd None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:53:50.620568 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-be51e384-6b80-414b-897e-203dffb2edfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:53:50.694297 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 606/1207] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:53:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:53:50.719855 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534d3d0-1dc2-46d6-8bed-87c98fcf76af None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:50.840311 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534d3d0-1dc2-46d6-8bed-87c98fcf76af None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:50.879434 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c802c72-d549-4d07-8b07-60bc0816f2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:50.880351 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c802c72-d549-4d07-8b07-60bc0816f2b2 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:50.880672 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c802c72-d549-4d07-8b07-60bc0816f2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:50.880999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c802c72-d549-4d07-8b07-60bc0816f2b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:50.893924 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c802c72-d549-4d07-8b07-60bc0816f2b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-9c802c72-d549-4d07-8b07-60bc0816f2b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:53:50.901775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:53:50.902736 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:53:50.903804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:53:50.903804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:53:50.903804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:53:50.903804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:53:50.903804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:53:50.903804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:53:50.903804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:53:50.903804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:53:50.903804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:53:50.903804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:53:50.903804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:53:50.903804 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:53:50.907934 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 602/1208] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:53:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:53:50.909573 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51e384-6b80-414b-897e-203dffb2edfd None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:50.967938 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:51.044361 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51e384-6b80-414b-897e-203dffb2edfd None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:51.084107 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7131ac0-cd80-4a3a-87d0-dbb6d0649334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:51.084807 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7131ac0-cd80-4a3a-87d0-dbb6d0649334 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:51.085093 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7131ac0-cd80-4a3a-87d0-dbb6d0649334 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:51.085372 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7131ac0-cd80-4a3a-87d0-dbb6d0649334 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:51.086131 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7131ac0-cd80-4a3a-87d0-dbb6d0649334 None admin] RBAC: Adding query filter params (name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:53:51.086483 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7131ac0-cd80-4a3a-87d0-dbb6d0649334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:51.088055 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7131ac0-cd80-4a3a-87d0-dbb6d0649334 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:51.097891 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 607/1209] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:53:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:53:51.103770 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c802c72-d549-4d07-8b07-60bc0816f2b2 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:51.162479 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:51.219983 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c802c72-d549-4d07-8b07-60bc0816f2b2 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:51.257972 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108d8d1-54ff-492e-a693-5b053b484644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:51.258948 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108d8d1-54ff-492e-a693-5b053b484644 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:51.259229 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108d8d1-54ff-492e-a693-5b053b484644 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:51.259525 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108d8d1-54ff-492e-a693-5b053b484644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:51.271467 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108d8d1-54ff-492e-a693-5b053b484644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:51.273196 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108d8d1-54ff-492e-a693-5b053b484644 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:51.294205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-e108d8d1-54ff-492e-a693-5b053b484644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:53:51.294205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:53:51.294205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:53:51.294205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:53:51.294205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:53:51.294205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:53:51.294205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:53:51.294205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:53:51.294205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:53:51.294205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:53:51.294205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:53:51.294205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:53:51.294205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:53:51.294205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:53:51.294205 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:53:51.295395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:53:51.296445 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:53:51.296445 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:53:51.296445 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:53:51.296445 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:53:51.297314 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 603/1210] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:53:51.303385 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7131ac0-cd80-4a3a-87d0-dbb6d0649334 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:51.413762 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:51.417314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7131ac0-cd80-4a3a-87d0-dbb6d0649334 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:51.456438 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a54df-2d10-49b8-b2b7-9dbba1982c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:51.457108 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a54df-2d10-49b8-b2b7-9dbba1982c1d None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:51.460430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a54df-2d10-49b8-b2b7-9dbba1982c1d None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:51.460430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a54df-2d10-49b8-b2b7-9dbba1982c1d None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:51.460430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a54df-2d10-49b8-b2b7-9dbba1982c1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:53:51.460430 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a54df-2d10-49b8-b2b7-9dbba1982c1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:51.460761 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a54df-2d10-49b8-b2b7-9dbba1982c1d None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:51.474369 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 608/1211] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:53:51.480010 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108d8d1-54ff-492e-a693-5b053b484644 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:51.592662 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108d8d1-54ff-492e-a693-5b053b484644 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:51.594440 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:51.633100 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92b1ae-f6f6-4434-aeb1-17b0ef582577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:51.634056 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92b1ae-f6f6-4434-aeb1-17b0ef582577 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:51.634304 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92b1ae-f6f6-4434-aeb1-17b0ef582577 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:51.634623 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92b1ae-f6f6-4434-aeb1-17b0ef582577 None admin] PATH_INFO: `/v3/users/placement` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:51.642819 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92b1ae-f6f6-4434-aeb1-17b0ef582577 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:51.644358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92b1ae-f6f6-4434-aeb1-17b0ef582577 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-3e92b1ae-f6f6-4434-aeb1-17b0ef582577 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:53:51.659505 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:53:51.660564 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:53:51.661965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:53:51.661965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:53:51.661965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:53:51.661965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:53:51.661965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:53:51.661965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:53:51.661965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:53:51.661965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:53:51.661965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:53:51.661965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:53:51.661965 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 604/1212] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:53:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:53:51.666975 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a54df-2d10-49b8-b2b7-9dbba1982c1d None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:51.790521 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a54df-2d10-49b8-b2b7-9dbba1982c1d None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:51.827397 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacaabf-1269-4e6c-95d7-29ad07e10f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:51.828353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacaabf-1269-4e6c-95d7-29ad07e10f15 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:51.828642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacaabf-1269-4e6c-95d7-29ad07e10f15 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:51.828925 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacaabf-1269-4e6c-95d7-29ad07e10f15 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:51.829779 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacaabf-1269-4e6c-95d7-29ad07e10f15 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:53:51.830091 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacaabf-1269-4e6c-95d7-29ad07e10f15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:51.831571 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacaabf-1269-4e6c-95d7-29ad07e10f15 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:51.851805 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 609/1213] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 19:53:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:53:51.858246 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92b1ae-f6f6-4434-aeb1-17b0ef582577 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:51.972614 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92b1ae-f6f6-4434-aeb1-17b0ef582577 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:52.010020 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992c295-248e-45e9-9f77-230d7b16fa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:52.010779 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992c295-248e-45e9-9f77-230d7b16fa45 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:52.011110 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992c295-248e-45e9-9f77-230d7b16fa45 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:52.011429 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992c295-248e-45e9-9f77-230d7b16fa45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:52.046117 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:52.090654 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992c295-248e-45e9-9f77-230d7b16fa45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:52.092217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992c295-248e-45e9-9f77-230d7b16fa45 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:52.218125 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 605/1214] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:53:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:53:52.224741 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacaabf-1269-4e6c-95d7-29ad07e10f15 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:52.250546 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:52.348698 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacaabf-1269-4e6c-95d7-29ad07e10f15 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:52.388929 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a42718-0b63-4a80-90fc-a6a0a0e08f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:52.389968 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a42718-0b63-4a80-90fc-a6a0a0e08f05 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:52.390335 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a42718-0b63-4a80-90fc-a6a0a0e08f05 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:52.390675 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a42718-0b63-4a80-90fc-a6a0a0e08f05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:52.402659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a42718-0b63-4a80-90fc-a6a0a0e08f05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:52.404126 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a42718-0b63-4a80-90fc-a6a0a0e08f05 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:52.421914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-80a42718-0b63-4a80-90fc-a6a0a0e08f05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:53:52.421914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:53:52.421914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:53:52.421914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:53:52.421914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:53:52.421914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:53:52.421914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:53:52.421914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:53:52.421914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:53:52.421914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:53:52.421914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:53:52.421914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:53:52.421914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:53:52.421914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:53:52.421914 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:53:52.422984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:53:52.431500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:53:52.431500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:53:52.431500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:53:52.431500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:53:52.431500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 610/1215] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:53:52.431839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992c295-248e-45e9-9f77-230d7b16fa45 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:52.501642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:52.549154 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992c295-248e-45e9-9f77-230d7b16fa45 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:52.592281 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b38a2-c461-4c11-84eb-a7da73faf54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:52.592914 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b38a2-c461-4c11-84eb-a7da73faf54f None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:52.593502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b38a2-c461-4c11-84eb-a7da73faf54f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:52.593502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b38a2-c461-4c11-84eb-a7da73faf54f None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:52.594200 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b38a2-c461-4c11-84eb-a7da73faf54f None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:53:52.594878 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b38a2-c461-4c11-84eb-a7da73faf54f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:52.596260 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b38a2-c461-4c11-84eb-a7da73faf54f None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:52.610676 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 606/1216] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:53:52.616688 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a42718-0b63-4a80-90fc-a6a0a0e08f05 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:52.712331 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:52.742088 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a42718-0b63-4a80-90fc-a6a0a0e08f05 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:52.784723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9810a-2279-4ae0-8cc2-b0a2146ecf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:52.785760 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9810a-2279-4ae0-8cc2-b0a2146ecf64 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:52.786062 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9810a-2279-4ae0-8cc2-b0a2146ecf64 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:52.786423 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9810a-2279-4ae0-8cc2-b0a2146ecf64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:52.794874 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9810a-2279-4ae0-8cc2-b0a2146ecf64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:52.796343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9810a-2279-4ae0-8cc2-b0a2146ecf64 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:52.814949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-40d9810a-2279-4ae0-8cc2-b0a2146ecf64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:53:52.814949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:53:52.814949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:53:52.814949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:53:52.814949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:53:52.814949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:53:52.814949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:53:52.814949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:53:52.814949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:53:52.814949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:53:52.814949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:53:52.814949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:53:52.814949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:53:52.814949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:53:52.814949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:53:52.815871 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:53:52.816704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:53:52.816704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:53:52.816704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:53:52.816704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:53:52.816704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:53:52.816704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:53:52.816704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:53:52.816704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:53:52.816704 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 611/1217] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:53:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:53:52.823420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b38a2-c461-4c11-84eb-a7da73faf54f None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:52.950726 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b38a2-c461-4c11-84eb-a7da73faf54f None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:52.978873 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:52.992101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ef8a6-5adb-4e09-bd00-05a1adae2863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:52.992882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ef8a6-5adb-4e09-bd00-05a1adae2863 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:52.993199 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ef8a6-5adb-4e09-bd00-05a1adae2863 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:52.993484 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ef8a6-5adb-4e09-bd00-05a1adae2863 None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:52.994503 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ef8a6-5adb-4e09-bd00-05a1adae2863 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:53:52.994877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ef8a6-5adb-4e09-bd00-05a1adae2863 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:52.996420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ef8a6-5adb-4e09-bd00-05a1adae2863 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:53.009854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 607/1218] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:53:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:53:53.016044 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9810a-2279-4ae0-8cc2-b0a2146ecf64 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:53.137861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9810a-2279-4ae0-8cc2-b0a2146ecf64 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:53.170148 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:53.176543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61cc13-bfe1-4072-9642-6020ba25b4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:53.177276 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61cc13-bfe1-4072-9642-6020ba25b4a3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:53.177535 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61cc13-bfe1-4072-9642-6020ba25b4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:53.177812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61cc13-bfe1-4072-9642-6020ba25b4a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:53.178731 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61cc13-bfe1-4072-9642-6020ba25b4a3 None admin] RBAC: Adding query filter params (role.id=3387bd4782b047ae8d880e386b0351f0, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=14b4b91113ed40f0bd0d2bcc6818138c) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:53:53.179483 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61cc13-bfe1-4072-9642-6020ba25b4a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:53.181545 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61cc13-bfe1-4072-9642-6020ba25b4a3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:53.198562 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 612/1219] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:53:53 2022] GET /identity/v3/role_assignments?user.id=14b4b91113ed40f0bd0d2bcc6818138c&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=3387bd4782b047ae8d880e386b0351f0 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:53:55.575346 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1b11a-bb8f-49cd-a7bc-d78abef07fa4 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:55.575999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1b11a-bb8f-49cd-a7bc-d78abef07fa4 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:55.575999 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1b11a-bb8f-49cd-a7bc-d78abef07fa4 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:55.577433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 608/1220] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:53:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:53:55.583188 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1a174-9bcf-4cd4-bdf1-f7a4c22ae94f None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:55.583649 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1a174-9bcf-4cd4-bdf1-f7a4c22ae94f None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:55.583746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1a174-9bcf-4cd4-bdf1-f7a4c22ae94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:55.603428 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-b4f1a174-9bcf-4cd4-bdf1-f7a4c22ae94f None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:53:55.604726 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-b4f1a174-9bcf-4cd4-bdf1-f7a4c22ae94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:53:55.677065 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 613/1221] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:53:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:53:55.700433 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1b11a-bb8f-49cd-a7bc-d78abef07fa4 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:55.812676 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1b11a-bb8f-49cd-a7bc-d78abef07fa4 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:55.852691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eccf60-7d9b-42de-b9ff-b867de975399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:55.853558 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eccf60-7d9b-42de-b9ff-b867de975399 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:55.853837 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eccf60-7d9b-42de-b9ff-b867de975399 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:55.854107 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eccf60-7d9b-42de-b9ff-b867de975399 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:55.864580 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eccf60-7d9b-42de-b9ff-b867de975399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-04eccf60-7d9b-42de-b9ff-b867de975399 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:53:55.872800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:53:55.873910 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:53:55.880035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:53:55.880035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:53:55.880035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:53:55.880035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:53:55.880035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:53:55.880035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:53:55.880035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:53:55.880035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:53:55.880035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:53:55.880035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:53:55.880035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:53:55.880035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:53:55.880035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 609/1222] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:53:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:53:55.881967 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1a174-9bcf-4cd4-bdf1-f7a4c22ae94f None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:56.011002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1a174-9bcf-4cd4-bdf1-f7a4c22ae94f None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:56.040408 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:56.054237 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c757911-5a6e-4b04-9840-08fdf01dfb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:56.055543 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c757911-5a6e-4b04-9840-08fdf01dfb35 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:56.056038 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c757911-5a6e-4b04-9840-08fdf01dfb35 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:56.056524 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c757911-5a6e-4b04-9840-08fdf01dfb35 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:56.057910 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c757911-5a6e-4b04-9840-08fdf01dfb35 None admin] RBAC: Adding query filter params (name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:53:56.058486 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c757911-5a6e-4b04-9840-08fdf01dfb35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:56.060948 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c757911-5a6e-4b04-9840-08fdf01dfb35 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:56.070910 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 614/1223] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:53:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:53:56.077353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eccf60-7d9b-42de-b9ff-b867de975399 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:56.126412 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:56.190618 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eccf60-7d9b-42de-b9ff-b867de975399 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:56.229010 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64b073-4214-45cc-8a02-bdb5203c34c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:56.229792 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64b073-4214-45cc-8a02-bdb5203c34c9 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:56.230071 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64b073-4214-45cc-8a02-bdb5203c34c9 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:56.230389 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64b073-4214-45cc-8a02-bdb5203c34c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:56.243509 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64b073-4214-45cc-8a02-bdb5203c34c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:56.245092 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64b073-4214-45cc-8a02-bdb5203c34c9 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:56.264755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-aa64b073-4214-45cc-8a02-bdb5203c34c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:53:56.264755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:53:56.264755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:53:56.264755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:53:56.264755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:53:56.264755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:53:56.264755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:53:56.264755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:53:56.264755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:53:56.264755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:53:56.264755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:53:56.264755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:53:56.264755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:53:56.264755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:53:56.264755 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:53:56.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:53:56.266885 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:53:56.266885 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:53:56.266885 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:53:56.266885 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:53:56.266885 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 610/1224] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:53:56.272265 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c757911-5a6e-4b04-9840-08fdf01dfb35 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:56.386412 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c757911-5a6e-4b04-9840-08fdf01dfb35 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:56.422522 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beea4d4-84c2-4ee7-b1db-49e8bd73c341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:56.423197 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beea4d4-84c2-4ee7-b1db-49e8bd73c341 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:56.423479 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beea4d4-84c2-4ee7-b1db-49e8bd73c341 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:56.423754 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beea4d4-84c2-4ee7-b1db-49e8bd73c341 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:56.424490 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beea4d4-84c2-4ee7-b1db-49e8bd73c341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:53:56.424797 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beea4d4-84c2-4ee7-b1db-49e8bd73c341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:56.426177 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beea4d4-84c2-4ee7-b1db-49e8bd73c341 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:56.438312 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 615/1225] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:53:56.444400 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64b073-4214-45cc-8a02-bdb5203c34c9 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:56.556156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:56.563986 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64b073-4214-45cc-8a02-bdb5203c34c9 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:56.600978 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b2106-2fa9-4dc8-ad5d-922a00be8c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:56.603292 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b2106-2fa9-4dc8-ad5d-922a00be8c72 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:56.603781 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b2106-2fa9-4dc8-ad5d-922a00be8c72 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:56.604203 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b2106-2fa9-4dc8-ad5d-922a00be8c72 None admin] PATH_INFO: `/v3/users/placement` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:56.614241 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b2106-2fa9-4dc8-ad5d-922a00be8c72 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:56.615740 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b2106-2fa9-4dc8-ad5d-922a00be8c72 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-5d3b2106-2fa9-4dc8-ad5d-922a00be8c72 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:53:56.631005 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:53:56.632036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:53:56.633065 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:53:56.633065 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:53:56.633065 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:53:56.633065 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:53:56.633065 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:53:56.633065 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:53:56.633065 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:53:56.633065 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:53:56.633065 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:53:56.633065 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:53:56.633065 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 611/1226] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:53:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:53:56.638605 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beea4d4-84c2-4ee7-b1db-49e8bd73c341 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:56.688963 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:56.752233 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beea4d4-84c2-4ee7-b1db-49e8bd73c341 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:56.794897 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7f4b7-51e8-4467-b3ec-83cc8b41e8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:56.795626 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7f4b7-51e8-4467-b3ec-83cc8b41e8b6 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:56.795909 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7f4b7-51e8-4467-b3ec-83cc8b41e8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:56.796184 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7f4b7-51e8-4467-b3ec-83cc8b41e8b6 None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:56.797027 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7f4b7-51e8-4467-b3ec-83cc8b41e8b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:53:56.797331 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7f4b7-51e8-4467-b3ec-83cc8b41e8b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:56.798890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7f4b7-51e8-4467-b3ec-83cc8b41e8b6 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:56.818507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 616/1227] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 19:53:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:53:56.824670 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b2106-2fa9-4dc8-ad5d-922a00be8c72 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:56.940550 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b2106-2fa9-4dc8-ad5d-922a00be8c72 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:56.977903 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb3ce3-a833-4a8a-a8f6-5a9810a5e0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:56.978883 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb3ce3-a833-4a8a-a8f6-5a9810a5e0bf None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:56.980426 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb3ce3-a833-4a8a-a8f6-5a9810a5e0bf None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:56.980426 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb3ce3-a833-4a8a-a8f6-5a9810a5e0bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:56.994628 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb3ce3-a833-4a8a-a8f6-5a9810a5e0bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:56.996235 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb3ce3-a833-4a8a-a8f6-5a9810a5e0bf None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:57.014840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-babb3ce3-a833-4a8a-a8f6-5a9810a5e0bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:53:57.014840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:53:57.014840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:53:57.014840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:53:57.014840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:53:57.014840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:53:57.014840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:53:57.014840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:53:57.014840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:53:57.014840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:53:57.014840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:53:57.014840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:53:57.014840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:53:57.014840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:53:57.014840 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:53:57.015813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:53:57.016740 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:53:57.016740 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:53:57.016740 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:53:57.016740 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:53:57.016740 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 612/1228] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:53:57.022375 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7f4b7-51e8-4467-b3ec-83cc8b41e8b6 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:57.132906 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7f4b7-51e8-4467-b3ec-83cc8b41e8b6 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:57.135915 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:57.172629 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe437e37-e848-40cd-be67-b1f11dce4cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:57.173420 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe437e37-e848-40cd-be67-b1f11dce4cfa None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:57.173702 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe437e37-e848-40cd-be67-b1f11dce4cfa None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:57.174021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe437e37-e848-40cd-be67-b1f11dce4cfa None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:57.174904 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe437e37-e848-40cd-be67-b1f11dce4cfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:53:57.175211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe437e37-e848-40cd-be67-b1f11dce4cfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:57.176600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe437e37-e848-40cd-be67-b1f11dce4cfa None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:57.189458 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 617/1229] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:53:57.195528 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb3ce3-a833-4a8a-a8f6-5a9810a5e0bf None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:57.205491 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:57.312547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb3ce3-a833-4a8a-a8f6-5a9810a5e0bf None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:57.356292 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8582e0a-0655-430b-bca0-799a3f9136b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:57.357102 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8582e0a-0655-430b-bca0-799a3f9136b5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:57.357381 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8582e0a-0655-430b-bca0-799a3f9136b5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:57.357671 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8582e0a-0655-430b-bca0-799a3f9136b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:57.365413 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8582e0a-0655-430b-bca0-799a3f9136b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:57.366974 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8582e0a-0655-430b-bca0-799a3f9136b5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:57.382224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-a8582e0a-0655-430b-bca0-799a3f9136b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:53:57.382224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:53:57.382224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:53:57.382224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:53:57.382224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:53:57.382224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:53:57.382224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:53:57.382224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:53:57.382224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:53:57.382224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:53:57.382224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:53:57.382224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:53:57.382224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:53:57.382224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:53:57.382224 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:53:57.383438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:53:57.384546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:53:57.384546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:53:57.384546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:53:57.384546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:53:57.384546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:53:57.384546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:53:57.384546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:53:57.384546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:53:57.384546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 613/1230] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:53:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:53:57.389866 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe437e37-e848-40cd-be67-b1f11dce4cfa None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:57.499488 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe437e37-e848-40cd-be67-b1f11dce4cfa None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:57.536855 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c603b-7e81-4797-9111-9351e4e2df43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:57.537518 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c603b-7e81-4797-9111-9351e4e2df43 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:57.537799 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c603b-7e81-4797-9111-9351e4e2df43 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:57.538070 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c603b-7e81-4797-9111-9351e4e2df43 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:57.538898 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c603b-7e81-4797-9111-9351e4e2df43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:53:57.539203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c603b-7e81-4797-9111-9351e4e2df43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:57.540727 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c603b-7e81-4797-9111-9351e4e2df43 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:57.553069 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 618/1231] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:53:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:53:57.560197 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8582e0a-0655-430b-bca0-799a3f9136b5 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:53:57.632666 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:53:57.670670 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8582e0a-0655-430b-bca0-799a3f9136b5 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:53:57.709666 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5988e-431e-43a7-ae82-b84e36f47478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:53:57.710644 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5988e-431e-43a7-ae82-b84e36f47478 None admin] REQUEST_METHOD: `PUT` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:53:57.710930 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5988e-431e-43a7-ae82-b84e36f47478 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:53:57.711207 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5988e-431e-43a7-ae82-b84e36f47478 None admin] PATH_INFO: `/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/14b4b91113ed40f0bd0d2bcc6818138c/roles/3387bd4782b047ae8d880e386b0351f0` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:53:57.727441 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5988e-431e-43a7-ae82-b84e36f47478 None admin] RBAC: Authorizing `identity:create_grant(project_id=d638a29a793d41e7bfa686c0fcc29149, user_id=14b4b91113ed40f0bd0d2bcc6818138c, role_id=3387bd4782b047ae8d880e386b0351f0)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:53:57.729151 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5988e-431e-43a7-ae82-b84e36f47478 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:53:57.743471 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 614/1232] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:53:57 2022] PUT /identity/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/14b4b91113ed40f0bd0d2bcc6818138c/roles/3387bd4782b047ae8d880e386b0351f0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:54:00.086388 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ffd89-63ca-4e1a-ad51-48634ef24408 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:00.089158 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ffd89-63ca-4e1a-ad51-48634ef24408 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:00.089158 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ffd89-63ca-4e1a-ad51-48634ef24408 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:00.089158 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 619/1233] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:54:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:54:00.095163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd9bb1-296d-4332-b6d8-8cf98d5d4b42 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:00.095614 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd9bb1-296d-4332-b6d8-8cf98d5d4b42 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:00.095713 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd9bb1-296d-4332-b6d8-8cf98d5d4b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:00.113134 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-46dd9bb1-296d-4332-b6d8-8cf98d5d4b42 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:54:00.114375 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-46dd9bb1-296d-4332-b6d8-8cf98d5d4b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:54:00.186959 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 615/1234] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:54:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:54:00.212335 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ffd89-63ca-4e1a-ad51-48634ef24408 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:00.241134 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:00.331066 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ffd89-63ca-4e1a-ad51-48634ef24408 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:00.377470 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae8fdd-8a6e-479a-8b23-da605b1bc57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:00.378448 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae8fdd-8a6e-479a-8b23-da605b1bc57b None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:00.378891 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae8fdd-8a6e-479a-8b23-da605b1bc57b None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:00.379354 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae8fdd-8a6e-479a-8b23-da605b1bc57b None admin] PATH_INFO: `/v3/roles/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:00.390878 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae8fdd-8a6e-479a-8b23-da605b1bc57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-90ae8fdd-8a6e-479a-8b23-da605b1bc57b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:54:00.399282 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:00.400451 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:00.401415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:00.401415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:00.401415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:00.401415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:00.401415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:54:00.401415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:54:00.401415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:54:00.401415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:54:00.401415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:54:00.401415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:54:00.401415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:54:00.401415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:54:00.401415 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 620/1235] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 19:54:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:54:00.406928 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd9bb1-296d-4332-b6d8-8cf98d5d4b42 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:00.462246 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:00.522647 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd9bb1-296d-4332-b6d8-8cf98d5d4b42 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:00.558534 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f305d-e214-4912-859e-0ac40b7bb548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:00.559242 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f305d-e214-4912-859e-0ac40b7bb548 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:00.559547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f305d-e214-4912-859e-0ac40b7bb548 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:00.559838 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f305d-e214-4912-859e-0ac40b7bb548 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:00.560966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f305d-e214-4912-859e-0ac40b7bb548 None admin] RBAC: Adding query filter params (name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:00.561352 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f305d-e214-4912-859e-0ac40b7bb548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:00.563095 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f305d-e214-4912-859e-0ac40b7bb548 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:00.572393 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 616/1236] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 19:54:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:00.578379 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae8fdd-8a6e-479a-8b23-da605b1bc57b None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:00.672193 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:00.690835 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae8fdd-8a6e-479a-8b23-da605b1bc57b None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:00.731188 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3952d-c62b-4941-a5f7-71b28b54c419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:00.732008 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3952d-c62b-4941-a5f7-71b28b54c419 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:00.732298 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3952d-c62b-4941-a5f7-71b28b54c419 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:00.732575 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3952d-c62b-4941-a5f7-71b28b54c419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:00.745373 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3952d-c62b-4941-a5f7-71b28b54c419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:00.746182 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3952d-c62b-4941-a5f7-71b28b54c419 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:00.766507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-efd3952d-c62b-4941-a5f7-71b28b54c419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:00.766507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:00.766507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:00.766507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:00.766507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:00.766507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:00.766507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:00.766507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:00.766507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:00.766507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:00.766507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:00.766507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:00.766507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:54:00.766507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:54:00.766507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:00.767796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:00.768857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:54:00.768857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:54:00.768857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:00.768857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:54:00.768857 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 621/1237] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:54:00.773992 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f305d-e214-4912-859e-0ac40b7bb548 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:00.888101 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f305d-e214-4912-859e-0ac40b7bb548 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:00.899089 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:00.927087 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b46ba2-1806-41cb-b183-15cd76927d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:00.927690 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b46ba2-1806-41cb-b183-15cd76927d50 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:00.927974 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b46ba2-1806-41cb-b183-15cd76927d50 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:00.928249 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b46ba2-1806-41cb-b183-15cd76927d50 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:00.928983 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b46ba2-1806-41cb-b183-15cd76927d50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:00.929291 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b46ba2-1806-41cb-b183-15cd76927d50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:00.930778 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b46ba2-1806-41cb-b183-15cd76927d50 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:00.942283 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 617/1238] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:00.948602 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3952d-c62b-4941-a5f7-71b28b54c419 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:01.054814 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3952d-c62b-4941-a5f7-71b28b54c419 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:01.093377 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b003a-9347-4c38-915c-8d1f4e83cbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:01.094195 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b003a-9347-4c38-915c-8d1f4e83cbb2 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:01.094834 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b003a-9347-4c38-915c-8d1f4e83cbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:01.095158 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b003a-9347-4c38-915c-8d1f4e83cbb2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:01.098433 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:01.104336 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b003a-9347-4c38-915c-8d1f4e83cbb2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:01.106042 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b003a-9347-4c38-915c-8d1f4e83cbb2 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-169b003a-9347-4c38-915c-8d1f4e83cbb2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:54:01.121942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:01.123127 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:01.124382 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:01.124382 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:01.124382 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:54:01.124382 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:54:01.124382 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:54:01.124382 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:54:01.124382 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:54:01.124382 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:54:01.124382 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:54:01.124382 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:54:01.124382 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 622/1239] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:54:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:54:01.129391 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b46ba2-1806-41cb-b183-15cd76927d50 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:01.240963 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b46ba2-1806-41cb-b183-15cd76927d50 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:01.277653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398e9ed-ae29-4cbc-bcc0-61da48dd9ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:01.278477 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e9ed-ae29-4cbc-bcc0-61da48dd9ab8 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:01.278861 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e9ed-ae29-4cbc-bcc0-61da48dd9ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:01.279216 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e9ed-ae29-4cbc-bcc0-61da48dd9ab8 None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:01.280269 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398e9ed-ae29-4cbc-bcc0-61da48dd9ab8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:01.280722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398e9ed-ae29-4cbc-bcc0-61da48dd9ab8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:01.282368 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398e9ed-ae29-4cbc-bcc0-61da48dd9ab8 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:01.303313 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 618/1240] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 19:54:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:01.309468 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b003a-9347-4c38-915c-8d1f4e83cbb2 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:01.425835 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b003a-9347-4c38-915c-8d1f4e83cbb2 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:01.462103 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746f39c-2779-404f-b24c-a639d6d38eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:01.462797 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746f39c-2779-404f-b24c-a639d6d38eb3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:01.463081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746f39c-2779-404f-b24c-a639d6d38eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:01.463354 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746f39c-2779-404f-b24c-a639d6d38eb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:01.530821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:01.538610 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746f39c-2779-404f-b24c-a639d6d38eb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:01.540556 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746f39c-2779-404f-b24c-a639d6d38eb3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:01.648495 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 623/1241] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:54:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:54:01.654593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398e9ed-ae29-4cbc-bcc0-61da48dd9ab8 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:01.683881 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:01.763347 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398e9ed-ae29-4cbc-bcc0-61da48dd9ab8 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:01.801204 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb23ba7-2337-48b1-b8e6-e31c52ddaf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:01.802009 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb23ba7-2337-48b1-b8e6-e31c52ddaf1e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:01.802301 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb23ba7-2337-48b1-b8e6-e31c52ddaf1e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:01.802593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb23ba7-2337-48b1-b8e6-e31c52ddaf1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:01.813377 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb23ba7-2337-48b1-b8e6-e31c52ddaf1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:01.814927 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb23ba7-2337-48b1-b8e6-e31c52ddaf1e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:01.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-efb23ba7-2337-48b1-b8e6-e31c52ddaf1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:01.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:01.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:01.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:01.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:01.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:01.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:01.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:01.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:01.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:01.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:01.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:01.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:54:01.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:54:01.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:01.835502 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:01.841081 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:54:01.841081 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:54:01.841081 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:01.841081 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:01.841081 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 619/1242] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:54:01.841531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746f39c-2779-404f-b24c-a639d6d38eb3 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:01.946668 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:01.955128 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746f39c-2779-404f-b24c-a639d6d38eb3 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:01.993892 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e34856-e45f-4203-a52c-a7ac39fec9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:01.994603 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e34856-e45f-4203-a52c-a7ac39fec9e5 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:01.994882 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e34856-e45f-4203-a52c-a7ac39fec9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:01.995157 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e34856-e45f-4203-a52c-a7ac39fec9e5 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:01.995900 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e34856-e45f-4203-a52c-a7ac39fec9e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:01.996201 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e34856-e45f-4203-a52c-a7ac39fec9e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:01.997619 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e34856-e45f-4203-a52c-a7ac39fec9e5 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:02.010425 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 624/1243] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:02.016309 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb23ba7-2337-48b1-b8e6-e31c52ddaf1e None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:02.107306 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:02.131677 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb23ba7-2337-48b1-b8e6-e31c52ddaf1e None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:02.168119 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd801a-f459-4834-a5cc-22582a546cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:02.168932 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd801a-f459-4834-a5cc-22582a546cb1 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:02.169214 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd801a-f459-4834-a5cc-22582a546cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:02.169492 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd801a-f459-4834-a5cc-22582a546cb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:02.176567 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd801a-f459-4834-a5cc-22582a546cb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:02.178078 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd801a-f459-4834-a5cc-22582a546cb1 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:02.192336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-2bdd801a-f459-4834-a5cc-22582a546cb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:54:02.192336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:02.192336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:02.192336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:02.192336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:02.192336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:02.192336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:02.192336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:02.192336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:02.192336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:02.192336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:02.192336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:02.192336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:54:02.192336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:54:02.192336 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:02.193373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:02.194298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:54:02.194298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:54:02.194298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:54:02.194298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:54:02.194298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:54:02.194298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:54:02.194298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:54:02.194298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:02.194298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 620/1244] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:54:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:54:02.199760 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e34856-e45f-4203-a52c-a7ac39fec9e5 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:02.324932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e34856-e45f-4203-a52c-a7ac39fec9e5 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:02.372597 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088970c4-a185-4ce9-befa-d3a303e12662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:02.373313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088970c4-a185-4ce9-befa-d3a303e12662 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:02.373781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088970c4-a185-4ce9-befa-d3a303e12662 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:02.374035 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088970c4-a185-4ce9-befa-d3a303e12662 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:02.374883 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088970c4-a185-4ce9-befa-d3a303e12662 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:02.375243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088970c4-a185-4ce9-befa-d3a303e12662 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:02.376648 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088970c4-a185-4ce9-befa-d3a303e12662 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:02.391538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 625/1245] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:54:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:02.397590 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd801a-f459-4834-a5cc-22582a546cb1 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:02.509383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd801a-f459-4834-a5cc-22582a546cb1 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:02.545855 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ced4b-dd72-466d-bbbc-680c319f3d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:02.546635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ced4b-dd72-466d-bbbc-680c319f3d8e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:02.546882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ced4b-dd72-466d-bbbc-680c319f3d8e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:02.547163 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ced4b-dd72-466d-bbbc-680c319f3d8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:02.548037 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ced4b-dd72-466d-bbbc-680c319f3d8e None admin] RBAC: Adding query filter params (role.id=3387bd4782b047ae8d880e386b0351f0, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=14b4b91113ed40f0bd0d2bcc6818138c) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:02.548326 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ced4b-dd72-466d-bbbc-680c319f3d8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:02.549956 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ced4b-dd72-466d-bbbc-680c319f3d8e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:02.557319 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 621/1246] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:54:02 2022] GET /identity/v3/role_assignments?user.id=14b4b91113ed40f0bd0d2bcc6818138c&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=3387bd4782b047ae8d880e386b0351f0 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:04.838277 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e1595-84e5-4319-9a80-137b10688332 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:04.838836 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e1595-84e5-4319-9a80-137b10688332 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:04.838836 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e1595-84e5-4319-9a80-137b10688332 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:04.840225 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 626/1247] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:54:04.846046 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67dfd0-0f0b-471c-9df9-1f9a268a0651 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:04.846510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67dfd0-0f0b-471c-9df9-1f9a268a0651 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:04.846623 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67dfd0-0f0b-471c-9df9-1f9a268a0651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:04.851951 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:04.864809 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-1a67dfd0-0f0b-471c-9df9-1f9a268a0651 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:54:04.866017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-1a67dfd0-0f0b-471c-9df9-1f9a268a0651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:54:04.937589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 622/1248] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:54:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:54:04.967187 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e1595-84e5-4319-9a80-137b10688332 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:04.992808 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:05.091353 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e1595-84e5-4319-9a80-137b10688332 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:05.134114 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d607f4b-d53c-44f7-a982-d205809b8816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:05.134948 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d607f4b-d53c-44f7-a982-d205809b8816 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:05.135234 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d607f4b-d53c-44f7-a982-d205809b8816 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:05.135504 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d607f4b-d53c-44f7-a982-d205809b8816 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:05.156573 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d607f4b-d53c-44f7-a982-d205809b8816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-0d607f4b-d53c-44f7-a982-d205809b8816 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:54:05.165076 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:05.166462 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:05.167536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:05.167536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:05.167536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:05.167536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:05.167536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:54:05.167536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:54:05.167536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:54:05.167536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:54:05.167536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:54:05.167536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:54:05.167536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:54:05.167536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:54:05.167536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 627/1249] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:54:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:05.174498 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67dfd0-0f0b-471c-9df9-1f9a268a0651 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:05.304418 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67dfd0-0f0b-471c-9df9-1f9a268a0651 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:05.349141 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:05.360084 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72235b88-0b59-4152-8a38-b11379349876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:05.361118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72235b88-0b59-4152-8a38-b11379349876 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:05.361410 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72235b88-0b59-4152-8a38-b11379349876 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:05.361686 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72235b88-0b59-4152-8a38-b11379349876 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:05.362610 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72235b88-0b59-4152-8a38-b11379349876 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:05.362848 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72235b88-0b59-4152-8a38-b11379349876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:05.364353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72235b88-0b59-4152-8a38-b11379349876 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:05.376851 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 623/1250] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:54:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:05.383572 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d607f4b-d53c-44f7-a982-d205809b8816 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:05.482773 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:05.512498 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d607f4b-d53c-44f7-a982-d205809b8816 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:05.557143 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c240d2-2b87-44e1-9050-7bb9c0232622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:05.559411 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c240d2-2b87-44e1-9050-7bb9c0232622 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:05.559916 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c240d2-2b87-44e1-9050-7bb9c0232622 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:05.560424 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c240d2-2b87-44e1-9050-7bb9c0232622 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:05.575018 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c240d2-2b87-44e1-9050-7bb9c0232622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:05.576980 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c240d2-2b87-44e1-9050-7bb9c0232622 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:05.598245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-13c240d2-2b87-44e1-9050-7bb9c0232622 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:05.598245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:05.598245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:05.598245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:05.598245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:05.598245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:05.598245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:05.598245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:05.598245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:05.598245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:05.598245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:05.598245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:05.598245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:54:05.598245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:54:05.598245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:05.600242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:05.602065 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:54:05.602065 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:54:05.602065 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:05.602065 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:54:05.602065 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 628/1251] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:05.607128 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72235b88-0b59-4152-8a38-b11379349876 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:05.744623 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72235b88-0b59-4152-8a38-b11379349876 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:05.788425 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd51f1e-e75c-4f22-82f8-7149c84e9c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:05.789227 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd51f1e-e75c-4f22-82f8-7149c84e9c80 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:05.789581 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd51f1e-e75c-4f22-82f8-7149c84e9c80 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:05.789972 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd51f1e-e75c-4f22-82f8-7149c84e9c80 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:05.790859 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd51f1e-e75c-4f22-82f8-7149c84e9c80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:05.791239 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd51f1e-e75c-4f22-82f8-7149c84e9c80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:05.792919 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd51f1e-e75c-4f22-82f8-7149c84e9c80 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:05.814383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 624/1252] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:05.817877 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c240d2-2b87-44e1-9050-7bb9c0232622 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:05.947743 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c240d2-2b87-44e1-9050-7bb9c0232622 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:05.978803 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:05.987979 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db9174-e71f-428c-9047-944212ca7e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:05.988821 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db9174-e71f-428c-9047-944212ca7e9a None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:05.989089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db9174-e71f-428c-9047-944212ca7e9a None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:05.989370 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db9174-e71f-428c-9047-944212ca7e9a None admin] PATH_INFO: `/v3/users/placement` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:06.001013 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db9174-e71f-428c-9047-944212ca7e9a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:06.002593 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db9174-e71f-428c-9047-944212ca7e9a None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-51db9174-e71f-428c-9047-944212ca7e9a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:54:06.029696 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:06.031089 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:06.032441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:06.032441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:06.032441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:54:06.032441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:54:06.032441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:54:06.032441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:54:06.032441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:54:06.032441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:54:06.032441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:54:06.032441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:54:06.032441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 629/1253] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:54:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:06.037241 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd51f1e-e75c-4f22-82f8-7149c84e9c80 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:06.102165 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:06.151164 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd51f1e-e75c-4f22-82f8-7149c84e9c80 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:06.193561 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0584c5-4413-4396-91f3-3181d8795ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:06.194316 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0584c5-4413-4396-91f3-3181d8795ee8 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:06.194659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0584c5-4413-4396-91f3-3181d8795ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:06.194937 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0584c5-4413-4396-91f3-3181d8795ee8 None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:06.195882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0584c5-4413-4396-91f3-3181d8795ee8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:06.196318 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0584c5-4413-4396-91f3-3181d8795ee8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:06.197830 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0584c5-4413-4396-91f3-3181d8795ee8 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:06.222812 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 625/1254] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 19:54:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:54:06.224185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db9174-e71f-428c-9047-944212ca7e9a None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:06.348779 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db9174-e71f-428c-9047-944212ca7e9a None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:06.411165 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bca7d2-d626-4520-8e74-9ba9175f3ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:06.411814 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bca7d2-d626-4520-8e74-9ba9175f3ca8 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:06.412106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bca7d2-d626-4520-8e74-9ba9175f3ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:06.412578 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bca7d2-d626-4520-8e74-9ba9175f3ca8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:06.478911 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:06.492768 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bca7d2-d626-4520-8e74-9ba9175f3ca8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:06.494594 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bca7d2-d626-4520-8e74-9ba9175f3ca8 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:06.605666 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 630/1255] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:54:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:54:06.613842 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0584c5-4413-4396-91f3-3181d8795ee8 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:06.737531 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0584c5-4413-4396-91f3-3181d8795ee8 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:06.754127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:06.781036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58433230-6ab5-40fc-a505-d69b8fa2d99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:06.781907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58433230-6ab5-40fc-a505-d69b8fa2d99e None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:06.782322 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58433230-6ab5-40fc-a505-d69b8fa2d99e None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:06.782559 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58433230-6ab5-40fc-a505-d69b8fa2d99e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:06.793612 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58433230-6ab5-40fc-a505-d69b8fa2d99e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:06.795208 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58433230-6ab5-40fc-a505-d69b8fa2d99e None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:06.814913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-58433230-6ab5-40fc-a505-d69b8fa2d99e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:06.814913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:06.814913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:06.814913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:06.814913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:06.814913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:06.814913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:06.814913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:06.814913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:06.814913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:06.814913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:06.814913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:06.814913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:54:06.814913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:54:06.814913 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:06.815966 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:06.817104 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:54:06.817104 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:54:06.817104 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:06.817104 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:06.817104 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 626/1256] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:54:06.822816 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bca7d2-d626-4520-8e74-9ba9175f3ca8 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:06.927521 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:06.941245 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bca7d2-d626-4520-8e74-9ba9175f3ca8 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:06.983058 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b597263-a43e-4a7b-9d0a-a51cf5581783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:06.983874 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b597263-a43e-4a7b-9d0a-a51cf5581783 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:06.984053 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b597263-a43e-4a7b-9d0a-a51cf5581783 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:06.984409 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b597263-a43e-4a7b-9d0a-a51cf5581783 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:06.985203 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b597263-a43e-4a7b-9d0a-a51cf5581783 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:06.985548 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b597263-a43e-4a7b-9d0a-a51cf5581783 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:06.987183 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b597263-a43e-4a7b-9d0a-a51cf5581783 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:07.000417 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 631/1257] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:54:07.006379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58433230-6ab5-40fc-a505-d69b8fa2d99e None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:07.118712 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58433230-6ab5-40fc-a505-d69b8fa2d99e None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:07.156470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f57d62-7be5-4b65-9f22-9f93d61b9862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:07.157286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f57d62-7be5-4b65-9f22-9f93d61b9862 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:07.157569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f57d62-7be5-4b65-9f22-9f93d61b9862 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:07.157843 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f57d62-7be5-4b65-9f22-9f93d61b9862 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:07.166643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f57d62-7be5-4b65-9f22-9f93d61b9862 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:07.168435 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f57d62-7be5-4b65-9f22-9f93d61b9862 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:07.182563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-f1f57d62-7be5-4b65-9f22-9f93d61b9862 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:54:07.182563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:07.182563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:07.182563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:07.182563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:07.182563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:07.182563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:07.182563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:07.182563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:07.182563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:07.182563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:07.182563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:07.182563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:54:07.182563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:54:07.182563 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:07.183772 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:07.184816 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:54:07.184816 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:54:07.184816 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:54:07.184816 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:54:07.184816 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:54:07.184816 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:54:07.184816 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:54:07.184816 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:07.184816 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 627/1258] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:54:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:07.189779 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b597263-a43e-4a7b-9d0a-a51cf5581783 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:07.305956 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b597263-a43e-4a7b-9d0a-a51cf5581783 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:07.344244 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b2af8-efec-4bdd-8c71-7475e6090ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:07.345518 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b2af8-efec-4bdd-8c71-7475e6090ab4 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:07.345830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b2af8-efec-4bdd-8c71-7475e6090ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:07.346129 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b2af8-efec-4bdd-8c71-7475e6090ab4 None admin] PATH_INFO: `/v3/projects` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:07.347188 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b2af8-efec-4bdd-8c71-7475e6090ab4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:07.347588 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b2af8-efec-4bdd-8c71-7475e6090ab4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:07.349204 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b2af8-efec-4bdd-8c71-7475e6090ab4 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:07.366311 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 632/1259] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:54:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:07.371499 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f57d62-7be5-4b65-9f22-9f93d61b9862 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:07.401703 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:07.485717 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f57d62-7be5-4b65-9f22-9f93d61b9862 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:07.521809 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162ca4f-b6c1-47ae-a284-7d247853e151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:07.522602 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162ca4f-b6c1-47ae-a284-7d247853e151 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:07.522957 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162ca4f-b6c1-47ae-a284-7d247853e151 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:07.523302 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162ca4f-b6c1-47ae-a284-7d247853e151 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:07.524408 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162ca4f-b6c1-47ae-a284-7d247853e151 None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=14b4b91113ed40f0bd0d2bcc6818138c) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:07.524801 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162ca4f-b6c1-47ae-a284-7d247853e151 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:07.526426 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162ca4f-b6c1-47ae-a284-7d247853e151 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:07.532159 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 628/1260] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:54:07 2022] GET /identity/v3/role_assignments?user.id=14b4b91113ed40f0bd0d2bcc6818138c&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:09.746289 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f337ad-d574-40a6-9e59-5ca05bddc1a9 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:09.746807 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f337ad-d574-40a6-9e59-5ca05bddc1a9 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:09.746899 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f337ad-d574-40a6-9e59-5ca05bddc1a9 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:09.748277 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 633/1261] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:54:09.753598 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef95dd-d269-45b9-9c01-5bd14e835c8a None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:09.754047 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef95dd-d269-45b9-9c01-5bd14e835c8a None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:09.754146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef95dd-d269-45b9-9c01-5bd14e835c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:09.772741 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-0cef95dd-d269-45b9-9c01-5bd14e835c8a None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:54:09.774189 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-0cef95dd-d269-45b9-9c01-5bd14e835c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:54:09.846043 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 629/1262] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:54:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:54:09.868542 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f337ad-d574-40a6-9e59-5ca05bddc1a9 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:09.879692 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:09.972215 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f337ad-d574-40a6-9e59-5ca05bddc1a9 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:10.004578 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eebd8c-211e-47ab-837b-42da502cebf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:10.005396 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eebd8c-211e-47ab-837b-42da502cebf2 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:10.005672 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eebd8c-211e-47ab-837b-42da502cebf2 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:10.005942 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eebd8c-211e-47ab-837b-42da502cebf2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:10.016747 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eebd8c-211e-47ab-837b-42da502cebf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-f3eebd8c-211e-47ab-837b-42da502cebf2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:54:10.024314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:10.025364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:10.026531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:10.026531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:10.026531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:10.026531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:10.026531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:54:10.026531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:54:10.026531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:54:10.026531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:54:10.026531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:54:10.026531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:54:10.026531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:54:10.026531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:54:10.026531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 634/1263] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:54:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:10.032261 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef95dd-d269-45b9-9c01-5bd14e835c8a None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:10.054367 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:10.139721 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef95dd-d269-45b9-9c01-5bd14e835c8a None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:10.176384 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e770a7b-2d23-4e00-af43-53ebfb6bd9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:10.177049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e770a7b-2d23-4e00-af43-53ebfb6bd9c5 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:10.177324 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e770a7b-2d23-4e00-af43-53ebfb6bd9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:10.177595 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e770a7b-2d23-4e00-af43-53ebfb6bd9c5 None admin] PATH_INFO: `/v3/roles` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:10.178439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e770a7b-2d23-4e00-af43-53ebfb6bd9c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:10.178745 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e770a7b-2d23-4e00-af43-53ebfb6bd9c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:10.180177 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e770a7b-2d23-4e00-af43-53ebfb6bd9c5 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:10.188277 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 630/1264] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:54:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:10.194296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eebd8c-211e-47ab-837b-42da502cebf2 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:10.267763 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:10.302124 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eebd8c-211e-47ab-837b-42da502cebf2 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:10.338345 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8044751-a9f5-47fd-9294-04cec0851580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:10.339323 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8044751-a9f5-47fd-9294-04cec0851580 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:10.339656 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8044751-a9f5-47fd-9294-04cec0851580 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:10.339957 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8044751-a9f5-47fd-9294-04cec0851580 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:10.354636 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8044751-a9f5-47fd-9294-04cec0851580 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:10.356207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8044751-a9f5-47fd-9294-04cec0851580 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:10.377983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-b8044751-a9f5-47fd-9294-04cec0851580 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:10.377983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:10.377983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:10.377983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:10.377983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:10.377983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:10.377983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:10.377983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:10.377983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:10.377983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:10.377983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:10.377983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:10.377983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:54:10.377983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:54:10.377983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:10.379220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:10.380309 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:54:10.380309 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:54:10.380309 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:10.380309 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:54:10.380309 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 635/1265] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:10.385294 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e770a7b-2d23-4e00-af43-53ebfb6bd9c5 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:10.477335 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:10.500358 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e770a7b-2d23-4e00-af43-53ebfb6bd9c5 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:10.546631 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ae73d-2984-48f8-bb4a-fdd33febae17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:10.547638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ae73d-2984-48f8-bb4a-fdd33febae17 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:10.547931 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ae73d-2984-48f8-bb4a-fdd33febae17 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:10.548349 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ae73d-2984-48f8-bb4a-fdd33febae17 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:10.549490 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ae73d-2984-48f8-bb4a-fdd33febae17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:10.549911 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ae73d-2984-48f8-bb4a-fdd33febae17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:10.552114 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ae73d-2984-48f8-bb4a-fdd33febae17 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:10.565361 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 631/1266] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:10.574427 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8044751-a9f5-47fd-9294-04cec0851580 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:10.688599 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8044751-a9f5-47fd-9294-04cec0851580 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:10.709733 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:10.725890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e27d0-1421-48c0-9bc0-8cbca01889f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:10.726992 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e27d0-1421-48c0-9bc0-8cbca01889f0 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:10.726992 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e27d0-1421-48c0-9bc0-8cbca01889f0 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:10.727313 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e27d0-1421-48c0-9bc0-8cbca01889f0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:10.735180 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58e27d0-1421-48c0-9bc0-8cbca01889f0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:10.736623 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58e27d0-1421-48c0-9bc0-8cbca01889f0 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-f58e27d0-1421-48c0-9bc0-8cbca01889f0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:54:10.750978 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:10.752531 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:10.753759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:10.753759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:10.753759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:54:10.753759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:54:10.753759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:54:10.753759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:54:10.753759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:54:10.753759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:54:10.753759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:54:10.753759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:54:10.753759 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 636/1267] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:54:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:10.758447 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ae73d-2984-48f8-bb4a-fdd33febae17 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:10.862560 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ae73d-2984-48f8-bb4a-fdd33febae17 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:10.901902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a09ac-df93-46ac-b40d-e24feb0eba9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:10.902589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a09ac-df93-46ac-b40d-e24feb0eba9f None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:10.902865 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a09ac-df93-46ac-b40d-e24feb0eba9f None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:10.903159 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a09ac-df93-46ac-b40d-e24feb0eba9f None admin] PATH_INFO: `/v3/users` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:10.903983 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a09ac-df93-46ac-b40d-e24feb0eba9f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:10.904295 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a09ac-df93-46ac-b40d-e24feb0eba9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:10.906018 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a09ac-df93-46ac-b40d-e24feb0eba9f None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:10.915658 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:10.929631 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 632/1268] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 19:54:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:10.939162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e27d0-1421-48c0-9bc0-8cbca01889f0 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:11.049129 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e27d0-1421-48c0-9bc0-8cbca01889f0 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:11.088833 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50422ee2-f0c3-456e-abb3-ac891a4426b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:11.089614 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50422ee2-f0c3-456e-abb3-ac891a4426b3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:11.089898 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50422ee2-f0c3-456e-abb3-ac891a4426b3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:11.090175 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50422ee2-f0c3-456e-abb3-ac891a4426b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:11.102624 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50422ee2-f0c3-456e-abb3-ac891a4426b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:11.104094 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50422ee2-f0c3-456e-abb3-ac891a4426b3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:11.125865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-50422ee2-f0c3-456e-abb3-ac891a4426b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:11.125865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:11.125865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:11.125865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:11.125865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:11.125865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:11.125865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:11.125865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:11.125865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:11.125865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:11.125865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:11.125865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:11.125865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:54:11.125865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:54:11.125865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:11.126890 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:11.127838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:54:11.127838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:54:11.127838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:11.127838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:54:11.127838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 637/1269] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:11.132793 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a09ac-df93-46ac-b40d-e24feb0eba9f None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:11.235470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a09ac-df93-46ac-b40d-e24feb0eba9f None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:11.268996 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4dd743-e1d5-47e7-bee8-5835d97c82e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:11.269653 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dd743-e1d5-47e7-bee8-5835d97c82e4 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:11.269941 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dd743-e1d5-47e7-bee8-5835d97c82e4 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:11.270221 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dd743-e1d5-47e7-bee8-5835d97c82e4 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:11.270997 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dd743-e1d5-47e7-bee8-5835d97c82e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:11.271312 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dd743-e1d5-47e7-bee8-5835d97c82e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:11.272700 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dd743-e1d5-47e7-bee8-5835d97c82e4 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:11.284292 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 633/1270] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:11.293719 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50422ee2-f0c3-456e-abb3-ac891a4426b3 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:11.338768 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:11.413694 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50422ee2-f0c3-456e-abb3-ac891a4426b3 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:11.451143 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd92f7e-27f3-41cc-a0f4-6bb2a4324833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:11.451938 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd92f7e-27f3-41cc-a0f4-6bb2a4324833 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:11.452220 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd92f7e-27f3-41cc-a0f4-6bb2a4324833 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:11.452495 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd92f7e-27f3-41cc-a0f4-6bb2a4324833 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:11.459716 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd92f7e-27f3-41cc-a0f4-6bb2a4324833 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:11.461243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd92f7e-27f3-41cc-a0f4-6bb2a4324833 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:11.475876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-2dd92f7e-27f3-41cc-a0f4-6bb2a4324833 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:54:11.475876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:11.475876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:11.475876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:11.475876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:11.475876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:11.475876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:11.475876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:11.475876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:11.475876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:11.475876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:11.475876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:11.475876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:54:11.475876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:54:11.475876 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:11.476861 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:11.477791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:54:11.477791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:54:11.477791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:54:11.477791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:54:11.477791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:54:11.477791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:54:11.477791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:54:11.477791 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:54:11.481898 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 638/1271] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:54:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:11.483146 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4dd743-e1d5-47e7-bee8-5835d97c82e4 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:11.545468 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:11.590362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4dd743-e1d5-47e7-bee8-5835d97c82e4 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:11.628643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1df0a9-1dc0-472b-8e52-8dd9e91fde12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:11.629297 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1df0a9-1dc0-472b-8e52-8dd9e91fde12 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:11.629584 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1df0a9-1dc0-472b-8e52-8dd9e91fde12 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:11.629869 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1df0a9-1dc0-472b-8e52-8dd9e91fde12 None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:11.630703 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1df0a9-1dc0-472b-8e52-8dd9e91fde12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:11.631018 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1df0a9-1dc0-472b-8e52-8dd9e91fde12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:11.632469 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1df0a9-1dc0-472b-8e52-8dd9e91fde12 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:11.645144 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 634/1272] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:54:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:54:11.654612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd92f7e-27f3-41cc-a0f4-6bb2a4324833 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:11.762303 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:11.772421 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd92f7e-27f3-41cc-a0f4-6bb2a4324833 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:11.807618 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7e958-47be-4196-aaa5-27358c1a38e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:11.808511 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7e958-47be-4196-aaa5-27358c1a38e0 None admin] REQUEST_METHOD: `PUT` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:11.808786 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7e958-47be-4196-aaa5-27358c1a38e0 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:11.809052 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7e958-47be-4196-aaa5-27358c1a38e0 None admin] PATH_INFO: `/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/14b4b91113ed40f0bd0d2bcc6818138c/roles/2cd6d3dd613c4824a26cacf53fb6c07f` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:11.811114 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7e958-47be-4196-aaa5-27358c1a38e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d638a29a793d41e7bfa686c0fcc29149, user_id=14b4b91113ed40f0bd0d2bcc6818138c, role_id=2cd6d3dd613c4824a26cacf53fb6c07f)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:11.812559 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7e958-47be-4196-aaa5-27358c1a38e0 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:11.828711 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 639/1273] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 19:54:11 2022] PUT /identity/v3/projects/d638a29a793d41e7bfa686c0fcc29149/users/14b4b91113ed40f0bd0d2bcc6818138c/roles/2cd6d3dd613c4824a26cacf53fb6c07f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:54:14.023685 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a6a8-aa74-4b75-aecc-742d286a5d50 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:14.024217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a6a8-aa74-4b75-aecc-742d286a5d50 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:14.024217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a6a8-aa74-4b75-aecc-742d286a5d50 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:14.025576 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 635/1274] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:54:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:54:14.034582 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683e48b-d212-4722-9804-569973d7ec5e None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:14.035331 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683e48b-d212-4722-9804-569973d7ec5e None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:14.035469 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683e48b-d212-4722-9804-569973d7ec5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:14.059778 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-8683e48b-d212-4722-9804-569973d7ec5e None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:54:14.061011 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-8683e48b-d212-4722-9804-569973d7ec5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:54:14.134288 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 640/1275] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:54:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:54:14.159218 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783a6a8-aa74-4b75-aecc-742d286a5d50 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:14.277378 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783a6a8-aa74-4b75-aecc-742d286a5d50 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:14.294770 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:14.313298 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01339b5d-f28a-4e01-978b-60cfca4f97f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:14.314088 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01339b5d-f28a-4e01-978b-60cfca4f97f1 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:14.314399 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01339b5d-f28a-4e01-978b-60cfca4f97f1 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:14.314670 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01339b5d-f28a-4e01-978b-60cfca4f97f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:14.326723 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01339b5d-f28a-4e01-978b-60cfca4f97f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-01339b5d-f28a-4e01-978b-60cfca4f97f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:54:14.336070 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:14.337510 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:14.338353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:14.338353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:14.338353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:14.338353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:14.338353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:54:14.338353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:54:14.338353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:54:14.338353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:54:14.338353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:54:14.338353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:54:14.338353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:54:14.338353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:14.338353 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 636/1276] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 19:54:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:14.344481 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683e48b-d212-4722-9804-569973d7ec5e None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:14.438181 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:14.459748 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683e48b-d212-4722-9804-569973d7ec5e None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:14.498147 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136666f-67d3-4664-a0cc-0d3e7720e664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:14.498801 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136666f-67d3-4664-a0cc-0d3e7720e664 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:14.499081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136666f-67d3-4664-a0cc-0d3e7720e664 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:14.499355 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136666f-67d3-4664-a0cc-0d3e7720e664 None admin] PATH_INFO: `/v3/roles` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:14.500115 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136666f-67d3-4664-a0cc-0d3e7720e664 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:14.500424 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136666f-67d3-4664-a0cc-0d3e7720e664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:14.501799 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136666f-67d3-4664-a0cc-0d3e7720e664 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:14.511035 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 641/1277] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 19:54:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:14.517806 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01339b5d-f28a-4e01-978b-60cfca4f97f1 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:14.636601 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01339b5d-f28a-4e01-978b-60cfca4f97f1 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:14.671667 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefaf76-fbf0-45a8-9fde-de0bcdac1fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:14.672452 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefaf76-fbf0-45a8-9fde-de0bcdac1fa0 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:14.672731 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefaf76-fbf0-45a8-9fde-de0bcdac1fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:14.673003 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefaf76-fbf0-45a8-9fde-de0bcdac1fa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:14.684201 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefaf76-fbf0-45a8-9fde-de0bcdac1fa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:14.685669 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefaf76-fbf0-45a8-9fde-de0bcdac1fa0 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:14.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-acefaf76-fbf0-45a8-9fde-de0bcdac1fa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:14.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:14.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:14.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:14.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:14.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:14.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:14.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:14.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:14.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:14.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:14.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:14.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:54:14.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:54:14.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:14.703979 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:14.704836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:54:14.704836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:54:14.704836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:14.704836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:14.704836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 637/1278] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:14.710484 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136666f-67d3-4664-a0cc-0d3e7720e664 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:14.827661 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136666f-67d3-4664-a0cc-0d3e7720e664 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:14.864054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4a4f6-ae17-4722-b58e-b6600312a679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:14.864709 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4a4f6-ae17-4722-b58e-b6600312a679 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:14.864988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4a4f6-ae17-4722-b58e-b6600312a679 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:14.865261 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4a4f6-ae17-4722-b58e-b6600312a679 None admin] PATH_INFO: `/v3/domains` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:14.865988 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4a4f6-ae17-4722-b58e-b6600312a679 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:14.866300 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4a4f6-ae17-4722-b58e-b6600312a679 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:14.867933 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4a4f6-ae17-4722-b58e-b6600312a679 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:14.880340 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 642/1279] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:14.886470 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefaf76-fbf0-45a8-9fde-de0bcdac1fa0 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:14.920382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:14.995244 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefaf76-fbf0-45a8-9fde-de0bcdac1fa0 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:15.033073 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa68081-4d5c-45bd-85de-99d9e6fc8ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:15.033870 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa68081-4d5c-45bd-85de-99d9e6fc8ccc None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:15.034150 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa68081-4d5c-45bd-85de-99d9e6fc8ccc None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:15.034451 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa68081-4d5c-45bd-85de-99d9e6fc8ccc None admin] PATH_INFO: `/v3/users/placement` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:15.042449 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa68081-4d5c-45bd-85de-99d9e6fc8ccc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:15.043906 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa68081-4d5c-45bd-85de-99d9e6fc8ccc None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-eaa68081-4d5c-45bd-85de-99d9e6fc8ccc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:54:15.058854 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:15.059907 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:15.064329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:15.064329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:15.064329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:54:15.064329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:54:15.064329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:54:15.064329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:54:15.064329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:54:15.064329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:54:15.064329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:54:15.064329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:15.064329 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 638/1280] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:54:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:54:15.069045 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4a4f6-ae17-4722-b58e-b6600312a679 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:15.075183 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:15.188842 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4a4f6-ae17-4722-b58e-b6600312a679 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:15.225358 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acecf33-fd5a-4d1e-b63a-7585bd5ba0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:15.226017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acecf33-fd5a-4d1e-b63a-7585bd5ba0eb None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:15.226366 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acecf33-fd5a-4d1e-b63a-7585bd5ba0eb None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:15.226640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acecf33-fd5a-4d1e-b63a-7585bd5ba0eb None admin] PATH_INFO: `/v3/users` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:15.227489 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acecf33-fd5a-4d1e-b63a-7585bd5ba0eb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:15.227794 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acecf33-fd5a-4d1e-b63a-7585bd5ba0eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:15.229237 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acecf33-fd5a-4d1e-b63a-7585bd5ba0eb None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:15.246580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 643/1281] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 19:54:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:15.255392 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa68081-4d5c-45bd-85de-99d9e6fc8ccc None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:15.358771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:15.368620 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa68081-4d5c-45bd-85de-99d9e6fc8ccc None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:15.403083 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfb95b-23cc-4888-a216-9dad1c819426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:15.403843 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfb95b-23cc-4888-a216-9dad1c819426 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:15.404240 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfb95b-23cc-4888-a216-9dad1c819426 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:15.404521 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfb95b-23cc-4888-a216-9dad1c819426 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:15.472918 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfb95b-23cc-4888-a216-9dad1c819426 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:15.474307 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfb95b-23cc-4888-a216-9dad1c819426 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:15.571245 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:15.582711 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 639/1282] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 19:54:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:54:15.590474 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acecf33-fd5a-4d1e-b63a-7585bd5ba0eb None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:15.669441 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:15.709051 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acecf33-fd5a-4d1e-b63a-7585bd5ba0eb None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:15.744727 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd5e6a-9b86-4008-9a3f-66c873f33626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:15.745450 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd5e6a-9b86-4008-9a3f-66c873f33626 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:15.745727 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd5e6a-9b86-4008-9a3f-66c873f33626 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:15.746000 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd5e6a-9b86-4008-9a3f-66c873f33626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:15.756549 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd5e6a-9b86-4008-9a3f-66c873f33626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:15.758018 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd5e6a-9b86-4008-9a3f-66c873f33626 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:15.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-6fbd5e6a-9b86-4008-9a3f-66c873f33626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:15.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:15.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:15.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:15.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:15.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:15.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:15.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:15.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:15.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:15.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:15.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:15.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:54:15.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:54:15.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:15.776677 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:15.778067 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:54:15.778067 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:54:15.778067 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:54:15.778067 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:54:15.778067 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 644/1283] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 19:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:15.785858 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfb95b-23cc-4888-a216-9dad1c819426 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:15.912140 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfb95b-23cc-4888-a216-9dad1c819426 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:15.950065 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc154f-9f56-4c53-bf4f-fb2c5da3b226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:15.950766 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc154f-9f56-4c53-bf4f-fb2c5da3b226 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:15.951054 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc154f-9f56-4c53-bf4f-fb2c5da3b226 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:15.951331 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc154f-9f56-4c53-bf4f-fb2c5da3b226 None admin] PATH_INFO: `/v3/domains` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:15.952078 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc154f-9f56-4c53-bf4f-fb2c5da3b226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:15.952385 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc154f-9f56-4c53-bf4f-fb2c5da3b226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:15.953774 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc154f-9f56-4c53-bf4f-fb2c5da3b226 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:15.966087 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 640/1284] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 19:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:15.975640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd5e6a-9b86-4008-9a3f-66c873f33626 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:16.089072 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd5e6a-9b86-4008-9a3f-66c873f33626 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:16.111016 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:16.135310 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48aa673-d5df-4030-b7da-6bfe33deddd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:16.136115 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48aa673-d5df-4030-b7da-6bfe33deddd3 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:16.136399 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48aa673-d5df-4030-b7da-6bfe33deddd3 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:16.136674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48aa673-d5df-4030-b7da-6bfe33deddd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:16.143903 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48aa673-d5df-4030-b7da-6bfe33deddd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:16.145359 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48aa673-d5df-4030-b7da-6bfe33deddd3 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:16.161343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application [None req-f48aa673-d5df-4030-b7da-6bfe33deddd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:54:16.161343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:16.161343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:16.161343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:16.161343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:16.161343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:16.161343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:16.161343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:16.161343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:16.161343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:16.161343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:16.161343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:16.161343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:54:16.161343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:54:16.161343 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application with Lock( May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:16.162338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:16.163436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:54:16.163436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:54:16.163436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:54:16.163436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:54:16.163436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:54:16.163436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:54:16.163436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:54:16.163436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: ERROR keystone.server.flask.application May 19 19:54:16.163436 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 645/1285] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 19:54:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:16.169635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc154f-9f56-4c53-bf4f-fb2c5da3b226 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:16.236981 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:16.289123 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc154f-9f56-4c53-bf4f-fb2c5da3b226 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:16.334355 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85058c-085d-43c1-b646-7d710ca4f9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:16.335482 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85058c-085d-43c1-b646-7d710ca4f9bf None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:16.336081 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85058c-085d-43c1-b646-7d710ca4f9bf None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:16.336508 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85058c-085d-43c1-b646-7d710ca4f9bf None admin] PATH_INFO: `/v3/projects` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:16.337354 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85058c-085d-43c1-b646-7d710ca4f9bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:16.337775 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85058c-085d-43c1-b646-7d710ca4f9bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:16.339687 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85058c-085d-43c1-b646-7d710ca4f9bf None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:16.355123 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 641/1286] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 19:54:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:16.361293 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48aa673-d5df-4030-b7da-6bfe33deddd3 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:16.465348 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48aa673-d5df-4030-b7da-6bfe33deddd3 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:16.501997 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeeda7e-8842-4768-99e7-1a77a01dda87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:16.502682 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeeda7e-8842-4768-99e7-1a77a01dda87 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:16.502965 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeeda7e-8842-4768-99e7-1a77a01dda87 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:16.503287 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeeda7e-8842-4768-99e7-1a77a01dda87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:16.504162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeeda7e-8842-4768-99e7-1a77a01dda87 None admin] RBAC: Adding query filter params (role.id=2cd6d3dd613c4824a26cacf53fb6c07f, scope.project.id=d638a29a793d41e7bfa686c0fcc29149, user.id=14b4b91113ed40f0bd0d2bcc6818138c) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:16.504482 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeeda7e-8842-4768-99e7-1a77a01dda87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:16.505943 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeeda7e-8842-4768-99e7-1a77a01dda87 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:16.512390 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 646/1287] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 19:54:16 2022] GET /identity/v3/role_assignments?user.id=14b4b91113ed40f0bd0d2bcc6818138c&scope.project.id=d638a29a793d41e7bfa686c0fcc29149&role.id=2cd6d3dd613c4824a26cacf53fb6c07f => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:18.787601 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ed474-1c20-439c-b4ee-1713f08f618c None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:18.788492 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ed474-1c20-439c-b4ee-1713f08f618c None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:18.788492 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ed474-1c20-439c-b4ee-1713f08f618c None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:18.790835 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 642/1288] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:54:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:54:18.799682 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd533857-35d4-4704-9f38-ce60e95e983f None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:18.800120 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd533857-35d4-4704-9f38-ce60e95e983f None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:18.800257 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd533857-35d4-4704-9f38-ce60e95e983f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:18.819293 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-cd533857-35d4-4704-9f38-ce60e95e983f None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:54:18.820746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-cd533857-35d4-4704-9f38-ce60e95e983f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:54:18.865243 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:18.909527 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 647/1289] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:54:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:54:18.932627 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ed474-1c20-439c-b4ee-1713f08f618c None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:19.049357 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ed474-1c20-439c-b4ee-1713f08f618c None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:19.060118 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:19.085937 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e38337-3f3b-4dc4-9c42-288faff34c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:19.086802 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e38337-3f3b-4dc4-9c42-288faff34c12 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:19.087087 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e38337-3f3b-4dc4-9c42-288faff34c12 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:19.087369 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e38337-3f3b-4dc4-9c42-288faff34c12 None admin] PATH_INFO: `/v3/services/placement` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:19.088032 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e38337-3f3b-4dc4-9c42-288faff34c12 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:19.089439 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e38337-3f3b-4dc4-9c42-288faff34c12 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-f1e38337-3f3b-4dc4-9c42-288faff34c12 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:19.102049 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:19.103233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:19.104683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:19.104683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:54:19.104683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:19.104683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:19.104683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:19.104683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:19.104683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:19.104683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:19.104683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:19.104683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:19.104683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:54:19.104683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:54:19.104683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:54:19.104683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:19.104683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 643/1290] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:54:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:19.112296 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd533857-35d4-4704-9f38-ce60e95e983f None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:19.215322 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd533857-35d4-4704-9f38-ce60e95e983f None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:19.255039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bc7e0-054d-4b31-8feb-014170ae1f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:19.255699 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bc7e0-054d-4b31-8feb-014170ae1f6e None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:19.255984 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bc7e0-054d-4b31-8feb-014170ae1f6e None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:19.256263 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bc7e0-054d-4b31-8feb-014170ae1f6e None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:19.256981 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bc7e0-054d-4b31-8feb-014170ae1f6e None admin] RBAC: Adding query filter params (name=placement) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:19.257288 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bc7e0-054d-4b31-8feb-014170ae1f6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:19.258813 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bc7e0-054d-4b31-8feb-014170ae1f6e None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:19.263844 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 648/1291] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 19:54:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:19.272937 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e38337-3f3b-4dc4-9c42-288faff34c12 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:19.400428 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e38337-3f3b-4dc4-9c42-288faff34c12 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:19.436681 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127e9f1-514c-469b-9638-7d6d51f3d1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:19.437352 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127e9f1-514c-469b-9638-7d6d51f3d1a9 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:19.437642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127e9f1-514c-469b-9638-7d6d51f3d1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:19.437924 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127e9f1-514c-469b-9638-7d6d51f3d1a9 None admin] PATH_INFO: `/v3/services` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:19.438675 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127e9f1-514c-469b-9638-7d6d51f3d1a9 None admin] RBAC: Adding query filter params (type=placement) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:19.438989 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127e9f1-514c-469b-9638-7d6d51f3d1a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:19.440376 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127e9f1-514c-469b-9638-7d6d51f3d1a9 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:19.445233 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 644/1292] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 19:54:19 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:21.659908 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb55c98-787c-4b8c-bd13-558fd9bb14fc None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:21.660796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb55c98-787c-4b8c-bd13-558fd9bb14fc None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:21.660796 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb55c98-787c-4b8c-bd13-558fd9bb14fc None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:21.663010 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 649/1293] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:54:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:54:21.668522 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ebe73-08f8-4769-9746-80924085e549 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:21.668969 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ebe73-08f8-4769-9746-80924085e549 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:21.669067 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ebe73-08f8-4769-9746-80924085e549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:21.687157 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-cb8ebe73-08f8-4769-9746-80924085e549 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:54:21.688351 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-cb8ebe73-08f8-4769-9746-80924085e549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:54:21.722088 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:21.766683 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 645/1294] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:54:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:54:21.790319 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb55c98-787c-4b8c-bd13-558fd9bb14fc None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:21.823537 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:21.905372 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb55c98-787c-4b8c-bd13-558fd9bb14fc None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:21.942851 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526162c9-b7d8-4564-9765-7225b1d793e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:21.943428 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526162c9-b7d8-4564-9765-7225b1d793e6 None admin] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:21.943690 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526162c9-b7d8-4564-9765-7225b1d793e6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:21.943967 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526162c9-b7d8-4564-9765-7225b1d793e6 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:21.944697 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526162c9-b7d8-4564-9765-7225b1d793e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:21.946156 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526162c9-b7d8-4564-9765-7225b1d793e6 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:21.968594 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 650/1295] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 19:54:21 2022] POST /identity/v3/services => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:54:24.203017 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808ac15-78f1-4518-9ee6-98fa9e0b392e None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:24.203585 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808ac15-78f1-4518-9ee6-98fa9e0b392e None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:24.203585 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808ac15-78f1-4518-9ee6-98fa9e0b392e None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:24.205657 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 646/1296] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:54:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:54:24.214650 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404d86a-0e7a-438c-852c-44e56a841981 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:24.215056 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404d86a-0e7a-438c-852c-44e56a841981 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:24.215347 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404d86a-0e7a-438c-852c-44e56a841981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:24.237563 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-9404d86a-0e7a-438c-852c-44e56a841981 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:54:24.238800 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-9404d86a-0e7a-438c-852c-44e56a841981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:54:24.327921 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 651/1297] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:54:24 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:54:24.360557 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808ac15-78f1-4518-9ee6-98fa9e0b392e None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:24.496741 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808ac15-78f1-4518-9ee6-98fa9e0b392e None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:24.533362 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81c8b8-54c4-460f-9cda-c1da1712af80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:24.534162 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81c8b8-54c4-460f-9cda-c1da1712af80 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:24.534481 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81c8b8-54c4-460f-9cda-c1da1712af80 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:24.534760 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81c8b8-54c4-460f-9cda-c1da1712af80 None admin] PATH_INFO: `/v3/services/placement` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:24.535426 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81c8b8-54c4-460f-9cda-c1da1712af80 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:24.536818 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81c8b8-54c4-460f-9cda-c1da1712af80 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-4a81c8b8-54c4-460f-9cda-c1da1712af80 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:24.549514 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:24.550652 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:24.551589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:24.551589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:54:24.551589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:24.551589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:24.551589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:24.551589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:24.551589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:24.551589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:24.551589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:24.551589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:24.551589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:54:24.551589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:54:24.551589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:54:24.551589 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:24.554479 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 647/1298] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:54:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:24.557144 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404d86a-0e7a-438c-852c-44e56a841981 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:24.578828 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:24.666040 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404d86a-0e7a-438c-852c-44e56a841981 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:24.702723 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b18391-a713-4970-b8a8-d06cea362d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:24.703400 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b18391-a713-4970-b8a8-d06cea362d30 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:24.703686 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b18391-a713-4970-b8a8-d06cea362d30 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:24.703967 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b18391-a713-4970-b8a8-d06cea362d30 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:24.704700 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b18391-a713-4970-b8a8-d06cea362d30 None admin] RBAC: Adding query filter params (name=placement) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:24.705002 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b18391-a713-4970-b8a8-d06cea362d30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:24.706414 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b18391-a713-4970-b8a8-d06cea362d30 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:24.712242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 652/1299] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 19:54:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:24.718414 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81c8b8-54c4-460f-9cda-c1da1712af80 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:24.755330 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:24.835707 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81c8b8-54c4-460f-9cda-c1da1712af80 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:24.881333 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a596a99-33d1-4507-b248-f3c2e3b9d30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:24.881987 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a596a99-33d1-4507-b248-f3c2e3b9d30d None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:24.882312 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a596a99-33d1-4507-b248-f3c2e3b9d30d None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:24.882831 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a596a99-33d1-4507-b248-f3c2e3b9d30d None admin] PATH_INFO: `/v3/endpoints` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:24.883405 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a596a99-33d1-4507-b248-f3c2e3b9d30d None admin] RBAC: Adding query filter params (interface=public, service_id=67f7315f7314442d80784c0e9133049a, region_id=RegionOne) {{(pid=150549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:24.883715 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a596a99-33d1-4507-b248-f3c2e3b9d30d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:24.885099 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a596a99-33d1-4507-b248-f3c2e3b9d30d None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:24.890349 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 648/1300] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 19:54:24 2022] GET /identity/v3/endpoints?service_id=67f7315f7314442d80784c0e9133049a&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:24.896473 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b18391-a713-4970-b8a8-d06cea362d30 None admin] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:24.984838 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:25.007447 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b18391-a713-4970-b8a8-d06cea362d30 None admin] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:25.045781 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbd5a5-a60e-402a-9242-544b0ff65304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:25.046784 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbd5a5-a60e-402a-9242-544b0ff65304 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:25.047162 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbd5a5-a60e-402a-9242-544b0ff65304 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:25.047528 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbd5a5-a60e-402a-9242-544b0ff65304 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:25.048579 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbd5a5-a60e-402a-9242-544b0ff65304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:25.050797 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbd5a5-a60e-402a-9242-544b0ff65304 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:25.062719 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 653/1301] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 19:54:24 2022] GET /identity/v3/services => generated 2191 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:54:27.320675 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771da7c-0642-479f-86c0-fffc9a17efb1 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:27.321554 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771da7c-0642-479f-86c0-fffc9a17efb1 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:27.321554 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771da7c-0642-479f-86c0-fffc9a17efb1 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:27.323810 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 649/1302] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:54:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:54:27.329301 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8650ce1-161d-42a5-a22e-cb520b6e7393 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:27.329751 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8650ce1-161d-42a5-a22e-cb520b6e7393 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:27.329848 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8650ce1-161d-42a5-a22e-cb520b6e7393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:27.349321 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-d8650ce1-161d-42a5-a22e-cb520b6e7393 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:54:27.350535 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-d8650ce1-161d-42a5-a22e-cb520b6e7393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:54:27.435236 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 654/1303] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:54:27 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:54:27.463460 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771da7c-0642-479f-86c0-fffc9a17efb1 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:27.575720 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:27.584484 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771da7c-0642-479f-86c0-fffc9a17efb1 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:27.622940 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3f5bc-05c5-4a94-a526-e56fe89c0cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:27.623749 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3f5bc-05c5-4a94-a526-e56fe89c0cd9 None admin] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:27.624035 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3f5bc-05c5-4a94-a526-e56fe89c0cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:27.624309 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3f5bc-05c5-4a94-a526-e56fe89c0cd9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:27.624978 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3f5bc-05c5-4a94-a526-e56fe89c0cd9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:27.626457 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3f5bc-05c5-4a94-a526-e56fe89c0cd9 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application [None req-25f3f5bc-05c5-4a94-a526-e56fe89c0cd9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:54:27.638643 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:54:27.639742 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:54:27.640659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:54:27.640659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application with Lock( May 19 19:54:27.640659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:54:27.640659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self._enter() May 19 19:54:27.640659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:54:27.640659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:54:27.640659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:54:27.640659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application return self.creator() May 19 19:54:27.640659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:54:27.640659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application created_value = creator( May 19 19:54:27.640659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:54:27.640659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:54:27.640659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:54:27.640659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: ERROR keystone.server.flask.application May 19 19:54:27.640659 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 650/1304] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 19:54:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:54:27.646758 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8650ce1-161d-42a5-a22e-cb520b6e7393 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:27.729507 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:27.754920 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8650ce1-161d-42a5-a22e-cb520b6e7393 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:27.791129 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449f109-b32f-4a2c-9224-cee0b3d632d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:27.791795 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449f109-b32f-4a2c-9224-cee0b3d632d6 None admin] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:27.792080 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449f109-b32f-4a2c-9224-cee0b3d632d6 None admin] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:27.792363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449f109-b32f-4a2c-9224-cee0b3d632d6 None admin] PATH_INFO: `/v3/services` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:27.793082 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e449f109-b32f-4a2c-9224-cee0b3d632d6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=150548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:54:27.793398 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e449f109-b32f-4a2c-9224-cee0b3d632d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:27.794846 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e449f109-b32f-4a2c-9224-cee0b3d632d6 None admin] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:27.800185 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 655/1305] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 19:54:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:54:27.806560 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3f5bc-05c5-4a94-a526-e56fe89c0cd9 None admin] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:54:27.913026 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3f5bc-05c5-4a94-a526-e56fe89c0cd9 None admin] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:54:27.952843 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570d500-e4eb-4dc4-a90b-875be93e6607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d8f48fa0c344f1aed458217ad0c20b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:54:27.953513 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570d500-e4eb-4dc4-a90b-875be93e6607 None admin] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:54:27.953797 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570d500-e4eb-4dc4-a90b-875be93e6607 None admin] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:54:27.954073 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570d500-e4eb-4dc4-a90b-875be93e6607 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:54:27.954877 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570d500-e4eb-4dc4-a90b-875be93e6607 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:54:27.956264 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570d500-e4eb-4dc4-a90b-875be93e6607 None admin] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:54:27.979567 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 651/1306] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 19:54:27 2022] POST /identity/v3/endpoints => generated 331 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:54:37.796951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=30 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:54:37.974209 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:33.696536 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e7426-ec83-4556-b986-491007de6973 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:33.697081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e7426-ec83-4556-b986-491007de6973 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:33.697081 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e7426-ec83-4556-b986-491007de6973 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:33.698931 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 656/1307] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:33.705237 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00703c-6d05-4082-9ec0-e4a7f80cb760 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:33.705237 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00703c-6d05-4082-9ec0-e4a7f80cb760 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:33.705237 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00703c-6d05-4082-9ec0-e4a7f80cb760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:33.869088 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-8a00703c-6d05-4082-9ec0-e4a7f80cb760 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:33.870243 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-8a00703c-6d05-4082-9ec0-e4a7f80cb760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:34.042920 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 652/1308] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:55:34.097346 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16d9ce-8f51-4892-b39a-37a47de29067 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:34.097888 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16d9ce-8f51-4892-b39a-37a47de29067 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:34.098406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16d9ce-8f51-4892-b39a-37a47de29067 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:34.102328 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 657/1309] 10.222.0.84 () {54 vars in 964 bytes} [Thu May 19 19:55:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:34.110115 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a227390-319a-4985-bdcb-90784d0b3dcb None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:34.110898 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a227390-319a-4985-bdcb-90784d0b3dcb None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:34.111552 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a227390-319a-4985-bdcb-90784d0b3dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:34.274061 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-7a227390-319a-4985-bdcb-90784d0b3dcb None None] MFA Rules not processed for user `24d916d5c44c4078958734bd59a220b3`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:34.275350 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-7a227390-319a-4985-bdcb-90784d0b3dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:34.313415 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:34.375655 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 653/1310] 10.222.0.84 () {58 vars in 1061 bytes} [Thu May 19 19:55:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:55:34.385227 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16d9ce-8f51-4892-b39a-37a47de29067 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:34.387146 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-1e16d9ce-8f51-4892-b39a-37a47de29067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:34.503087 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16d9ce-8f51-4892-b39a-37a47de29067 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:34.581205 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57585218-b509-4d72-8391-edbd73defa7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d916d5c44c4078958734bd59a220b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:34.581867 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57585218-b509-4d72-8391-edbd73defa7d service neutron] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:34.582149 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57585218-b509-4d72-8391-edbd73defa7d service neutron] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:34.582445 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57585218-b509-4d72-8391-edbd73defa7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:34.584179 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-57585218-b509-4d72-8391-edbd73defa7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:34.640950 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:34.710213 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57585218-b509-4d72-8391-edbd73defa7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:34.711714 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57585218-b509-4d72-8391-edbd73defa7d service neutron] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:34.732393 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 658/1311] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:55:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:55:37.262156 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4a6ae-55c4-4107-93a3-fc76d7025079 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:37.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4a6ae-55c4-4107-93a3-fc76d7025079 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:37.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4a6ae-55c4-4107-93a3-fc76d7025079 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:37.268090 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 654/1312] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:55:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:37.269574 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fc246-b5a9-43ea-aa0b-e4dcef4a0b0b None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:37.269995 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fc246-b5a9-43ea-aa0b-e4dcef4a0b0b None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:37.270612 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fc246-b5a9-43ea-aa0b-e4dcef4a0b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:37.356853 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-1d1fc246-b5a9-43ea-aa0b-e4dcef4a0b0b None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:37.358017 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-1d1fc246-b5a9-43ea-aa0b-e4dcef4a0b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:37.469431 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 659/1313] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:55:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:55:37.507642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c4a6ae-55c4-4107-93a3-fc76d7025079 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:37.509423 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-a2c4a6ae-55c4-4107-93a3-fc76d7025079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:37.619600 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c4a6ae-55c4-4107-93a3-fc76d7025079 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:37.695303 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:37.698476 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b43d77-1c32-4aac-a03e-0666077b479c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d916d5c44c4078958734bd59a220b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:37.699450 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b43d77-1c32-4aac-a03e-0666077b479c service neutron] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:37.699881 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b43d77-1c32-4aac-a03e-0666077b479c service neutron] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:37.700293 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b43d77-1c32-4aac-a03e-0666077b479c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:37.702187 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-64b43d77-1c32-4aac-a03e-0666077b479c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:37.821883 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b43d77-1c32-4aac-a03e-0666077b479c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:37.823642 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b43d77-1c32-4aac-a03e-0666077b479c service neutron] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:37.845036 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 655/1314] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:55:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:55:40.510981 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742f3cc-9beb-4cbb-9368-a58c046f48eb None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:40.511829 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742f3cc-9beb-4cbb-9368-a58c046f48eb None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:40.511829 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742f3cc-9beb-4cbb-9368-a58c046f48eb None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:40.514106 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 660/1315] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:55:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:40.523380 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff0270-a317-4fd7-875e-87117e042fe8 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:40.524135 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff0270-a317-4fd7-875e-87117e042fe8 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:40.524281 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff0270-a317-4fd7-875e-87117e042fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:40.600316 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-14ff0270-a317-4fd7-875e-87117e042fe8 None None] MFA Rules not processed for user `85b48f8334a3449caeae652db3f7ae7f`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:40.601553 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-14ff0270-a317-4fd7-875e-87117e042fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:40.657314 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:40.706286 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 656/1316] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:55:40 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:55:40.749213 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742f3cc-9beb-4cbb-9368-a58c046f48eb None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:40.750846 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-d742f3cc-9beb-4cbb-9368-a58c046f48eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:40.761725 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:40.866741 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742f3cc-9beb-4cbb-9368-a58c046f48eb None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:40.946695 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73709c-14ea-4c65-8c87-d83a2528e91f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d916d5c44c4078958734bd59a220b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:40.947349 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73709c-14ea-4c65-8c87-d83a2528e91f service neutron] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:40.947628 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73709c-14ea-4c65-8c87-d83a2528e91f service neutron] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:40.947906 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73709c-14ea-4c65-8c87-d83a2528e91f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:40.949601 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-4b73709c-14ea-4c65-8c87-d83a2528e91f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:41.013683 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:41.062211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73709c-14ea-4c65-8c87-d83a2528e91f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:41.063695 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73709c-14ea-4c65-8c87-d83a2528e91f service neutron] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:41.082538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 661/1317] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:55:40 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 19:55:43.924617 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca840466-ed46-4b08-89f4-0192b473bd93 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:43.925203 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca840466-ed46-4b08-89f4-0192b473bd93 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:43.925203 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca840466-ed46-4b08-89f4-0192b473bd93 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:43.926671 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 657/1318] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:43.933725 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f613843-a318-4d55-9d6d-4ca2590b0452 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:43.934607 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f613843-a318-4d55-9d6d-4ca2590b0452 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:43.934806 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f613843-a318-4d55-9d6d-4ca2590b0452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:44.020731 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-4f613843-a318-4d55-9d6d-4ca2590b0452 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:44.021911 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-4f613843-a318-4d55-9d6d-4ca2590b0452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:44.134652 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 662/1319] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:55:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:55:44.198846 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e896c20-8b76-4fdd-908b-9e666f574c98 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:44.199236 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e896c20-8b76-4fdd-908b-9e666f574c98 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:44.199512 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e896c20-8b76-4fdd-908b-9e666f574c98 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:44.201717 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 658/1320] 10.222.0.84 () {54 vars in 964 bytes} [Thu May 19 19:55:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:44.208058 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf85a5-0ab1-4580-8336-e781b7d04157 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:44.208378 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf85a5-0ab1-4580-8336-e781b7d04157 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:44.208659 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf85a5-0ab1-4580-8336-e781b7d04157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:44.231746 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:44.286591 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-12cf85a5-0ab1-4580-8336-e781b7d04157 None None] MFA Rules not processed for user `24d916d5c44c4078958734bd59a220b3`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:44.287815 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-12cf85a5-0ab1-4580-8336-e781b7d04157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:44.369491 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 663/1321] 10.222.0.84 () {58 vars in 1061 bytes} [Thu May 19 19:55:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:55:44.378838 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e896c20-8b76-4fdd-908b-9e666f574c98 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:44.380459 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-6e896c20-8b76-4fdd-908b-9e666f574c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:44.508569 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e896c20-8b76-4fdd-908b-9e666f574c98 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:44.592420 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c223a9-909c-4d62-bd3c-338073231cf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d916d5c44c4078958734bd59a220b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:44.593296 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c223a9-909c-4d62-bd3c-338073231cf4 service neutron] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:44.593596 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c223a9-909c-4d62-bd3c-338073231cf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:44.593868 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c223a9-909c-4d62-bd3c-338073231cf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:44.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-a4c223a9-909c-4d62-bd3c-338073231cf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:44.604116 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:44.718882 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c223a9-909c-4d62-bd3c-338073231cf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:44.720383 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c223a9-909c-4d62-bd3c-338073231cf4 service neutron] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:44.738938 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 659/1322] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:55:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:55:48.201585 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c786666-0733-407f-9987-3d9cca4da6c2 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.202141 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c786666-0733-407f-9987-3d9cca4da6c2 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.202141 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c786666-0733-407f-9987-3d9cca4da6c2 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.203582 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 664/1323] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:48.209282 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad111713-5be9-4d10-8ac9-ca41e6d3fa63 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.209730 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad111713-5be9-4d10-8ac9-ca41e6d3fa63 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.209893 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad111713-5be9-4d10-8ac9-ca41e6d3fa63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.280011 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-ad111713-5be9-4d10-8ac9-ca41e6d3fa63 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:48.281262 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-ad111713-5be9-4d10-8ac9-ca41e6d3fa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:48.344317 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:48.388121 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 660/1324] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:55:48.424023 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c786666-0733-407f-9987-3d9cca4da6c2 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:48.425570 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-4c786666-0733-407f-9987-3d9cca4da6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:48.534297 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c786666-0733-407f-9987-3d9cca4da6c2 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:48.546932 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:48.612207 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9086930-e8ef-4c11-87ef-946956e23467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d916d5c44c4078958734bd59a220b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:48.612866 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9086930-e8ef-4c11-87ef-946956e23467 service neutron] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.613148 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9086930-e8ef-4c11-87ef-946956e23467 service neutron] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.613417 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9086930-e8ef-4c11-87ef-946956e23467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.615225 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-f9086930-e8ef-4c11-87ef-946956e23467 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:48.737498 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9086930-e8ef-4c11-87ef-946956e23467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:48.739553 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9086930-e8ef-4c11-87ef-946956e23467 service neutron] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:48.758093 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 665/1325] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:55:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 19:55:51.097164 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20867de-2d29-4f72-b52c-bc4ec246af21 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:51.097862 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20867de-2d29-4f72-b52c-bc4ec246af21 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:51.097862 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20867de-2d29-4f72-b52c-bc4ec246af21 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:51.099344 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 661/1326] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:51.105031 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf42c5d-a7fd-4df1-b798-a8c3eead3d1e None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:51.105465 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf42c5d-a7fd-4df1-b798-a8c3eead3d1e None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:51.105591 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf42c5d-a7fd-4df1-b798-a8c3eead3d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:51.154949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:51.174812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-edf42c5d-a7fd-4df1-b798-a8c3eead3d1e None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:51.176124 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-edf42c5d-a7fd-4df1-b798-a8c3eead3d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:51.279286 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 666/1327] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:55:51.315311 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20867de-2d29-4f72-b52c-bc4ec246af21 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:51.316905 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-d20867de-2d29-4f72-b52c-bc4ec246af21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:51.427426 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20867de-2d29-4f72-b52c-bc4ec246af21 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:51.503512 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f389162-3496-4b91-980c-f345589533e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d916d5c44c4078958734bd59a220b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:51.504213 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f389162-3496-4b91-980c-f345589533e3 service neutron] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:51.504500 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f389162-3496-4b91-980c-f345589533e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:51.504800 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f389162-3496-4b91-980c-f345589533e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:51.506613 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-1f389162-3496-4b91-980c-f345589533e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:51.525691 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:51.620724 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f389162-3496-4b91-980c-f345589533e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:51.622496 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f389162-3496-4b91-980c-f345589533e3 service neutron] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:51.642373 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 662/1328] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:55:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:55:53.905600 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4e726-bee3-497c-ab44-d3698114abe1 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:53.906155 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4e726-bee3-497c-ab44-d3698114abe1 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:53.906155 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4e726-bee3-497c-ab44-d3698114abe1 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:53.908215 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 667/1329] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:55:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:53.914887 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37029e81-eaa0-4f30-b83b-12e42982071a None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:53.915351 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37029e81-eaa0-4f30-b83b-12e42982071a None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:53.915509 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37029e81-eaa0-4f30-b83b-12e42982071a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:53.982633 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-37029e81-eaa0-4f30-b83b-12e42982071a None None] MFA Rules not processed for user `85b48f8334a3449caeae652db3f7ae7f`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:53.983862 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-37029e81-eaa0-4f30-b83b-12e42982071a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:54.051232 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:54.098044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 663/1330] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:55:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:55:54.138287 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4e726-bee3-497c-ab44-d3698114abe1 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:54.139840 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-75e4e726-bee3-497c-ab44-d3698114abe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:54.268967 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4e726-bee3-497c-ab44-d3698114abe1 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:54.290135 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:54.351921 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e6def-dea8-43c7-a165-b42a96c4eeff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d916d5c44c4078958734bd59a220b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:54.352994 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e6def-dea8-43c7-a165-b42a96c4eeff service neutron] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:54.353442 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e6def-dea8-43c7-a165-b42a96c4eeff service neutron] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:54.353882 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e6def-dea8-43c7-a165-b42a96c4eeff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:54.356630 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-331e6def-dea8-43c7-a165-b42a96c4eeff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:54.482211 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e6def-dea8-43c7-a165-b42a96c4eeff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:54.483690 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e6def-dea8-43c7-a165-b42a96c4eeff service neutron] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:54.503178 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 668/1331] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:55:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:55:56.962992 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b93020-2d33-41a5-b993-a9ece1fe4a0c None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:56.963881 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b93020-2d33-41a5-b993-a9ece1fe4a0c None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:56.963881 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b93020-2d33-41a5-b993-a9ece1fe4a0c None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:56.966165 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 664/1332] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:55:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:56.972948 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8b7dc-f2ff-4e67-af86-fd77b21bff12 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:56.973580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8b7dc-f2ff-4e67-af86-fd77b21bff12 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:56.973580 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8b7dc-f2ff-4e67-af86-fd77b21bff12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:57.039873 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:57.043003 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-2fc8b7dc-f2ff-4e67-af86-fd77b21bff12 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:57.044278 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-2fc8b7dc-f2ff-4e67-af86-fd77b21bff12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:57.156253 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 669/1333] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:55:57.195428 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b93020-2d33-41a5-b993-a9ece1fe4a0c None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:57.197841 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-37b93020-2d33-41a5-b993-a9ece1fe4a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:57.338986 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b93020-2d33-41a5-b993-a9ece1fe4a0c None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:57.419016 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef2691-c19b-4c22-9344-e980996348ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d916d5c44c4078958734bd59a220b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:57.419662 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef2691-c19b-4c22-9344-e980996348ee service neutron] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:57.419942 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef2691-c19b-4c22-9344-e980996348ee service neutron] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:57.420217 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef2691-c19b-4c22-9344-e980996348ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:57.421902 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-34ef2691-c19b-4c22-9344-e980996348ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:57.450402 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:57.545889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef2691-c19b-4c22-9344-e980996348ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:57.547401 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef2691-c19b-4c22-9344-e980996348ee service neutron] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:57.568438 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 665/1334] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:55:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:55:59.843539 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b287f-dc1e-4814-844a-ee4ce1cb2738 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:59.844395 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b287f-dc1e-4814-844a-ee4ce1cb2738 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:59.844395 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b287f-dc1e-4814-844a-ee4ce1cb2738 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:59.846479 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 670/1335] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:55:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:59.852535 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c118a7-408d-4cb5-bd4d-c703b0c41cef None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:59.852987 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c118a7-408d-4cb5-bd4d-c703b0c41cef None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:59.853088 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c118a7-408d-4cb5-bd4d-c703b0c41cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:59.918605 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-c7c118a7-408d-4cb5-bd4d-c703b0c41cef None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:59.919830 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-c7c118a7-408d-4cb5-bd4d-c703b0c41cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:59.996459 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:00.019700 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 666/1336] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:56:00.059565 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b287f-dc1e-4814-844a-ee4ce1cb2738 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:00.061144 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-306b287f-dc1e-4814-844a-ee4ce1cb2738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:00.177020 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b287f-dc1e-4814-844a-ee4ce1cb2738 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:00.208991 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:00.260478 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be534c29-1421-47de-9d75-e86566717d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d916d5c44c4078958734bd59a220b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:00.261056 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be534c29-1421-47de-9d75-e86566717d46 service neutron] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:00.261338 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be534c29-1421-47de-9d75-e86566717d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:00.261605 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be534c29-1421-47de-9d75-e86566717d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:00.263302 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-be534c29-1421-47de-9d75-e86566717d46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:00.384812 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be534c29-1421-47de-9d75-e86566717d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:00.386311 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be534c29-1421-47de-9d75-e86566717d46 service neutron] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:00.410039 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 671/1337] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:56:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:56:02.990127 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8358d-e453-4536-80fe-199c5b899e87 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:02.990971 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8358d-e453-4536-80fe-199c5b899e87 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:02.990971 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8358d-e453-4536-80fe-199c5b899e87 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:02.993190 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 667/1338] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:56:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:03.001820 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e6f6f-c72a-4e00-b3b8-ef65bef4881c None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:03.002279 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e6f6f-c72a-4e00-b3b8-ef65bef4881c None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:03.002458 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e6f6f-c72a-4e00-b3b8-ef65bef4881c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:03.085949 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-274e6f6f-c72a-4e00-b3b8-ef65bef4881c None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:03.087236 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-274e6f6f-c72a-4e00-b3b8-ef65bef4881c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:03.096095 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:03.192473 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 672/1339] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:56:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:56:03.230661 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8358d-e453-4536-80fe-199c5b899e87 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:03.232337 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-a0f8358d-e453-4536-80fe-199c5b899e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:03.337211 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8358d-e453-4536-80fe-199c5b899e87 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:03.431580 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20278e-ac32-459b-9efe-4754ed5d7c9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d916d5c44c4078958734bd59a220b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:03.432250 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20278e-ac32-459b-9efe-4754ed5d7c9f service neutron] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:03.432537 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20278e-ac32-459b-9efe-4754ed5d7c9f service neutron] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:03.432815 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20278e-ac32-459b-9efe-4754ed5d7c9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:03.434546 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-dd20278e-ac32-459b-9efe-4754ed5d7c9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:03.474048 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:03.557106 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20278e-ac32-459b-9efe-4754ed5d7c9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:03.558599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20278e-ac32-459b-9efe-4754ed5d7c9f service neutron] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:03.580889 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 668/1340] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:56:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 19:56:08.046948 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01681fbc-5581-4eee-8134-c0bbdb2c1eae None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:08.047526 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01681fbc-5581-4eee-8134-c0bbdb2c1eae None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:08.047526 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01681fbc-5581-4eee-8134-c0bbdb2c1eae None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:08.048877 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 673/1341] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:08.054839 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fc57c-0a6e-4ae0-8cc0-cbf43e937b5a None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:08.055275 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fc57c-0a6e-4ae0-8cc0-cbf43e937b5a None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:08.055399 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fc57c-0a6e-4ae0-8cc0-cbf43e937b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:08.137215 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-075fc57c-0a6e-4ae0-8cc0-cbf43e937b5a None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:08.138370 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-075fc57c-0a6e-4ae0-8cc0-cbf43e937b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:08.255821 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=100 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:08.271593 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 669/1342] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 19:56:08.314806 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01681fbc-5581-4eee-8134-c0bbdb2c1eae None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:08.317752 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-01681fbc-5581-4eee-8134-c0bbdb2c1eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:08.451363 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01681fbc-5581-4eee-8134-c0bbdb2c1eae None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:08.455440 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e558c-fc46-4dc4-ba32-d6ed26774f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d916d5c44c4078958734bd59a220b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:08.456109 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e558c-fc46-4dc4-ba32-d6ed26774f04 service neutron] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:08.456390 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e558c-fc46-4dc4-ba32-d6ed26774f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:08.456674 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e558c-fc46-4dc4-ba32-d6ed26774f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:08.468054 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e558c-fc46-4dc4-ba32-d6ed26774f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:08.469538 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e558c-fc46-4dc4-ba32-d6ed26774f04 service neutron] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:08.480273 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 674/1343] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:56:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:56:11.391719 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2ad16-cbd1-4abb-8da8-b36f41bd8a0c None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:11.392299 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2ad16-cbd1-4abb-8da8-b36f41bd8a0c None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:11.392299 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2ad16-cbd1-4abb-8da8-b36f41bd8a0c None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:11.393635 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 670/1344] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:11.399021 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386a4d1-12a2-4203-9967-5decc2ed5cea None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:11.399472 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386a4d1-12a2-4203-9967-5decc2ed5cea None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:11.399560 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386a4d1-12a2-4203-9967-5decc2ed5cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:11.428277 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-3386a4d1-12a2-4203-9967-5decc2ed5cea None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:11.429480 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-3386a4d1-12a2-4203-9967-5decc2ed5cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:11.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:11.508242 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 675/1345] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:56:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:56:11.544722 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2ad16-cbd1-4abb-8da8-b36f41bd8a0c None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:11.556083 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2ad16-cbd1-4abb-8da8-b36f41bd8a0c None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:11.559879 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18d3cf-0800-4c40-ae60-e8bad019f47a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d916d5c44c4078958734bd59a220b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:11.560519 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18d3cf-0800-4c40-ae60-e8bad019f47a service neutron] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:11.560801 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18d3cf-0800-4c40-ae60-e8bad019f47a service neutron] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:11.561072 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18d3cf-0800-4c40-ae60-e8bad019f47a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:11.570605 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18d3cf-0800-4c40-ae60-e8bad019f47a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:11.572058 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18d3cf-0800-4c40-ae60-e8bad019f47a service neutron] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:11.581682 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 671/1346] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:56:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:56:15.893917 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d72a9-1f37-4427-8fe3-6ae223899d91 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:15.894479 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d72a9-1f37-4427-8fe3-6ae223899d91 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:15.894577 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d72a9-1f37-4427-8fe3-6ae223899d91 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:15.896267 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 676/1347] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:15.901364 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08c7bf-74e3-457e-8b9a-56b32725ab79 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:15.901813 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08c7bf-74e3-457e-8b9a-56b32725ab79 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:15.901906 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08c7bf-74e3-457e-8b9a-56b32725ab79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:15.919721 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-ce08c7bf-74e3-457e-8b9a-56b32725ab79 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:15.920926 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-ce08c7bf-74e3-457e-8b9a-56b32725ab79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:15.929305 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 672/1348] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:56:15.968801 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d72a9-1f37-4427-8fe3-6ae223899d91 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:15.981579 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d72a9-1f37-4427-8fe3-6ae223899d91 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:15.985843 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490fd5f-e6e0-455e-af93-a8d66883a901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d916d5c44c4078958734bd59a220b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:15.986652 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490fd5f-e6e0-455e-af93-a8d66883a901 service neutron] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:15.986935 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490fd5f-e6e0-455e-af93-a8d66883a901 service neutron] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:15.987215 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490fd5f-e6e0-455e-af93-a8d66883a901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:16.000951 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490fd5f-e6e0-455e-af93-a8d66883a901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:16.002969 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490fd5f-e6e0-455e-af93-a8d66883a901 service neutron] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:16.026504 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 677/1349] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:56:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:56:21.419745 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f477fe-0838-4b6a-9aa6-eee5923db752 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:21.420304 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f477fe-0838-4b6a-9aa6-eee5923db752 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:21.420547 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f477fe-0838-4b6a-9aa6-eee5923db752 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:21.422319 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 673/1350] 10.222.0.84 () {54 vars in 921 bytes} [Thu May 19 19:56:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:21.427642 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb89ce-9499-4c67-93e2-b5828901894c None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:21.428068 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb89ce-9499-4c67-93e2-b5828901894c None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:21.428407 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb89ce-9499-4c67-93e2-b5828901894c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:21.500195 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-1fdb89ce-9499-4c67-93e2-b5828901894c None None] MFA Rules not processed for user `14b4b91113ed40f0bd0d2bcc6818138c`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:21.501599 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-1fdb89ce-9499-4c67-93e2-b5828901894c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:21.580570 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 678/1351] 10.222.0.84 () {58 vars in 1018 bytes} [Thu May 19 19:56:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:56:25.914395 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=13 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:25.924269 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92875e6-3222-49f6-9830-bdd2a3edb580 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:25.924599 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92875e6-3222-49f6-9830-bdd2a3edb580 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:25.924962 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92875e6-3222-49f6-9830-bdd2a3edb580 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:25.926523 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 674/1352] 10.222.0.84 () {54 vars in 921 bytes} [Thu May 19 19:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:25.931482 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb507d-2138-4ba8-8d09-4dca036c1352 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:25.932345 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb507d-2138-4ba8-8d09-4dca036c1352 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:25.932826 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb507d-2138-4ba8-8d09-4dca036c1352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:25.952172 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-f9cb507d-2138-4ba8-8d09-4dca036c1352 None None] MFA Rules not processed for user `14b4b91113ed40f0bd0d2bcc6818138c`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:25.953364 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-f9cb507d-2138-4ba8-8d09-4dca036c1352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:25.962739 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 679/1353] 10.222.0.84 () {58 vars in 1018 bytes} [Thu May 19 19:56:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:56:27.647408 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460802b-985a-491a-ac3b-b555770f7d05 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:27.647961 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460802b-985a-491a-ac3b-b555770f7d05 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:27.647961 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460802b-985a-491a-ac3b-b555770f7d05 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:27.649290 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 675/1354] 10.222.0.84 () {54 vars in 921 bytes} [Thu May 19 19:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:27.654553 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626eba4-f005-47ee-90ef-d8d00b2ef0db None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:27.654946 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626eba4-f005-47ee-90ef-d8d00b2ef0db None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:27.655111 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626eba4-f005-47ee-90ef-d8d00b2ef0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:27.674417 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-7626eba4-f005-47ee-90ef-d8d00b2ef0db None None] MFA Rules not processed for user `14b4b91113ed40f0bd0d2bcc6818138c`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:27.675676 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-7626eba4-f005-47ee-90ef-d8d00b2ef0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:27.684720 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 680/1355] 10.222.0.84 () {58 vars in 1018 bytes} [Thu May 19 19:56:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:56:31.458638 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6cb78-22c1-4e6b-8d8b-780170522808 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:31.459227 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6cb78-22c1-4e6b-8d8b-780170522808 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:31.459487 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6cb78-22c1-4e6b-8d8b-780170522808 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:31.461419 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 676/1356] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:56:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:31.466406 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973df7e3-42cb-490c-a1b1-e4f2f37fe239 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:31.466832 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973df7e3-42cb-490c-a1b1-e4f2f37fe239 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:31.466961 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973df7e3-42cb-490c-a1b1-e4f2f37fe239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:31.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-973df7e3-42cb-490c-a1b1-e4f2f37fe239 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:31.486844 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-973df7e3-42cb-490c-a1b1-e4f2f37fe239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:31.497029 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 681/1357] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:56:31.551663 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabbb9f-8d82-4430-89c4-3440cc279eb5 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:31.552130 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabbb9f-8d82-4430-89c4-3440cc279eb5 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:31.553148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabbb9f-8d82-4430-89c4-3440cc279eb5 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:31.555708 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 677/1358] 10.222.0.84 () {54 vars in 961 bytes} [Thu May 19 19:56:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:31.564804 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df7f0a-24fb-4ff5-9bd7-4d3a609dbd48 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:31.566150 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df7f0a-24fb-4ff5-9bd7-4d3a609dbd48 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:31.567052 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df7f0a-24fb-4ff5-9bd7-4d3a609dbd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:31.615145 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=100 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:31.624294 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-08df7f0a-24fb-4ff5-9bd7-4d3a609dbd48 None None] MFA Rules not processed for user `0da7f490d9604107b974e9e461ebc38b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:31.625547 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-08df7f0a-24fb-4ff5-9bd7-4d3a609dbd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:31.723043 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 682/1359] 10.222.0.84 () {58 vars in 1058 bytes} [Thu May 19 19:56:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 19:56:31.738795 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabbb9f-8d82-4430-89c4-3440cc279eb5 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:31.757379 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabbb9f-8d82-4430-89c4-3440cc279eb5 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:31.763148 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2551e-c5ab-4620-a7b6-e6d4b8ef408a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0da7f490d9604107b974e9e461ebc38b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:31.763992 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2551e-c5ab-4620-a7b6-e6d4b8ef408a service nova] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:31.764284 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2551e-c5ab-4620-a7b6-e6d4b8ef408a service nova] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:31.764565 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2551e-c5ab-4620-a7b6-e6d4b8ef408a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:31.773570 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2551e-c5ab-4620-a7b6-e6d4b8ef408a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:31.775230 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2551e-c5ab-4620-a7b6-e6d4b8ef408a service nova] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:31.785314 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 678/1360] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:56:34.209500 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca0d7e-70d3-4d5f-8b2e-c778fa363c78 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:34.210027 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca0d7e-70d3-4d5f-8b2e-c778fa363c78 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:34.210027 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca0d7e-70d3-4d5f-8b2e-c778fa363c78 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:34.211514 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 683/1361] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:34.217074 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06ebab-0db4-45ca-88e9-fe1fbf1d5269 None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:34.217793 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06ebab-0db4-45ca-88e9-fe1fbf1d5269 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:34.217993 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06ebab-0db4-45ca-88e9-fe1fbf1d5269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:34.237042 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-bd06ebab-0db4-45ca-88e9-fe1fbf1d5269 None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:34.238280 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-bd06ebab-0db4-45ca-88e9-fe1fbf1d5269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:34.247360 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 679/1362] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:56:34.292979 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca0d7e-70d3-4d5f-8b2e-c778fa363c78 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:34.308178 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca0d7e-70d3-4d5f-8b2e-c778fa363c78 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:34.313629 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed7cca-df26-47c0-8a0c-36872d0e4099 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0da7f490d9604107b974e9e461ebc38b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:34.314284 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed7cca-df26-47c0-8a0c-36872d0e4099 service nova] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:34.314585 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed7cca-df26-47c0-8a0c-36872d0e4099 service nova] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:34.314879 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed7cca-df26-47c0-8a0c-36872d0e4099 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:34.323872 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed7cca-df26-47c0-8a0c-36872d0e4099 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:34.325520 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed7cca-df26-47c0-8a0c-36872d0e4099 service nova] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:34.335776 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 684/1363] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:56:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:56:36.892856 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8399b-5dde-40dc-a07d-3c46b023c5c2 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:36.893700 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8399b-5dde-40dc-a07d-3c46b023c5c2 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:36.893700 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8399b-5dde-40dc-a07d-3c46b023c5c2 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:36.894879 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 680/1364] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:36.900520 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae2c64-6360-477a-adcb-a25232d9ccad None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:36.900977 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae2c64-6360-477a-adcb-a25232d9ccad None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:36.901055 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae2c64-6360-477a-adcb-a25232d9ccad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:36.919782 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-ecae2c64-6360-477a-adcb-a25232d9ccad None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:36.920998 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-ecae2c64-6360-477a-adcb-a25232d9ccad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:36.929504 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 685/1365] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:56:36.976543 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d7eb2-f962-422d-9dc6-0cfeb4b75884 None None] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:36.976892 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d7eb2-f962-422d-9dc6-0cfeb4b75884 None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:36.977273 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d7eb2-f962-422d-9dc6-0cfeb4b75884 None None] PATH_INFO: `/` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:36.979137 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 681/1366] 10.222.0.84 () {54 vars in 961 bytes} [Thu May 19 19:56:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:36.984442 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1da11-fa9a-49ab-82b6-478d458899b4 None None] REQUEST_METHOD: `POST` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:36.984711 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1da11-fa9a-49ab-82b6-478d458899b4 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:36.985034 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1da11-fa9a-49ab-82b6-478d458899b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:37.004361 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.auth.core [None req-52b1da11-fa9a-49ab-82b6-478d458899b4 None None] MFA Rules not processed for user `0da7f490d9604107b974e9e461ebc38b`. Rule list: `[]` (Enabled: `True`). {{(pid=150548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:37.005561 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.fernet_utils [None req-52b1da11-fa9a-49ab-82b6-478d458899b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:37.015983 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 686/1367] 10.222.0.84 () {58 vars in 1058 bytes} [Thu May 19 19:56:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 19:56:37.024044 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d7eb2-f962-422d-9dc6-0cfeb4b75884 None None] Authenticating user token {{(pid=150549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:37.055313 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d7eb2-f962-422d-9dc6-0cfeb4b75884 None None] Validating token access rules against request {{(pid=150549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:37.064469 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a331e-5655-4b5e-b9c6-47660ab19894 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0da7f490d9604107b974e9e461ebc38b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:37.065194 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a331e-5655-4b5e-b9c6-47660ab19894 service nova] REQUEST_METHOD: `GET` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:37.065496 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a331e-5655-4b5e-b9c6-47660ab19894 service nova] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:37.065771 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a331e-5655-4b5e-b9c6-47660ab19894 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:37.075762 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a331e-5655-4b5e-b9c6-47660ab19894 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:37.077382 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a331e-5655-4b5e-b9c6-47660ab19894 service nova] RBAC: Authorization granted {{(pid=150549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:37.087738 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 682/1368] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:56:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:56:37.909314 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728a382-4733-4d96-9ebf-6d91768842c2 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:37.909945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728a382-4733-4d96-9ebf-6d91768842c2 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:37.909945 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728a382-4733-4d96-9ebf-6d91768842c2 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:37.911320 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 687/1369] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:37.917399 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81169430-73dd-4dec-a1f9-08d3b9167d6f None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:37.917942 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81169430-73dd-4dec-a1f9-08d3b9167d6f None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:37.917942 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81169430-73dd-4dec-a1f9-08d3b9167d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:37.937455 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-81169430-73dd-4dec-a1f9-08d3b9167d6f None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:37.938718 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-81169430-73dd-4dec-a1f9-08d3b9167d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:37.947264 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 683/1370] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:56:39.395865 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185fa83-2d8f-456d-9f99-a79e0a717a73 None None] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:39.396425 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185fa83-2d8f-456d-9f99-a79e0a717a73 None None] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:39.396425 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185fa83-2d8f-456d-9f99-a79e0a717a73 None None] PATH_INFO: `/` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:39.397806 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 688/1371] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 19:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:39.403251 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59aec58-4c71-47e5-9a86-a35aa556201c None None] REQUEST_METHOD: `POST` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:39.403680 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59aec58-4c71-47e5-9a86-a35aa556201c None None] SCRIPT_NAME: `/identity` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:39.403847 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59aec58-4c71-47e5-9a86-a35aa556201c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:39.425836 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.auth.core [None req-a59aec58-4c71-47e5-9a86-a35aa556201c None None] MFA Rules not processed for user `15d8f48fa0c344f1aed458217ad0c20b`. Rule list: `[]` (Enabled: `True`). {{(pid=150549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:39.427745 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG keystone.common.fernet_utils [None req-a59aec58-4c71-47e5-9a86-a35aa556201c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:39.446325 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: [pid: 150549|app: 0|req: 684/1372] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 19:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:56:39.491534 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185fa83-2d8f-456d-9f99-a79e0a717a73 None None] Authenticating user token {{(pid=150548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:39.507724 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185fa83-2d8f-456d-9f99-a79e0a717a73 None None] Validating token access rules against request {{(pid=150548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:39.513640 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580b3bd-9da7-4cc9-9a5b-c207057664c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0da7f490d9604107b974e9e461ebc38b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd638a29a793d41e7bfa686c0fcc29149', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:39.514308 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580b3bd-9da7-4cc9-9a5b-c207057664c9 service nova] REQUEST_METHOD: `GET` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:39.514596 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580b3bd-9da7-4cc9-9a5b-c207057664c9 service nova] SCRIPT_NAME: `/identity` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:39.514875 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580b3bd-9da7-4cc9-9a5b-c207057664c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=150548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:39.523323 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580b3bd-9da7-4cc9-9a5b-c207057664c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:39.524927 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580b3bd-9da7-4cc9-9a5b-c207057664c9 service nova] RBAC: Authorization granted {{(pid=150548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:39.534647 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: [pid: 150548|app: 0|req: 689/1373] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 19:56:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:56:49.418657 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: DEBUG dbcounter [-] [150549] Writing DB stats keystone:SELECT=21 {{(pid=150549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:49.530231 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: DEBUG dbcounter [-] [150548] Writing DB stats keystone:SELECT=54 {{(pid=150548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:01.811512 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: Thu May 19 19:57:01 2022 - graceful shutdown triggered... May 19 19:57:01.811661 dv-cinder-835881-13-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 19:57:01.812287 dv-cinder-835881-13-iscsi devstack@keystone.service[150548]: Gracefully killing worker 1 (pid: 150548)... May 19 19:57:01.812726 dv-cinder-835881-13-iscsi devstack@keystone.service[150549]: Gracefully killing worker 2 (pid: 150549)... May 19 19:57:02.813438 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: worker 1 buried after 1 seconds May 19 19:57:02.813438 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: worker 2 buried after 1 seconds May 19 19:57:02.813438 dv-cinder-835881-13-iscsi devstack@keystone.service[150536]: goodbye to uWSGI. May 19 19:57:02.816416 dv-cinder-835881-13-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 19:57:02.817296 dv-cinder-835881-13-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 19 20:03:00.134235 dv-cinder-835881-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 20:03:00.142692 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 20:03:00.144669 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 20:03:00 2022] *** May 19 20:03:00.144669 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 20:03:00.144669 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 20:03:00.144669 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: nodename: dv-cinder-835881-13-iscsi May 19 20:03:00.144669 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: machine: x86_64 May 19 20:03:00.144669 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: clock source: unix May 19 20:03:00.144669 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: pcre jit disabled May 19 20:03:00.144669 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: detected number of CPU cores: 4 May 19 20:03:00.144669 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: current working directory: / May 19 20:03:00.144669 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: detected binary path: /usr/bin/uwsgi-core May 19 20:03:00.144669 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: your processes number limit is 47680 May 19 20:03:00.144669 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: your memory page size is 4096 bytes May 19 20:03:00.144669 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: detected max file descriptor number: 2048 May 19 20:03:00.144669 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: lock engine: pthread robust mutexes May 19 20:03:00.154704 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: thunder lock: enabled May 19 20:03:00.154704 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 20:03:00.154704 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 20:03:00.190786 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: Python main interpreter initialized at 0x5625a2e8e9c0 May 19 20:03:00.190786 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: python threads support enabled May 19 20:03:00.190786 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: your server socket listen backlog is limited to 100 connections May 19 20:03:00.190786 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: your mercy for graceful operations on workers is 90 seconds May 19 20:03:00.191162 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: mapped 403077 bytes (393 KB) for 2 cores May 19 20:03:00.191242 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: *** Operational MODE: preforking *** May 19 20:03:00.191475 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: *** uWSGI is running in multiple interpreter mode *** May 19 20:03:00.191475 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: spawned uWSGI master process (pid: 214300) May 19 20:03:00.191691 dv-cinder-835881-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 20:03:00.194670 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: spawned uWSGI worker 1 (pid: 214314, cores: 1) May 19 20:03:00.194670 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: spawned uWSGI worker 2 (pid: 214315, cores: 1) May 19 20:03:00.194670 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 20:03:05.824329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:03:05.824329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: from cryptography.utils import int_from_bytes May 19 20:03:05.825212 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:03:05.825212 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: from cryptography.utils import int_from_bytes May 19 20:03:05.825679 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:03:05.825679 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: from cryptography.utils import int_from_bytes May 19 20:03:05.826210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:03:05.826210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: from cryptography.utils import int_from_bytes May 19 20:03:05.874133 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 20:03:05.874385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 20:03:05.874485 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] command line args: [] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 20:03:05.874773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 20:03:05.874920 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ================================================================================ {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 20:03:05.875021 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 20:03:05.875174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 20:03:05.875346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] admin_token = **** {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.875431 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] command line args: [] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 20:03:05.875507 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] config_dir = [] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.875587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.875670 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 20:03:05.875744 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] config_source = [] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.875817 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ================================================================================ {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 20:03:05.875891 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] debug = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.875975 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] admin_token = **** {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.876055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.876413 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] config_dir = [] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.876413 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.876556 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.876629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] config_source = [] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.876701 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] insecure_debug = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.876779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] debug = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.876853 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.876930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.877035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.877205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.877299 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] list_limit = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.877379 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] insecure_debug = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.877456 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] log_config_append = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.877529 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.877601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.877737 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] log_dir = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.877828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.877903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] log_file = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.878018 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] list_limit = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.878103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.878178 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] log_config_append = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.878396 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.878544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.878640 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] log_dir = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.878717 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.878791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] log_file = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.878864 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.878941 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.879021 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.879207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.879304 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.879378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.879452 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.879658 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.879817 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.879817 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.879986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.880061 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.880136 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.880213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.880287 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.880520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.880677 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.880779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.880867 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.880964 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.881049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] notification_format = cadf {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.881147 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.881231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.881313 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.881533 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.881684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.84/identity {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.881799 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] notification_format = cadf {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.881882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] publish_errors = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.882043 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.882204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.882403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.882530 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.882707 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.84/identity {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.882863 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.883026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] publish_errors = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.883152 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.883308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.883464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] standard_threads = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.883593 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.883593 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.883849 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] strict_password_check = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.884005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.884130 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.884130 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] use_eventlog = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.884382 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.884511 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] use_journal = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.884600 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] standard_threads = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.884729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] use_json = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.884729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] use_stderr = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.884897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] strict_password_check = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.884987 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] use_syslog = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.885108 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.885198 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] watch_log_file = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.885291 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] use_eventlog = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.885291 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] use_journal = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.885445 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.885531 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] use_json = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.885624 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.885711 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] use_stderr = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.885711 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] use_syslog = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.886032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.886130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] watch_log_file = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:03:05.886218 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.886316 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.886419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.886502 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.886582 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.886670 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.886763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.886763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.886931 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.887019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.887146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.887306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.887477 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.887579 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.887675 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.887928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.888032 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.888122 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.888204 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.888291 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.888381 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.888469 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.888557 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.888651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.888747 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.888851 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.888947 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.889035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.889116 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.889202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.889289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.889377 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.889469 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.889557 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.889644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.889770 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.889899 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.889993 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.890088 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.890178 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.890289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.890384 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.890468 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.890548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.890640 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.890804 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.890952 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.891051 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.891136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.891223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.891326 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.891455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.891555 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.891722 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.891872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.891974 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.892067 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.892157 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.892245 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.892373 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.892479 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.892611 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.892718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.892866 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.893002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.893134 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.893266 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.893401 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.893533 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.893658 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.893789 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.893914 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.894045 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.894182 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.894343 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.894477 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.894607 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.894738 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.894869 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.895012 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.895153 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.895285 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.895416 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.895546 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.895678 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.895814 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.895948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.896072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.896202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.896326 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.896464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.896601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.896733 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.896857 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.896988 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.897119 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.897251 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.897386 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.897517 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.897647 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.897779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.897910 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.898036 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.898177 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.898328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.898463 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.898595 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.898725 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.898864 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.899001 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.899134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.899257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.899386 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.899517 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.899648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.899784 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.899908 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.900031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.900166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.900302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.900425 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.900551 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.900681 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.900811 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.900946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.901077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.901209 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.901339 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.901475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.901606 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.901730 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.901853 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.901983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.902123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.902255 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.902471 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.902604 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.902735 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.902953 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.903095 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.903228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.903354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.903484 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.903615 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.enabled = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.903745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.903879 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.904010 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.904143 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.904276 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.904408 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.904540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.904676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.904807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.enabled = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.904940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.905080 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.905211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.905361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.905485 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.905632 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.905767 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.906029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.906166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.906310 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.906447 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.906580 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.906711 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.906835 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.906972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.907104 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.907234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.907377 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.907511 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.907641 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.907778 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.907910 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.908045 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.908176 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.908308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.908439 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.908646 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.908788 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.908919 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.909051 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.909051 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.909290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.909290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.909544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.909677 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.909807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.909931 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.910053 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.910188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.910365 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.910499 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.910642 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.910767 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.910897 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.911034 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.911165 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.911301 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.911431 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] auth.external = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.911561 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.911686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] auth.mapped = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.911818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.911942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.912067 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.912197 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.912327 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.912460 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] auth.password = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.912602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] auth.external = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.912740 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] auth.token = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.912866 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] auth.mapped = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.912997 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.912997 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] catalog.caching = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.913244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.913376 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.913512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.913643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.913773 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] auth.password = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.913906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.914040 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] auth.token = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.914177 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.914329 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.914463 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.914588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] catalog.caching = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.914718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] credential.caching = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.914851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.914981 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] credential.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.915118 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.915252 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.915384 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.915529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.915668 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.915794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.915944 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.916076 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.916206 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] credential.caching = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.916336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.916461 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] credential.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.916591 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.916725 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.916849 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.916980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.917105 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.917235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.917366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.917510 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.917510 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.917743 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.917743 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.917996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.918126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.918297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.918462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.918599 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.918731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.918989 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.919132 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] federation.caching = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.919271 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.919402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.919540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.919671 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] federation.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.919802 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.919925 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.920057 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.920189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.920319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.920452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.920591 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] federation.caching = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.920716 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.920848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.920972 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.921095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] federation.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.921231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.921362 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.921499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.921624 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.921755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.921927 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.922061 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.922187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.922339 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity.caching = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.922467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.922591 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.922722 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.922858 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.922997 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.923131 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.923257 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.923386 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.923510 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.923633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.923756 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity.caching = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.923891 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.924028 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.924159 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.924290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.924420 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.924551 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.924693 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.924827 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.924951 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.924951 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.925197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.925327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.925465 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.925596 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.925720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.925850 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.925981 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.926103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.926240 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.926411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.926534 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.926534 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.926779 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.926909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.927047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.927179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.927302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.927435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.927565 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.927696 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.927829 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.927829 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.928080 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.928204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.928333 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.928457 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.928589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.928720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.928856 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.928980 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.929109 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.929235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.929366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.929490 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.929622 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.929753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.929876 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.930005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.930138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.930286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.930422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.930553 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.930686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.930819 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.930954 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.931084 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.931207 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.931330 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.931458 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.931664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.password = **** {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.931798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.932013 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.932150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.932281 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.932412 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.932549 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.932674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.932796 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.932926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.933049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.933171 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.933303 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.933427 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.password = **** {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.933557 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.933681 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.933811 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.933942 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.934291 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.934508 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.934731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.934859 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.934990 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.935122 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.935256 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.935397 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.935530 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.935661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.935785 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.935916 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.936058 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.936184 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.936319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.936450 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.936574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.936704 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.936840 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.936973 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.937108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.937239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.937369 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.937499 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.937642 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.937773 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.937905 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.938028 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.938170 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.938331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.938471 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.938602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.938733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.938856 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.938994 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.939124 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.939258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.939382 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.939511 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.939642 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.939764 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.939887 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.940024 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.940148 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.940278 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.940409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.940532 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.940661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.940797 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.940927 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.941058 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.941189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.941319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.941442 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.941574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.941697 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.941826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.941950 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.942072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.942195 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.942385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.942519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.942643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.942766 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.942889 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.943012 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.943149 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] policy.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.943274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.943404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.943527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.943649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.943779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.943915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.944039 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.944161 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] receipt.caching = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.944285 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.944407 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.944543 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.944674 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.944809 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] policy.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.944934 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.945057 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.945179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.945316 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.945447 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.945571 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.945693 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] resource.caching = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.945816 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] receipt.caching = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.945938 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.946069 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.946192 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] resource.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.946329 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.946465 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.946590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.946728 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.946866 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.947028 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.947169 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.947300 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] revoke.caching = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.947425 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] resource.caching = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.947555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.947688 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.947819 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.947947 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] resource.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.948071 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] role.cache_time = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.948201 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.948331 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] role.caching = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.948465 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.948595 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] role.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.948726 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.948850 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] role.list_limit = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.948979 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] revoke.caching = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.949118 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.949252 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.949382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.949508 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.949669 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.949819 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] role.cache_time = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.949950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.950088 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] role.caching = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.950220 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.950385 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] role.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.950518 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.950653 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] role.list_limit = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.950784 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.950921 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.951051 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.951175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.951298 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.951427 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.951563 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.951700 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.951824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.951948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.952077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.952214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.952351 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.952489 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.952615 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.952738 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.952860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.952982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.953104 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.953240 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.953364 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.953487 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.953609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.953740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.953881 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.954005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.954128 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.954273 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.954393 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.954529 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.954668 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.954800 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.954923 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.955052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.955183 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.955307 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.955440 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.955440 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.955679 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.955679 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.955926 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.955926 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.956174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.956298 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.956421 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.956550 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.956673 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.956796 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.956994 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.957126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] token.cache_time = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.957249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.957371 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] token.caching = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.957493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.957623 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.957761 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.957891 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] token.provider = fernet {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.958014 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.958144 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.958288 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.958413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.958550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.958690 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.958826 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.958956 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.959079 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] token.cache_time = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.959207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.959343 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] token.caching = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.959466 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.959589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.959711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] trust.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.959839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] token.provider = fernet {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.959970 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.960106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.960229 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.960356 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.960485 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.960630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.960771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.960904 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.961035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.961159 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.961288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.961411 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.961539 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.961671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] trust.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.961794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.961923 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.962059 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.connection = **** {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.962192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.962387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.962514 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.962643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.962768 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.962900 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.963031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.963155 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.963278 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.963407 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.963532 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.963663 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.963787 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.963917 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.964047 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.connection = **** {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.964179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.964312 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.964443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.964573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.964696 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.964826 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.964963 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.965093 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.965217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.965397 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.965523 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.965653 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.965787 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.965918 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.966042 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.966165 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.966305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.966442 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.966580 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.966705 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.966828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.966828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=214314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 20:03:05.967156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.967156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.967156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.967156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.967156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.967156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.967156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.967156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:03:05.967156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=214315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 20:03:05.988166 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:05.988395 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:05.988620 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:05.988710 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:05.988886 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:05.989116 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:05.997377 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:05.998021 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.025481 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.025670 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.026302 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.026529 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.078376 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.078540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.129807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.130014 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.130108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.130193 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.194820 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.195044 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.195149 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.195234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.195302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.195421 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.195517 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.195636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.195731 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.195875 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.196021 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.196107 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.222465 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.222751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.222889 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.222991 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.223128 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.223240 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.223324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.223410 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.223572 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.223716 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.223813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.223910 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.223999 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.224176 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.235159 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.235555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.249782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.249983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.250223 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.250383 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.250486 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.250583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.250672 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.250756 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.250844 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.250929 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.251049 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=214315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.251157 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.251341 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.251529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=214314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:03:06.268322 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.268486 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:03:06.268852 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.268997 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.269089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:03:06.269218 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:03:06.269324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.269562 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.269707 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.269707 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:03:06.269882 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.270059 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.270203 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.270346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.270588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.270837 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.281213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.281490 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.281745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.282019 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.282164 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.282320 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.282615 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.282891 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.290116 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.290903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.293582 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.293725 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.294115 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.294387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.294492 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.294582 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.294678 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.294799 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.295088 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.295234 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.295328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.295418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.295612 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.295767 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.295858 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.295951 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.296141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.296298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.296390 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.296495 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.296659 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.296924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.322925 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.323079 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.324538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.324685 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.324911 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.325057 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.330158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.330380 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.330671 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.330815 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.331041 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.331323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.337845 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.338122 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.338580 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.338862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.342864 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.343150 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.343438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.343583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.343668 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.343885 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.344031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.344432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.344531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.344747 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.344900 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.345001 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.345106 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.345262 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.345555 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.345820 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.363413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.363675 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.364577 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.364857 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.368233 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.369389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.371906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.372196 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.373064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.373356 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.377176 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.378299 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.380738 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.381896 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.384544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.384875 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.385192 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.385508 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.385701 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.385816 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.386021 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.386168 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.386290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.386592 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.386871 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.387163 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.404983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.405353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.405614 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.405884 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.406078 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.406479 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.406751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.407025 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.413984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.414974 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.415717 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.415988 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.416685 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.416965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.419309 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.419593 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.419888 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.420045 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.420141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.420337 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.420442 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.420622 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.420916 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.421192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.433625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.433886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.434326 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.434633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.434793 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.434894 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.435066 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.435203 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.435325 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.435447 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.435639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.435923 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.436209 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.436489 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.453427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.454528 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.456659 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.457779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.460143 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.461114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.461718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.462744 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.463382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.463665 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.463947 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.464396 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.464682 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.464967 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.471801 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.472813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.473398 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.474460 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.476697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.476985 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.477266 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.477535 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.477690 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.477957 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.478251 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.478554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.486853 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.487193 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.487475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.487940 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.488277 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.488561 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.495279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=214314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.495550 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.495832 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.496109 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.496390 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.496677 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=214315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:03:06.496793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.496877 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.496975 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.497248 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.497347 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.497434 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.497525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.497614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.497794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.497906 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.497994 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.498198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.498387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.498489 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.498582 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.498778 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.498885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=214314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.498976 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.499274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.499535 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.499814 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.500077 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=214315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:03:06.537464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5625a2e8e9c0 pid: 214314 (default app) May 19 20:03:06.541052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0eb03-9214-4f8c-951b-f762bd1ebe78 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:06.541319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0eb03-9214-4f8c-951b-f762bd1ebe78 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:06.541623 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0eb03-9214-4f8c-951b-f762bd1ebe78 None None] PATH_INFO: `/v3` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:06.543781 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5625a2e8e9c0 pid: 214315 (default app) May 19 20:03:06.546901 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 1/1] 10.222.0.84 () {50 vars in 791 bytes} [Thu May 19 20:03:06 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:14.640151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826f60a-d1f9-4730-af50-e1483f3f645f None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:14.640996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826f60a-d1f9-4730-af50-e1483f3f645f None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:14.640996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826f60a-d1f9-4730-af50-e1483f3f645f None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:14.647691 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 1/2] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:14 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:14.656330 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343961b-0eaf-4dca-afcb-92c7a3055ae4 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:14.656924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343961b-0eaf-4dca-afcb-92c7a3055ae4 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:14.656924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343961b-0eaf-4dca-afcb-92c7a3055ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:14.738956 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: INFO dbcounter [None req-b343961b-0eaf-4dca-afcb-92c7a3055ae4 None None] Registered counter for database keystone May 19 20:03:14.767131 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG oslo_db.sqlalchemy.engines [None req-b343961b-0eaf-4dca-afcb-92c7a3055ae4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=214314) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 20:03:14.778312 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writer thread running {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 20:03:14.992409 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG passlib.handlers.bcrypt [None req-b343961b-0eaf-4dca-afcb-92c7a3055ae4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=214314) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 20:03:14.992707 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG passlib.handlers.bcrypt [None req-b343961b-0eaf-4dca-afcb-92c7a3055ae4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=214314) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 20:03:15.053566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-b343961b-0eaf-4dca-afcb-92c7a3055ae4 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:15.055239 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-b343961b-0eaf-4dca-afcb-92c7a3055ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:15.210147 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 2/3] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 555 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 20:03:15.242324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826f60a-d1f9-4730-af50-e1483f3f645f None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:15.313013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: INFO dbcounter [None req-8826f60a-d1f9-4730-af50-e1483f3f645f None None] Registered counter for database keystone May 19 20:03:15.335551 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG oslo_db.sqlalchemy.engines [None req-8826f60a-d1f9-4730-af50-e1483f3f645f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=214315) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 20:03:15.347824 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writer thread running {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 20:03:15.477054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826f60a-d1f9-4730-af50-e1483f3f645f None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:15.482645 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a140ff-7c94-4af7-a7f9-fefbd97773f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026346139f294a8c8380faacc93b3be0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:15.483314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a140ff-7c94-4af7-a7f9-fefbd97773f6 admin admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:15.483597 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a140ff-7c94-4af7-a7f9-fefbd97773f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:15.483870 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a140ff-7c94-4af7-a7f9-fefbd97773f6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:15.492545 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a140ff-7c94-4af7-a7f9-fefbd97773f6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:15.544025 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: WARNING keystone.common.rbac_enforcer.enforcer [None req-95a140ff-7c94-4af7-a7f9-fefbd97773f6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 20:03:15.544372 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a140ff-7c94-4af7-a7f9-fefbd97773f6 admin admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:15.555495 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 2/4] 10.222.0.84 () {58 vars in 1324 bytes} [Thu May 19 20:03:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 20:03:15.561871 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343961b-0eaf-4dca-afcb-92c7a3055ae4 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:15.577750 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343961b-0eaf-4dca-afcb-92c7a3055ae4 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:15.584342 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249b09e-ff2a-4ba1-9f15-bd6deb1a578b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026346139f294a8c8380faacc93b3be0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:15.585488 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249b09e-ff2a-4ba1-9f15-bd6deb1a578b admin admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:15.585748 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249b09e-ff2a-4ba1-9f15-bd6deb1a578b admin admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:15.586058 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249b09e-ff2a-4ba1-9f15-bd6deb1a578b admin admin] PATH_INFO: `/v3/projects/026346139f294a8c8380faacc93b3be0` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:15.587989 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249b09e-ff2a-4ba1-9f15-bd6deb1a578b admin admin] RBAC: Authorizing `identity:get_project(project_id=026346139f294a8c8380faacc93b3be0)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:15.750945 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4249b09e-ff2a-4ba1-9f15-bd6deb1a578b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 20:03:15.751642 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249b09e-ff2a-4ba1-9f15-bd6deb1a578b admin admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:15.756077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 3/5] 10.222.0.84 () {56 vars in 1207 bytes} [Thu May 19 20:03:15 2022] GET /identity/v3/projects/026346139f294a8c8380faacc93b3be0 => generated 344 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:18.012258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db1b12-f50e-456f-8835-c254bce103a2 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:18.012785 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db1b12-f50e-456f-8835-c254bce103a2 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:18.012935 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db1b12-f50e-456f-8835-c254bce103a2 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:18.017236 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 3/6] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:03:18.024121 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d0577-31a2-4e13-ae71-41f7906633e8 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:18.024868 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d0577-31a2-4e13-ae71-41f7906633e8 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:18.025041 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d0577-31a2-4e13-ae71-41f7906633e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:18.052768 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-d06d0577-31a2-4e13-ae71-41f7906633e8 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:18.054295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-d06d0577-31a2-4e13-ae71-41f7906633e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:18.064232 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 4/7] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 20:03:18.087476 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db1b12-f50e-456f-8835-c254bce103a2 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:18.107059 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db1b12-f50e-456f-8835-c254bce103a2 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:18.113466 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40bc10-adfe-4e1a-bb36-29cb1d650313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026346139f294a8c8380faacc93b3be0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:18.114119 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40bc10-adfe-4e1a-bb36-29cb1d650313 admin admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:18.114445 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40bc10-adfe-4e1a-bb36-29cb1d650313 admin admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:18.114724 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40bc10-adfe-4e1a-bb36-29cb1d650313 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:18.123210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40bc10-adfe-4e1a-bb36-29cb1d650313 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:18.125025 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40bc10-adfe-4e1a-bb36-29cb1d650313 admin admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:18.134106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 4/8] 10.222.0.84 () {58 vars in 1324 bytes} [Thu May 19 20:03:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 20:03:18.140696 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d0577-31a2-4e13-ae71-41f7906633e8 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:18.154858 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d0577-31a2-4e13-ae71-41f7906633e8 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:18.160581 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac4cb7-c9b6-4a1c-9410-6166bce54f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026346139f294a8c8380faacc93b3be0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:18.161456 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac4cb7-c9b6-4a1c-9410-6166bce54f5a admin admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:18.161742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac4cb7-c9b6-4a1c-9410-6166bce54f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:18.162021 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac4cb7-c9b6-4a1c-9410-6166bce54f5a admin admin] PATH_INFO: `/v3/users/def9ab1b20e24b65870b4a851d7650b7` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:18.163606 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac4cb7-c9b6-4a1c-9410-6166bce54f5a admin admin] RBAC: Authorizing `identity:get_user(user_id=def9ab1b20e24b65870b4a851d7650b7)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:18.165483 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac4cb7-c9b6-4a1c-9410-6166bce54f5a admin admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:18.167100 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 5/9] 10.222.0.84 () {56 vars in 1198 bytes} [Thu May 19 20:03:18 2022] GET /identity/v3/users/def9ab1b20e24b65870b4a851d7650b7 => generated 247 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:20.433031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd62a7-7ddd-4ddc-9e16-9dd01170bfc4 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:20.433617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd62a7-7ddd-4ddc-9e16-9dd01170bfc4 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:20.433617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd62a7-7ddd-4ddc-9e16-9dd01170bfc4 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:20.434991 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 5/10] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:20.440562 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fab13e-f15e-4029-aef3-e87cb50a866f None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:20.441055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fab13e-f15e-4029-aef3-e87cb50a866f None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:20.441254 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fab13e-f15e-4029-aef3-e87cb50a866f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:20.459098 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-c4fab13e-f15e-4029-aef3-e87cb50a866f None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:20.460498 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-c4fab13e-f15e-4029-aef3-e87cb50a866f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:20.481106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b790a-2db9-47af-886d-2e49ee944780 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:20.481420 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b790a-2db9-47af-886d-2e49ee944780 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:20.481700 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b790a-2db9-47af-886d-2e49ee944780 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:20.483088 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 6/11] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:03:20.488576 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d6010-595a-4eb3-a8d0-72e084aeef07 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:20.488862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d6010-595a-4eb3-a8d0-72e084aeef07 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:20.489135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d6010-595a-4eb3-a8d0-72e084aeef07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:20.529381 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG passlib.handlers.bcrypt [None req-5c2d6010-595a-4eb3-a8d0-72e084aeef07 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=214315) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 20:03:20.529791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG passlib.handlers.bcrypt [None req-5c2d6010-595a-4eb3-a8d0-72e084aeef07 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=214315) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 20:03:20.548022 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 6/12] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:20.571510 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fab13e-f15e-4029-aef3-e87cb50a866f None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:20.576105 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-5c2d6010-595a-4eb3-a8d0-72e084aeef07 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:20.577725 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-5c2d6010-595a-4eb3-a8d0-72e084aeef07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:20.596392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:20.684643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 7/13] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:03:20.694437 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fab13e-f15e-4029-aef3-e87cb50a866f None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:20.707017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d6010-595a-4eb3-a8d0-72e084aeef07 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:20.732623 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20554c-35bb-449c-8931-5bb39b2d32d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:20.733460 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20554c-35bb-449c-8931-5bb39b2d32d2 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:20.733741 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20554c-35bb-449c-8931-5bb39b2d32d2 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:20.734021 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20554c-35bb-449c-8931-5bb39b2d32d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:20.748310 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20554c-35bb-449c-8931-5bb39b2d32d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-3a20554c-35bb-449c-8931-5bb39b2d32d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:20.760055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:20.761424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:20.762559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:20.762559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:20.762559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:20.762559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:20.762559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:20.762559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:20.762559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:20.762559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:20.762559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:20.762559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:20.762559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:20.762559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:03:20.762559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 7/14] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:03:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:03:20.769823 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20554c-35bb-449c-8931-5bb39b2d32d2 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:20.860839 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:20.880205 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20554c-35bb-449c-8931-5bb39b2d32d2 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:20.915015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eec087-0c2f-4738-8ede-e682a3401fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:20.915952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eec087-0c2f-4738-8ede-e682a3401fad None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:20.916233 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eec087-0c2f-4738-8ede-e682a3401fad None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:20.916510 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eec087-0c2f-4738-8ede-e682a3401fad None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:20.917263 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eec087-0c2f-4738-8ede-e682a3401fad None admin] RBAC: Adding query filter params (name=admin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:20.917567 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eec087-0c2f-4738-8ede-e682a3401fad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:20.919128 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eec087-0c2f-4738-8ede-e682a3401fad None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:20.921295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d6010-595a-4eb3-a8d0-72e084aeef07 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:20.923732 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:20.930975 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 8/15] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:20.936774 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eec087-0c2f-4738-8ede-e682a3401fad None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:20.961684 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e92ec0-95e7-423e-ae22-308f7e87b204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:20.962717 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e92ec0-95e7-423e-ae22-308f7e87b204 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:20.963088 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e92ec0-95e7-423e-ae22-308f7e87b204 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:20.963376 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e92ec0-95e7-423e-ae22-308f7e87b204 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:21.045760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e92ec0-95e7-423e-ae22-308f7e87b204 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:21.047337 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e92ec0-95e7-423e-ae22-308f7e87b204 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:21.050360 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eec087-0c2f-4738-8ede-e682a3401fad None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:21.084477 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b6045-8d18-4bbf-9d02-2deb70d0b159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:21.085709 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b6045-8d18-4bbf-9d02-2deb70d0b159 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:21.085991 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b6045-8d18-4bbf-9d02-2deb70d0b159 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:21.086286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b6045-8d18-4bbf-9d02-2deb70d0b159 None admin] PATH_INFO: `/v3/users/def9ab1b20e24b65870b4a851d7650b7` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:21.087535 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b6045-8d18-4bbf-9d02-2deb70d0b159 None admin] RBAC: Authorizing `identity:get_user(user_id=def9ab1b20e24b65870b4a851d7650b7)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:21.089019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b6045-8d18-4bbf-9d02-2deb70d0b159 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:21.091222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 9/16] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:20 2022] GET /identity/v3/users/def9ab1b20e24b65870b4a851d7650b7 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:21.096956 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b6045-8d18-4bbf-9d02-2deb70d0b159 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:21.111636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:21.158585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 8/17] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:03:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:03:21.164293 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e92ec0-95e7-423e-ae22-308f7e87b204 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:21.166973 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:21.200469 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b6045-8d18-4bbf-9d02-2deb70d0b159 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:21.236480 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af8bb4-2d72-4de3-a7de-04163228daa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:21.237284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af8bb4-2d72-4de3-a7de-04163228daa6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:21.237566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af8bb4-2d72-4de3-a7de-04163228daa6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:21.237841 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af8bb4-2d72-4de3-a7de-04163228daa6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:21.239059 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af8bb4-2d72-4de3-a7de-04163228daa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:21.240551 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af8bb4-2d72-4de3-a7de-04163228daa6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:21.242176 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 10/18] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:03:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:21.247863 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af8bb4-2d72-4de3-a7de-04163228daa6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:21.266958 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e92ec0-95e7-423e-ae22-308f7e87b204 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:21.303362 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a0c72-ab74-405a-8002-d4e5ad4159f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:21.304173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a0c72-ab74-405a-8002-d4e5ad4159f5 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:21.304460 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a0c72-ab74-405a-8002-d4e5ad4159f5 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:21.304742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a0c72-ab74-405a-8002-d4e5ad4159f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:21.324240 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a0c72-ab74-405a-8002-d4e5ad4159f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:21.326442 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a0c72-ab74-405a-8002-d4e5ad4159f5 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:21.331628 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:21.348174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-386a0c72-ab74-405a-8002-d4e5ad4159f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:03:21.348174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:21.348174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:21.348174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:21.348174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:21.348174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:21.348174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:21.348174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:21.348174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:21.348174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:21.348174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:21.348174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:21.348174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:03:21.348174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:03:21.348174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:21.349361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:21.350635 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:03:21.350635 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:03:21.350635 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:03:21.350635 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:21.350635 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 9/19] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:21.353957 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af8bb4-2d72-4de3-a7de-04163228daa6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:21.355322 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a0c72-ab74-405a-8002-d4e5ad4159f5 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:21.389160 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286eda01-c643-4e3f-82fe-13fc9d015321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:21.389845 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286eda01-c643-4e3f-82fe-13fc9d015321 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:21.390128 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286eda01-c643-4e3f-82fe-13fc9d015321 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:21.390447 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286eda01-c643-4e3f-82fe-13fc9d015321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:21.391285 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286eda01-c643-4e3f-82fe-13fc9d015321 None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.domain.id=default, user.id=def9ab1b20e24b65870b4a851d7650b7) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:21.391596 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286eda01-c643-4e3f-82fe-13fc9d015321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:21.393105 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286eda01-c643-4e3f-82fe-13fc9d015321 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:21.400018 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 11/20] 10.222.0.84 () {56 vars in 1322 bytes} [Thu May 19 20:03:21 2022] GET /identity/v3/role_assignments?user.id=def9ab1b20e24b65870b4a851d7650b7&scope.domain.id=default&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:21.418125 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:21.469614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a0c72-ab74-405a-8002-d4e5ad4159f5 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:21.509641 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37aa8cb-f3c5-430b-ac0c-6157f47e60e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:21.510515 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37aa8cb-f3c5-430b-ac0c-6157f47e60e9 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:21.510678 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37aa8cb-f3c5-430b-ac0c-6157f47e60e9 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:21.510913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37aa8cb-f3c5-430b-ac0c-6157f47e60e9 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:21.511666 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37aa8cb-f3c5-430b-ac0c-6157f47e60e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:21.512109 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37aa8cb-f3c5-430b-ac0c-6157f47e60e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:21.515078 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37aa8cb-f3c5-430b-ac0c-6157f47e60e9 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:21.535487 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 10/21] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:24.124463 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19035e6-6e3c-413d-8aa7-1a06cd778bb7 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:24.125518 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19035e6-6e3c-413d-8aa7-1a06cd778bb7 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:24.126014 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19035e6-6e3c-413d-8aa7-1a06cd778bb7 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:24.127964 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 12/22] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:24.133473 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d6a1d-7b34-416c-a2ab-c57aa326ab93 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:24.134526 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d6a1d-7b34-416c-a2ab-c57aa326ab93 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:24.135045 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d6a1d-7b34-416c-a2ab-c57aa326ab93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:24.159593 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-f19d6a1d-7b34-416c-a2ab-c57aa326ab93 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:24.161277 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-f19d6a1d-7b34-416c-a2ab-c57aa326ab93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:24.257625 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 11/23] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:24.291959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19035e6-6e3c-413d-8aa7-1a06cd778bb7 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:24.370045 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d16da7-1c28-4182-aacc-2ce6b34ab8b0 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:24.370751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d16da7-1c28-4182-aacc-2ce6b34ab8b0 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:24.371414 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d16da7-1c28-4182-aacc-2ce6b34ab8b0 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:24.374725 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 12/24] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:24.384116 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba04d4-4f2b-4298-80b5-20c0ad8e1ada None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:24.384775 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba04d4-4f2b-4298-80b5-20c0ad8e1ada None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:24.385261 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba04d4-4f2b-4298-80b5-20c0ad8e1ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:24.427357 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-f0ba04d4-4f2b-4298-80b5-20c0ad8e1ada None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:24.428969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-f0ba04d4-4f2b-4298-80b5-20c0ad8e1ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:24.470967 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19035e6-6e3c-413d-8aa7-1a06cd778bb7 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:24.510618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:24.544203 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0262bfc-41ae-48e9-a6b1-f33677247ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:24.545043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0262bfc-41ae-48e9-a6b1-f33677247ec2 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:24.545325 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0262bfc-41ae-48e9-a6b1-f33677247ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:24.545600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0262bfc-41ae-48e9-a6b1-f33677247ec2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:24.562314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 13/25] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:24.563685 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37531c1-7e84-450c-9a1e-a708009b3ec2 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:24.579611 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37531c1-7e84-450c-9a1e-a708009b3ec2 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:24.579904 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37531c1-7e84-450c-9a1e-a708009b3ec2 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:24.581771 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 14/26] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:24 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:24.582376 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0262bfc-41ae-48e9-a6b1-f33677247ec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-c0262bfc-41ae-48e9-a6b1-f33677247ec2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:24.594649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:24.595554 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:24.596382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:24.596382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:24.596382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:24.596382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:24.596382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:24.596382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:24.596382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:24.596382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:24.596382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:24.596382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:24.596382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:24.596382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:03:24.596382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 13/27] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:03:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:24.601443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab924d65-a4d0-42eb-90be-f1e10119c508 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:24.601732 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab924d65-a4d0-42eb-90be-f1e10119c508 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:24.602011 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab924d65-a4d0-42eb-90be-f1e10119c508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:24.608150 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0262bfc-41ae-48e9-a6b1-f33677247ec2 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:24.623652 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:24.639239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-ab924d65-a4d0-42eb-90be-f1e10119c508 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:24.640377 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-ab924d65-a4d0-42eb-90be-f1e10119c508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:24.757359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0262bfc-41ae-48e9-a6b1-f33677247ec2 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:24.795937 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266fcba-dd94-4011-b2ca-aa4c38e30b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:24.796620 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266fcba-dd94-4011-b2ca-aa4c38e30b5c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:24.796905 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266fcba-dd94-4011-b2ca-aa4c38e30b5c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:24.797179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266fcba-dd94-4011-b2ca-aa4c38e30b5c None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:24.797936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266fcba-dd94-4011-b2ca-aa4c38e30b5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:24.798243 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266fcba-dd94-4011-b2ca-aa4c38e30b5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:24.800072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266fcba-dd94-4011-b2ca-aa4c38e30b5c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:24.803185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 15/28] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:24.804045 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab924d65-a4d0-42eb-90be-f1e10119c508 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:24.825344 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 14/29] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:24.832584 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266fcba-dd94-4011-b2ca-aa4c38e30b5c None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:24.929310 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:24.956008 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab924d65-a4d0-42eb-90be-f1e10119c508 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:24.964795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:24.967877 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266fcba-dd94-4011-b2ca-aa4c38e30b5c None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:24.995852 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29b13d-8e6a-44ff-ac71-049986599fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:24.997773 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29b13d-8e6a-44ff-ac71-049986599fc4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:24.998329 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29b13d-8e6a-44ff-ac71-049986599fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:24.998759 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29b13d-8e6a-44ff-ac71-049986599fc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:25.006108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbae1e-0b61-48a1-95ab-41dc9bc9f1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:25.007250 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbae1e-0b61-48a1-95ab-41dc9bc9f1e6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:25.007722 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbae1e-0b61-48a1-95ab-41dc9bc9f1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:25.008193 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbae1e-0b61-48a1-95ab-41dc9bc9f1e6 None admin] PATH_INFO: `/v3/users/def9ab1b20e24b65870b4a851d7650b7` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:25.009574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbae1e-0b61-48a1-95ab-41dc9bc9f1e6 None admin] RBAC: Authorizing `identity:get_user(user_id=def9ab1b20e24b65870b4a851d7650b7)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:25.009959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29b13d-8e6a-44ff-ac71-049986599fc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:25.011457 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbae1e-0b61-48a1-95ab-41dc9bc9f1e6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:25.011593 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29b13d-8e6a-44ff-ac71-049986599fc4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:25.013668 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 15/30] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:24 2022] GET /identity/v3/users/def9ab1b20e24b65870b4a851d7650b7 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:25.015200 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ba56f-22f7-42a0-94a5-dea3f914edcb None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:25.015700 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ba56f-22f7-42a0-94a5-dea3f914edcb None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:25.016132 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ba56f-22f7-42a0-94a5-dea3f914edcb None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:25.017506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 16/31] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:25.018620 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ba56f-22f7-42a0-94a5-dea3f914edcb None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:25.046483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-0a29b13d-8e6a-44ff-ac71-049986599fc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:03:25.046483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:25.046483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:25.046483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:25.046483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:25.046483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:25.046483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:25.046483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:25.046483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:25.046483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:25.046483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:25.046483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:25.046483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:03:25.046483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:03:25.046483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:25.047687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:25.048815 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:03:25.048815 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:03:25.048815 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:03:25.048815 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:25.048815 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 16/32] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:25.048815 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29b13d-8e6a-44ff-ac71-049986599fc4 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:25.149309 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ba56f-22f7-42a0-94a5-dea3f914edcb None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:25.160891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29b13d-8e6a-44ff-ac71-049986599fc4 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:25.189377 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea601808-8266-4b8b-bf58-b43b7a2518bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:25.190612 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea601808-8266-4b8b-bf58-b43b7a2518bf None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:25.191048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea601808-8266-4b8b-bf58-b43b7a2518bf None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:25.191751 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea601808-8266-4b8b-bf58-b43b7a2518bf None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:25.193293 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea601808-8266-4b8b-bf58-b43b7a2518bf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:25.195022 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea601808-8266-4b8b-bf58-b43b7a2518bf None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:25.197769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:25.197959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07749f0-0f20-4b2f-af8b-501092134264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:25.199072 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07749f0-0f20-4b2f-af8b-501092134264 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:25.199603 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07749f0-0f20-4b2f-af8b-501092134264 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:25.200110 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07749f0-0f20-4b2f-af8b-501092134264 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:25.202874 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07749f0-0f20-4b2f-af8b-501092134264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:25.205019 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07749f0-0f20-4b2f-af8b-501092134264 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:25.208298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 17/33] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:03:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:25.209948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b446a51-db2a-44e7-979d-6321818676dc None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:25.210590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b446a51-db2a-44e7-979d-6321818676dc None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:25.211085 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b446a51-db2a-44e7-979d-6321818676dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:25.231385 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-5b446a51-db2a-44e7-979d-6321818676dc None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:25.233020 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-5b446a51-db2a-44e7-979d-6321818676dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:25.256431 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:25.312389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 18/34] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:25.313242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b446a51-db2a-44e7-979d-6321818676dc None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:25.336852 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 17/35] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 20:03:25 2022] POST /identity/v3/roles => generated 225 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:03:25.338139 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea601808-8266-4b8b-bf58-b43b7a2518bf None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:25.447238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b446a51-db2a-44e7-979d-6321818676dc None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:25.466984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea601808-8266-4b8b-bf58-b43b7a2518bf None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:25.490103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0282a0-92ac-4e9a-89a8-2a0ecec2b48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:25.491305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0282a0-92ac-4e9a-89a8-2a0ecec2b48d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:25.491879 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0282a0-92ac-4e9a-89a8-2a0ecec2b48d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:25.492381 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0282a0-92ac-4e9a-89a8-2a0ecec2b48d None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:25.493344 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0282a0-92ac-4e9a-89a8-2a0ecec2b48d None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:25.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0282a0-92ac-4e9a-89a8-2a0ecec2b48d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:25.495661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0282a0-92ac-4e9a-89a8-2a0ecec2b48d None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:25.509092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 19/36] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:25.510405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0282a0-92ac-4e9a-89a8-2a0ecec2b48d None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:25.518461 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604b158-4dd9-4280-9625-e35d154a8b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:25.519440 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604b158-4dd9-4280-9625-e35d154a8b87 None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:25.519732 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604b158-4dd9-4280-9625-e35d154a8b87 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:25.520018 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604b158-4dd9-4280-9625-e35d154a8b87 None admin] PATH_INFO: `/v3/domains/default/users/def9ab1b20e24b65870b4a851d7650b7/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:25.522900 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a604b158-4dd9-4280-9625-e35d154a8b87 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=def9ab1b20e24b65870b4a851d7650b7, role_id=b34e4ad4fe7b4a60bd2fc669e2b31a66)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:25.525058 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a604b158-4dd9-4280-9625-e35d154a8b87 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:25.539770 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:25.559881 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 18/37] 10.222.0.84 () {58 vars in 1361 bytes} [Thu May 19 20:03:25 2022] PUT /identity/v3/domains/default/users/def9ab1b20e24b65870b4a851d7650b7/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:03:25.561156 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604b158-4dd9-4280-9625-e35d154a8b87 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:25.677710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:25.711964 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0282a0-92ac-4e9a-89a8-2a0ecec2b48d None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:25.741544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604b158-4dd9-4280-9625-e35d154a8b87 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:25.757558 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb25d03-eb7d-48be-8011-3a45519adeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:25.758602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb25d03-eb7d-48be-8011-3a45519adeca None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:25.758892 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb25d03-eb7d-48be-8011-3a45519adeca None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:25.759177 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb25d03-eb7d-48be-8011-3a45519adeca None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:25.760299 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb25d03-eb7d-48be-8011-3a45519adeca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:25.763296 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb25d03-eb7d-48be-8011-3a45519adeca None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:25.792440 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae80819-dcc0-4be8-84fc-470986485329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:25.793666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae80819-dcc0-4be8-84fc-470986485329 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:25.793666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae80819-dcc0-4be8-84fc-470986485329 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:25.793906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae80819-dcc0-4be8-84fc-470986485329 None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:25.795142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae80819-dcc0-4be8-84fc-470986485329 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:25.795978 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae80819-dcc0-4be8-84fc-470986485329 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:25.828213 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 19/38] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:03:25 2022] POST /identity/v3/projects => generated 301 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:03:25.828213 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add71ab-ba19-4be0-8973-c93ec1b9993c None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:25.828213 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add71ab-ba19-4be0-8973-c93ec1b9993c None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:25.828213 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add71ab-ba19-4be0-8973-c93ec1b9993c None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:25.828213 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 20/39] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:25.828213 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6b391-ae7f-4267-b024-e19dca90c055 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:25.828213 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6b391-ae7f-4267-b024-e19dca90c055 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:25.838907 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6b391-ae7f-4267-b024-e19dca90c055 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:25.838907 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 21/40] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:25.842806 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0ccff-1d11-402b-bd5a-4426943bbb8d None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:25.843125 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0ccff-1d11-402b-bd5a-4426943bbb8d None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:25.843373 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0ccff-1d11-402b-bd5a-4426943bbb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:25.864764 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-0ca0ccff-1d11-402b-bd5a-4426943bbb8d None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:25.866459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-0ca0ccff-1d11-402b-bd5a-4426943bbb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:25.902608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 20/41] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 20:03:25 2022] POST /identity/v3/roles => generated 219 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:03:25.904914 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204213b-d371-4319-bb5c-bef833d937f1 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:25.905212 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204213b-d371-4319-bb5c-bef833d937f1 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:25.905502 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204213b-d371-4319-bb5c-bef833d937f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:25.954017 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 22/42] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:25.959800 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-5204213b-d371-4319-bb5c-bef833d937f1 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:25.961506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-5204213b-d371-4319-bb5c-bef833d937f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:26.002713 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0ccff-1d11-402b-bd5a-4426943bbb8d None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:26.072052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:26.079612 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 21/43] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:26.119123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204213b-d371-4319-bb5c-bef833d937f1 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:26.132756 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:26.165431 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0ccff-1d11-402b-bd5a-4426943bbb8d None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:26.226409 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746a71f-f009-40ed-ba5a-0c2383e55688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:26.228161 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746a71f-f009-40ed-ba5a-0c2383e55688 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:26.228651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746a71f-f009-40ed-ba5a-0c2383e55688 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:26.229111 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746a71f-f009-40ed-ba5a-0c2383e55688 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:26.230062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746a71f-f009-40ed-ba5a-0c2383e55688 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:26.235997 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746a71f-f009-40ed-ba5a-0c2383e55688 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:26.251827 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 23/44] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 20:03:26 2022] POST /identity/v3/roles => generated 223 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:03:26.263819 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204213b-d371-4319-bb5c-bef833d937f1 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:26.301610 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ebec0-90bd-41af-9b1d-3f337f31d94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:26.302464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ebec0-90bd-41af-9b1d-3f337f31d94d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:26.302750 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ebec0-90bd-41af-9b1d-3f337f31d94d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:26.303027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ebec0-90bd-41af-9b1d-3f337f31d94d None admin] PATH_INFO: `/v3/domains/default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:26.304239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ebec0-90bd-41af-9b1d-3f337f31d94d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:26.305983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ebec0-90bd-41af-9b1d-3f337f31d94d None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:26.307918 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 22/45] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:03:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:26.313656 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746a71f-f009-40ed-ba5a-0c2383e55688 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:26.419390 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:26.427986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746a71f-f009-40ed-ba5a-0c2383e55688 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:26.467494 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325904c-2239-4fd5-b03e-3aade36c9758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:26.468171 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325904c-2239-4fd5-b03e-3aade36c9758 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:26.468455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325904c-2239-4fd5-b03e-3aade36c9758 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:26.468735 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325904c-2239-4fd5-b03e-3aade36c9758 None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:26.469796 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325904c-2239-4fd5-b03e-3aade36c9758 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:26.471311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325904c-2239-4fd5-b03e-3aade36c9758 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:26.501024 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 24/46] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:03:26 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:03:27.904012 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7fd6b-99f4-4951-ac82-aaa5dbc3d6eb None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:27.904560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7fd6b-99f4-4951-ac82-aaa5dbc3d6eb None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:27.904560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7fd6b-99f4-4951-ac82-aaa5dbc3d6eb None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:27.905882 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 23/47] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:27.911453 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed74d0b-45db-47b8-966f-841867d9bba4 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:27.911883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed74d0b-45db-47b8-966f-841867d9bba4 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:27.911989 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed74d0b-45db-47b8-966f-841867d9bba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:27.928672 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-1ed74d0b-45db-47b8-966f-841867d9bba4 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:27.929853 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-1ed74d0b-45db-47b8-966f-841867d9bba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:27.999539 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 25/48] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:28.022987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7fd6b-99f4-4951-ac82-aaa5dbc3d6eb None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:28.104421 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:28.130649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7fd6b-99f4-4951-ac82-aaa5dbc3d6eb None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:28.166157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ffe27a-bdac-4d0d-a010-70185786e92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:28.167360 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffe27a-bdac-4d0d-a010-70185786e92c None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:28.167648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffe27a-bdac-4d0d-a010-70185786e92c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:28.167931 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffe27a-bdac-4d0d-a010-70185786e92c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:28.186437 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ffe27a-bdac-4d0d-a010-70185786e92c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-89ffe27a-bdac-4d0d-a010-70185786e92c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:28.195311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:28.196235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:28.197083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:28.197083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:28.197083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:28.197083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:28.197083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:28.197083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:28.197083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:28.197083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:28.197083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:28.197083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:28.197083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:28.197083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:28.197083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 24/49] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:03:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:28.205198 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed74d0b-45db-47b8-966f-841867d9bba4 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:28.291713 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:28.315972 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed74d0b-45db-47b8-966f-841867d9bba4 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:28.349944 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8b9d6-dd1c-471c-b227-e9ebd4718ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:28.350666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8b9d6-dd1c-471c-b227-e9ebd4718ec6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:28.350950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8b9d6-dd1c-471c-b227-e9ebd4718ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:28.351227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8b9d6-dd1c-471c-b227-e9ebd4718ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:28.351996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8b9d6-dd1c-471c-b227-e9ebd4718ec6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:28.352310 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8b9d6-dd1c-471c-b227-e9ebd4718ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:28.353736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8b9d6-dd1c-471c-b227-e9ebd4718ec6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:28.362561 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 26/50] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:28.368132 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ffe27a-bdac-4d0d-a010-70185786e92c None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:28.476644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ffe27a-bdac-4d0d-a010-70185786e92c None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:28.509076 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9408859-cf2b-44b1-bb25-756f60519204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:28.509995 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9408859-cf2b-44b1-bb25-756f60519204 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:28.510286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9408859-cf2b-44b1-bb25-756f60519204 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:28.510585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9408859-cf2b-44b1-bb25-756f60519204 None admin] PATH_INFO: `/v3/users/def9ab1b20e24b65870b4a851d7650b7` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:28.512184 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9408859-cf2b-44b1-bb25-756f60519204 None admin] RBAC: Authorizing `identity:get_user(user_id=def9ab1b20e24b65870b4a851d7650b7)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:28.513704 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9408859-cf2b-44b1-bb25-756f60519204 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:28.515612 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 25/51] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:28 2022] GET /identity/v3/users/def9ab1b20e24b65870b4a851d7650b7 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:28.520874 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8b9d6-dd1c-471c-b227-e9ebd4718ec6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:28.635576 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8b9d6-dd1c-471c-b227-e9ebd4718ec6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:28.672131 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fbb03-74f2-4975-bd11-7d7197b8924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:28.672940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fbb03-74f2-4975-bd11-7d7197b8924d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:28.673220 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fbb03-74f2-4975-bd11-7d7197b8924d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:28.673495 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fbb03-74f2-4975-bd11-7d7197b8924d None admin] PATH_INFO: `/v3/domains/default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:28.674702 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fbb03-74f2-4975-bd11-7d7197b8924d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:28.676162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fbb03-74f2-4975-bd11-7d7197b8924d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:28.677781 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 27/52] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:03:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:28.683776 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9408859-cf2b-44b1-bb25-756f60519204 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:28.686771 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:28.748663 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26547895-8639-48c9-ba99-f5a2f94eec67 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:28.748890 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26547895-8639-48c9-ba99-f5a2f94eec67 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:28.749090 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26547895-8639-48c9-ba99-f5a2f94eec67 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:28.750509 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 28/53] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:28.755734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167c8e7-02b7-4430-a956-2ff4d532f77a None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:28.756026 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167c8e7-02b7-4430-a956-2ff4d532f77a None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:28.756301 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167c8e7-02b7-4430-a956-2ff4d532f77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:28.773512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-1167c8e7-02b7-4430-a956-2ff4d532f77a None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:28.774749 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-1167c8e7-02b7-4430-a956-2ff4d532f77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:28.776818 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:28.791457 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9408859-cf2b-44b1-bb25-756f60519204 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:28.824965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf407d-0173-49e0-8722-c7266800eb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:28.825640 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf407d-0173-49e0-8722-c7266800eb03 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:28.825925 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf407d-0173-49e0-8722-c7266800eb03 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:28.826190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf407d-0173-49e0-8722-c7266800eb03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:28.827066 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf407d-0173-49e0-8722-c7266800eb03 None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.domain.id=default, user.id=def9ab1b20e24b65870b4a851d7650b7) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:28.827404 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf407d-0173-49e0-8722-c7266800eb03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:28.828882 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf407d-0173-49e0-8722-c7266800eb03 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:28.839354 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 26/54] 10.222.0.84 () {56 vars in 1322 bytes} [Thu May 19 20:03:28 2022] GET /identity/v3/role_assignments?user.id=def9ab1b20e24b65870b4a851d7650b7&scope.domain.id=default&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:28.845349 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 29/55] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:28.868801 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf407d-0173-49e0-8722-c7266800eb03 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:28.941654 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:28.981956 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf407d-0173-49e0-8722-c7266800eb03 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:29.020380 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3e7fe-1a65-4d98-bbb9-a853abaa9cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:29.021125 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3e7fe-1a65-4d98-bbb9-a853abaa9cdc None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:29.021412 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3e7fe-1a65-4d98-bbb9-a853abaa9cdc None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:29.021679 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3e7fe-1a65-4d98-bbb9-a853abaa9cdc None admin] PATH_INFO: `/v3/domains/default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:29.022946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3e7fe-1a65-4d98-bbb9-a853abaa9cdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:29.024481 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3e7fe-1a65-4d98-bbb9-a853abaa9cdc None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:29.026786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 27/56] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:03:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:29.032932 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167c8e7-02b7-4430-a956-2ff4d532f77a None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:29.144654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167c8e7-02b7-4430-a956-2ff4d532f77a None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:29.179864 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10d054-38a9-4acb-8435-0f14870c6f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:29.180527 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10d054-38a9-4acb-8435-0f14870c6f48 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:29.180811 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10d054-38a9-4acb-8435-0f14870c6f48 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:29.181090 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10d054-38a9-4acb-8435-0f14870c6f48 None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:29.181803 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10d054-38a9-4acb-8435-0f14870c6f48 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:29.183323 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10d054-38a9-4acb-8435-0f14870c6f48 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:29.191695 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:29.210462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 30/57] 10.222.0.84 () {60 vars in 1140 bytes} [Thu May 19 20:03:29 2022] POST /identity/v3/projects => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:03:31.437342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78815414-4e29-418d-bd6d-c54bf63d389a None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:31.437930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78815414-4e29-418d-bd6d-c54bf63d389a None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:31.437930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78815414-4e29-418d-bd6d-c54bf63d389a None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:31.440096 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 28/58] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:31.448801 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a09b4-fe0f-4df4-be2e-a3b6459ea6aa None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:31.449245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a09b4-fe0f-4df4-be2e-a3b6459ea6aa None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:31.449473 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a09b4-fe0f-4df4-be2e-a3b6459ea6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:31.468819 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-c11a09b4-fe0f-4df4-be2e-a3b6459ea6aa None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:31.470372 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-c11a09b4-fe0f-4df4-be2e-a3b6459ea6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:31.545136 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 31/59] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:31.579081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78815414-4e29-418d-bd6d-c54bf63d389a None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:31.684387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78815414-4e29-418d-bd6d-c54bf63d389a None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:31.719930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1aafd-a174-46e3-b684-5c4e4a6e9cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:31.720655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1aafd-a174-46e3-b684-5c4e4a6e9cb3 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:31.720935 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1aafd-a174-46e3-b684-5c4e4a6e9cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:31.721205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1aafd-a174-46e3-b684-5c4e4a6e9cb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:31.722432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1aafd-a174-46e3-b684-5c4e4a6e9cb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:31.723880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1aafd-a174-46e3-b684-5c4e4a6e9cb3 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:31.725572 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 29/60] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:03:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:31.731903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a09b4-fe0f-4df4-be2e-a3b6459ea6aa None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:31.847652 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a09b4-fe0f-4df4-be2e-a3b6459ea6aa None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:31.870241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:31.885863 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e5926-db9a-4e7e-939c-9b32ee0b861b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:31.886566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e5926-db9a-4e7e-939c-9b32ee0b861b None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:31.886919 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e5926-db9a-4e7e-939c-9b32ee0b861b None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:31.887118 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e5926-db9a-4e7e-939c-9b32ee0b861b None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:31.887941 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e5926-db9a-4e7e-939c-9b32ee0b861b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:31.889404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e5926-db9a-4e7e-939c-9b32ee0b861b None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:31.916941 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 32/61] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:03:31 2022] POST /identity/v3/users => generated 275 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:03:35.238426 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39fbdb-35d9-429a-a362-66a80870053d None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:35.238426 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39fbdb-35d9-429a-a362-66a80870053d None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:35.238426 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39fbdb-35d9-429a-a362-66a80870053d None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:35.238426 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 30/62] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:35.244051 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a061fad-ac5d-4f5b-a528-f5a34291889b None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:35.244862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a061fad-ac5d-4f5b-a528-f5a34291889b None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:35.245295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a061fad-ac5d-4f5b-a528-f5a34291889b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:35.275317 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-3a061fad-ac5d-4f5b-a528-f5a34291889b None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:35.276823 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-3a061fad-ac5d-4f5b-a528-f5a34291889b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:35.367638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 33/63] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:03:35.400558 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d89ac-1112-4e63-8ac8-3d52bde64e20 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:35.401197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d89ac-1112-4e63-8ac8-3d52bde64e20 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:35.401706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d89ac-1112-4e63-8ac8-3d52bde64e20 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:35.401823 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a061fad-ac5d-4f5b-a528-f5a34291889b None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:35.404201 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 31/64] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:35.409912 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177c643-0913-4879-9e8c-5ecbec7d42d0 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:35.410556 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177c643-0913-4879-9e8c-5ecbec7d42d0 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:35.411056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177c643-0913-4879-9e8c-5ecbec7d42d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:35.416415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:35.438249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-9177c643-0913-4879-9e8c-5ecbec7d42d0 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:35.439567 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-9177c643-0913-4879-9e8c-5ecbec7d42d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:35.536355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a061fad-ac5d-4f5b-a528-f5a34291889b None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:35.545209 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 32/65] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:03:35.554288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:35.579905 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177c643-0913-4879-9e8c-5ecbec7d42d0 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:35.613746 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682440d-0995-4cda-824f-e09bba1d63dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:35.614597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682440d-0995-4cda-824f-e09bba1d63dd None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:35.614877 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682440d-0995-4cda-824f-e09bba1d63dd None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:35.615153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682440d-0995-4cda-824f-e09bba1d63dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:35.636350 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682440d-0995-4cda-824f-e09bba1d63dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-8682440d-0995-4cda-824f-e09bba1d63dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:35.644202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:35.645148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:35.646007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:35.646007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:35.646007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:35.646007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:35.646007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:35.646007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:35.646007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:35.646007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:35.646007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:35.646007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:35.646007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:03:35.646007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:03:35.658433 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 34/66] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:03:35.659363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682440d-0995-4cda-824f-e09bba1d63dd None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:35.745152 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177c643-0913-4879-9e8c-5ecbec7d42d0 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:35.804747 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:35.814342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08cef4-97b9-4e99-a76e-28af8b6245b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:35.815167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08cef4-97b9-4e99-a76e-28af8b6245b2 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:35.815446 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08cef4-97b9-4e99-a76e-28af8b6245b2 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:35.815717 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08cef4-97b9-4e99-a76e-28af8b6245b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:35.838934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08cef4-97b9-4e99-a76e-28af8b6245b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-8d08cef4-97b9-4e99-a76e-28af8b6245b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:35.849860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:35.850867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:35.851784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:35.851784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:35.851784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:35.851784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:35.851784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:35.851784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:35.851784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:35.851784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:35.851784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:35.851784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:35.851784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:35.851784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:35.851784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 33/67] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:03:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:03:35.859587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08cef4-97b9-4e99-a76e-28af8b6245b2 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:35.894367 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682440d-0995-4cda-824f-e09bba1d63dd None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:35.938548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af8bfc-ef13-4de4-a370-b29930f8bc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:35.939642 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8bfc-ef13-4de4-a370-b29930f8bc36 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:35.939994 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8bfc-ef13-4de4-a370-b29930f8bc36 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:35.940382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8bfc-ef13-4de4-a370-b29930f8bc36 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:35.941157 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af8bfc-ef13-4de4-a370-b29930f8bc36 None admin] RBAC: Adding query filter params (name=member) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:35.941471 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af8bfc-ef13-4de4-a370-b29930f8bc36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:35.942946 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af8bfc-ef13-4de4-a370-b29930f8bc36 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:35.952767 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 35/68] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:03:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:35.954092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a821f2-f4ab-4bee-b172-ded12bc1af90 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:35.954427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a821f2-f4ab-4bee-b172-ded12bc1af90 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:35.954712 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a821f2-f4ab-4bee-b172-ded12bc1af90 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:35.955944 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 36/69] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:35.957018 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f44e8-3050-40e2-930c-e697ddc36969 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:35.957306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f44e8-3050-40e2-930c-e697ddc36969 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:35.957590 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f44e8-3050-40e2-930c-e697ddc36969 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:35.958839 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 37/70] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:35.959922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53ed0f-942f-473d-83cc-d294ddca2a08 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:35.960205 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53ed0f-942f-473d-83cc-d294ddca2a08 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:35.960484 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53ed0f-942f-473d-83cc-d294ddca2a08 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:35.961698 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 38/71] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:35.962428 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53ed0f-942f-473d-83cc-d294ddca2a08 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:35.982770 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08cef4-97b9-4e99-a76e-28af8b6245b2 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:35.994480 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:36.015965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1fcff-d520-4cc0-abf9-185ebd58a333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:36.016645 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1fcff-d520-4cc0-abf9-185ebd58a333 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:36.016929 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1fcff-d520-4cc0-abf9-185ebd58a333 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:36.017207 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1fcff-d520-4cc0-abf9-185ebd58a333 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:36.017968 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1fcff-d520-4cc0-abf9-185ebd58a333 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:36.018285 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1fcff-d520-4cc0-abf9-185ebd58a333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:36.019702 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1fcff-d520-4cc0-abf9-185ebd58a333 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:36.031689 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 34/72] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:36.032890 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb5591-f078-4c02-b901-ec30ea7bea56 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:36.033180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb5591-f078-4c02-b901-ec30ea7bea56 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:36.033456 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb5591-f078-4c02-b901-ec30ea7bea56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:36.051196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-cedb5591-f078-4c02-b901-ec30ea7bea56 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:36.052401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-cedb5591-f078-4c02-b901-ec30ea7bea56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:36.075001 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53ed0f-942f-473d-83cc-d294ddca2a08 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:36.104345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:36.109577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53db01d-bf54-40c1-a59d-8a6ea5e0ee5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:36.110466 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53db01d-bf54-40c1-a59d-8a6ea5e0ee5e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:36.110748 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53db01d-bf54-40c1-a59d-8a6ea5e0ee5e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:36.111024 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53db01d-bf54-40c1-a59d-8a6ea5e0ee5e None admin] PATH_INFO: `/v3/users/f1a2f68967994de49bab3d1e94aa44dc` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:36.123638 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 35/73] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:36.124933 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60006303-294d-4e7f-8882-d9a6bee7c0eb None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:36.125230 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60006303-294d-4e7f-8882-d9a6bee7c0eb None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:36.125514 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60006303-294d-4e7f-8882-d9a6bee7c0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:36.135950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53db01d-bf54-40c1-a59d-8a6ea5e0ee5e None admin] RBAC: Authorizing `identity:get_user(user_id=f1a2f68967994de49bab3d1e94aa44dc)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:36.137641 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53db01d-bf54-40c1-a59d-8a6ea5e0ee5e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:36.139531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 39/74] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:35 2022] GET /identity/v3/users/f1a2f68967994de49bab3d1e94aa44dc => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:36.140691 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3df0e-7126-4a3c-a3f6-fdfe6ea140c8 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:36.140975 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3df0e-7126-4a3c-a3f6-fdfe6ea140c8 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:36.141261 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3df0e-7126-4a3c-a3f6-fdfe6ea140c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:36.149064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-60006303-294d-4e7f-8882-d9a6bee7c0eb None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:36.149064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-60006303-294d-4e7f-8882-d9a6bee7c0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:36.165518 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-10a3df0e-7126-4a3c-a3f6-fdfe6ea140c8 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:36.166728 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-10a3df0e-7126-4a3c-a3f6-fdfe6ea140c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:36.212131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 36/75] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:36.215842 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60006303-294d-4e7f-8882-d9a6bee7c0eb None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:36.240559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 40/76] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:36.248368 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3df0e-7126-4a3c-a3f6-fdfe6ea140c8 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:36.273310 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:36.317970 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60006303-294d-4e7f-8882-d9a6bee7c0eb None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:36.331875 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:36.352534 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe634e-b847-44e5-888f-856568c8ffc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:36.353407 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe634e-b847-44e5-888f-856568c8ffc8 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:36.353724 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe634e-b847-44e5-888f-856568c8ffc8 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:36.354008 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe634e-b847-44e5-888f-856568c8ffc8 None admin] PATH_INFO: `/v3/users/def9ab1b20e24b65870b4a851d7650b7` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:36.355293 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe634e-b847-44e5-888f-856568c8ffc8 None admin] RBAC: Authorizing `identity:get_user(user_id=def9ab1b20e24b65870b4a851d7650b7)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:36.357136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe634e-b847-44e5-888f-856568c8ffc8 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:36.357657 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3df0e-7126-4a3c-a3f6-fdfe6ea140c8 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:36.359021 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 37/77] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:36 2022] GET /identity/v3/users/def9ab1b20e24b65870b4a851d7650b7 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:36.359599 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe634e-b847-44e5-888f-856568c8ffc8 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:36.393532 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef8284-8856-488a-ab82-fd05febd10c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:36.394287 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef8284-8856-488a-ab82-fd05febd10c6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:36.394576 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef8284-8856-488a-ab82-fd05febd10c6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:36.394857 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef8284-8856-488a-ab82-fd05febd10c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:36.462971 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe634e-b847-44e5-888f-856568c8ffc8 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:36.466756 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef8284-8856-488a-ab82-fd05febd10c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:36.468319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef8284-8856-488a-ab82-fd05febd10c6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:36.487699 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:36.500962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7e3a8-bf71-4ccf-9be5-47dff09633a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:36.501801 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7e3a8-bf71-4ccf-9be5-47dff09633a9 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:36.502487 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7e3a8-bf71-4ccf-9be5-47dff09633a9 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:36.502487 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7e3a8-bf71-4ccf-9be5-47dff09633a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:36.513904 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7e3a8-bf71-4ccf-9be5-47dff09633a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-c6c7e3a8-bf71-4ccf-9be5-47dff09633a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:36.522173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:36.523467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:36.524315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:36.524315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:36.524315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:36.524315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:36.524315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:36.524315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:36.524315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:36.524315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:36.524315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:36.524315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:36.524315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:03:36.524315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:36.524315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 38/78] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:03:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:36.526851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7e3a8-bf71-4ccf-9be5-47dff09633a9 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:36.572735 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:36.577785 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 41/79] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:03:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:03:36.578676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef8284-8856-488a-ab82-fd05febd10c6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:36.632948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7e3a8-bf71-4ccf-9be5-47dff09633a9 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:36.666096 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd97446-cbe2-4fb7-a8f5-6fceb9ee7a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:36.666959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd97446-cbe2-4fb7-a8f5-6fceb9ee7a87 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:36.667237 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd97446-cbe2-4fb7-a8f5-6fceb9ee7a87 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:36.667520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd97446-cbe2-4fb7-a8f5-6fceb9ee7a87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:36.668794 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd97446-cbe2-4fb7-a8f5-6fceb9ee7a87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:36.670282 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd97446-cbe2-4fb7-a8f5-6fceb9ee7a87 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:36.671996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 39/80] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:03:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:36.677388 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd97446-cbe2-4fb7-a8f5-6fceb9ee7a87 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:36.678920 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef8284-8856-488a-ab82-fd05febd10c6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:36.696992 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:36.716923 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26423c-1020-4907-8281-eb05d0e1f891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:36.717737 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26423c-1020-4907-8281-eb05d0e1f891 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:36.718022 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26423c-1020-4907-8281-eb05d0e1f891 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:36.718322 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26423c-1020-4907-8281-eb05d0e1f891 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:36.728817 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26423c-1020-4907-8281-eb05d0e1f891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-ea26423c-1020-4907-8281-eb05d0e1f891 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:36.736824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:36.737812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:36.738736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:36.738736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:36.738736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:36.738736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:36.738736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:36.738736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:36.738736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:36.738736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:36.738736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:36.738736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:36.738736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:03:36.738736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:03:36.738736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 42/81] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:36.739533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26423c-1020-4907-8281-eb05d0e1f891 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:36.782718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd97446-cbe2-4fb7-a8f5-6fceb9ee7a87 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:36.793059 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:36.817109 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da024a2-0ae1-4632-a6c3-b726ebc3e02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:36.817772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da024a2-0ae1-4632-a6c3-b726ebc3e02e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:36.818083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da024a2-0ae1-4632-a6c3-b726ebc3e02e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:36.818395 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da024a2-0ae1-4632-a6c3-b726ebc3e02e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:36.839140 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26423c-1020-4907-8281-eb05d0e1f891 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:36.873842 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6cd84-4aa5-47ff-baaa-507f4108a643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:36.874560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6cd84-4aa5-47ff-baaa-507f4108a643 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:36.874851 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6cd84-4aa5-47ff-baaa-507f4108a643 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:36.875131 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6cd84-4aa5-47ff-baaa-507f4108a643 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:36.875892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6cd84-4aa5-47ff-baaa-507f4108a643 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:36.876206 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6cd84-4aa5-47ff-baaa-507f4108a643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:36.877629 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6cd84-4aa5-47ff-baaa-507f4108a643 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:36.885989 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 43/82] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:03:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:36.886537 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6cd84-4aa5-47ff-baaa-507f4108a643 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:36.891223 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da024a2-0ae1-4632-a6c3-b726ebc3e02e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:36.892722 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da024a2-0ae1-4632-a6c3-b726ebc3e02e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:36.927520 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:36.990120 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6cd84-4aa5-47ff-baaa-507f4108a643 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:36.995029 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 40/83] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:03:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:03:36.995938 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da024a2-0ae1-4632-a6c3-b726ebc3e02e None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:37.007295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:37.029231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6db951-99bf-4131-bc4e-9f588046b256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:37.030114 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6db951-99bf-4131-bc4e-9f588046b256 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:37.030454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6db951-99bf-4131-bc4e-9f588046b256 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:37.030717 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6db951-99bf-4131-bc4e-9f588046b256 None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:37.031921 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6db951-99bf-4131-bc4e-9f588046b256 None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:37.033413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6db951-99bf-4131-bc4e-9f588046b256 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:37.035179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 44/84] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:36 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:37.035959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6db951-99bf-4131-bc4e-9f588046b256 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:37.097106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da024a2-0ae1-4632-a6c3-b726ebc3e02e None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:37.132099 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1c6c5-9277-4881-ab13-0a1d14701136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:37.132776 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1c6c5-9277-4881-ab13-0a1d14701136 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:37.133056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1c6c5-9277-4881-ab13-0a1d14701136 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:37.133353 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1c6c5-9277-4881-ab13-0a1d14701136 None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:37.134069 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1c6c5-9277-4881-ab13-0a1d14701136 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:37.135564 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1c6c5-9277-4881-ab13-0a1d14701136 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:37.141074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6db951-99bf-4131-bc4e-9f588046b256 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:37.148010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:37.162967 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 41/85] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:03:36 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:03:37.174551 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1c6c5-9277-4881-ab13-0a1d14701136 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:37.177127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1757f91-eb58-4dd4-bb2a-c749aa1fdce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:37.177827 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1757f91-eb58-4dd4-bb2a-c749aa1fdce7 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:37.178130 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1757f91-eb58-4dd4-bb2a-c749aa1fdce7 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:37.178431 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1757f91-eb58-4dd4-bb2a-c749aa1fdce7 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:37.179195 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1757f91-eb58-4dd4-bb2a-c749aa1fdce7 None admin] RBAC: Adding query filter params (name=member) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:37.179502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1757f91-eb58-4dd4-bb2a-c749aa1fdce7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:37.180944 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1757f91-eb58-4dd4-bb2a-c749aa1fdce7 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:37.190538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 45/86] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:03:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:37.191431 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1757f91-eb58-4dd4-bb2a-c749aa1fdce7 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:37.257929 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:37.297464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1c6c5-9277-4881-ab13-0a1d14701136 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:37.319496 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1757f91-eb58-4dd4-bb2a-c749aa1fdce7 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:37.346671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1517a-c109-4cb2-96c3-af2c727664f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:37.347588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1517a-c109-4cb2-96c3-af2c727664f1 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:37.352216 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1517a-c109-4cb2-96c3-af2c727664f1 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:37.352514 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1517a-c109-4cb2-96c3-af2c727664f1 None admin] PATH_INFO: `/v3/users/f1a2f68967994de49bab3d1e94aa44dc` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:37.354359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1517a-c109-4cb2-96c3-af2c727664f1 None admin] RBAC: Authorizing `identity:get_user(user_id=f1a2f68967994de49bab3d1e94aa44dc)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:37.356427 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1517a-c109-4cb2-96c3-af2c727664f1 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:37.360523 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99146406-46e7-4511-b5e4-b81603b51019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:37.361643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99146406-46e7-4511-b5e4-b81603b51019 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:37.361765 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 42/87] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:37 2022] GET /identity/v3/users/f1a2f68967994de49bab3d1e94aa44dc => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:37.362248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99146406-46e7-4511-b5e4-b81603b51019 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:37.362638 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1517a-c109-4cb2-96c3-af2c727664f1 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:37.362989 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99146406-46e7-4511-b5e4-b81603b51019 None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:37.364398 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99146406-46e7-4511-b5e4-b81603b51019 None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:37.366089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99146406-46e7-4511-b5e4-b81603b51019 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:37.368306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 46/88] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:37 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:37.369294 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99146406-46e7-4511-b5e4-b81603b51019 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:37.412203 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:37.494441 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1517a-c109-4cb2-96c3-af2c727664f1 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:37.500005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99146406-46e7-4511-b5e4-b81603b51019 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:37.541576 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260f112-222c-4d7a-8a09-bd5506a08a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:37.541576 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260f112-222c-4d7a-8a09-bd5506a08a49 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:37.541576 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260f112-222c-4d7a-8a09-bd5506a08a49 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:37.541969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260f112-222c-4d7a-8a09-bd5506a08a49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:37.542536 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260f112-222c-4d7a-8a09-bd5506a08a49 None admin] RBAC: Adding query filter params (role.id=6aee6f5679c84fc1be21561ddb7d1b56, scope.project.id=e490252619914b5d80405cb54917237f, user.id=f1a2f68967994de49bab3d1e94aa44dc) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:37.542965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260f112-222c-4d7a-8a09-bd5506a08a49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:37.544713 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260f112-222c-4d7a-8a09-bd5506a08a49 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:37.548097 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:37.549667 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c24952-787a-44cf-b98b-6b4fb6f798ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:37.551272 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c24952-787a-44cf-b98b-6b4fb6f798ac None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:37.551272 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c24952-787a-44cf-b98b-6b4fb6f798ac None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:37.551475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c24952-787a-44cf-b98b-6b4fb6f798ac None admin] PATH_INFO: `/v3/users/f1a2f68967994de49bab3d1e94aa44dc` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:37.551679 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 43/89] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:03:37 2022] GET /identity/v3/role_assignments?user.id=f1a2f68967994de49bab3d1e94aa44dc&scope.project.id=e490252619914b5d80405cb54917237f&role.id=6aee6f5679c84fc1be21561ddb7d1b56 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:37.553065 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c24952-787a-44cf-b98b-6b4fb6f798ac None admin] RBAC: Authorizing `identity:get_user(user_id=f1a2f68967994de49bab3d1e94aa44dc)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:37.554598 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c24952-787a-44cf-b98b-6b4fb6f798ac None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:37.559930 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 47/90] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:37 2022] GET /identity/v3/users/f1a2f68967994de49bab3d1e94aa44dc => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:37.559930 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c24952-787a-44cf-b98b-6b4fb6f798ac None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:37.560250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260f112-222c-4d7a-8a09-bd5506a08a49 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:37.712408 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260f112-222c-4d7a-8a09-bd5506a08a49 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:37.757885 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfbcd8-bb3d-4d20-bff0-cf779af00944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:37.758616 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfbcd8-bb3d-4d20-bff0-cf779af00944 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:37.758910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfbcd8-bb3d-4d20-bff0-cf779af00944 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:37.759670 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfbcd8-bb3d-4d20-bff0-cf779af00944 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:37.780612 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c24952-787a-44cf-b98b-6b4fb6f798ac None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:37.788279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:37.848680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:37.853715 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfbcd8-bb3d-4d20-bff0-cf779af00944 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:37.856991 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfbcd8-bb3d-4d20-bff0-cf779af00944 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:37.861571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23c96b-ba4c-450a-bc6b-a39ce6d8fe42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:37.862586 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23c96b-ba4c-450a-bc6b-a39ce6d8fe42 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:37.863088 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23c96b-ba4c-450a-bc6b-a39ce6d8fe42 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:37.863592 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23c96b-ba4c-450a-bc6b-a39ce6d8fe42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:37.864648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b23c96b-ba4c-450a-bc6b-a39ce6d8fe42 None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=e490252619914b5d80405cb54917237f, user.id=def9ab1b20e24b65870b4a851d7650b7) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:37.865188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b23c96b-ba4c-450a-bc6b-a39ce6d8fe42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:37.867326 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b23c96b-ba4c-450a-bc6b-a39ce6d8fe42 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:37.883818 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 48/91] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:03:37 2022] GET /identity/v3/role_assignments?user.id=def9ab1b20e24b65870b4a851d7650b7&scope.project.id=e490252619914b5d80405cb54917237f&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:37.886881 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23c96b-ba4c-450a-bc6b-a39ce6d8fe42 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:38.076210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 44/92] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:03:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:03:38.086019 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfbcd8-bb3d-4d20-bff0-cf779af00944 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:38.131391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23c96b-ba4c-450a-bc6b-a39ce6d8fe42 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:38.207657 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b01d34-b620-49d8-83b2-313552491e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:38.208385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01d34-b620-49d8-83b2-313552491e09 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:38.208677 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01d34-b620-49d8-83b2-313552491e09 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:38.208958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01d34-b620-49d8-83b2-313552491e09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:38.285857 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:38.289775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:38.291875 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfbcd8-bb3d-4d20-bff0-cf779af00944 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:38.334331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e004622-fa97-4d1e-9576-c5760328d5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:38.335143 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e004622-fa97-4d1e-9576-c5760328d5e7 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:38.335425 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e004622-fa97-4d1e-9576-c5760328d5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:38.335701 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e004622-fa97-4d1e-9576-c5760328d5e7 None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:38.337506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e004622-fa97-4d1e-9576-c5760328d5e7 None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:38.339339 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e004622-fa97-4d1e-9576-c5760328d5e7 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:38.341765 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 45/93] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:38 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:38.348976 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e004622-fa97-4d1e-9576-c5760328d5e7 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:38.375729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b01d34-b620-49d8-83b2-313552491e09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:38.377257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b01d34-b620-49d8-83b2-313552491e09 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:38.599394 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 49/94] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:03:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:03:38.604900 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b01d34-b620-49d8-83b2-313552491e09 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:38.621156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e004622-fa97-4d1e-9576-c5760328d5e7 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:38.702985 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af356c-93ac-479e-824f-5cd998dce77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:38.704067 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af356c-93ac-479e-824f-5cd998dce77b None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:38.704579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af356c-93ac-479e-824f-5cd998dce77b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:38.705071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af356c-93ac-479e-824f-5cd998dce77b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:38.706120 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af356c-93ac-479e-824f-5cd998dce77b None admin] RBAC: Adding query filter params (role.id=b21c1004651d40f1b44fb0b7becbe3c8, scope.project.id=e490252619914b5d80405cb54917237f, user.id=f1a2f68967994de49bab3d1e94aa44dc) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:38.706695 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af356c-93ac-479e-824f-5cd998dce77b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:38.708382 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af356c-93ac-479e-824f-5cd998dce77b None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:38.720552 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 46/95] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:03:38 2022] GET /identity/v3/role_assignments?user.id=f1a2f68967994de49bab3d1e94aa44dc&scope.project.id=e490252619914b5d80405cb54917237f&role.id=b21c1004651d40f1b44fb0b7becbe3c8 => generated 259 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:38.751914 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:38.801710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b01d34-b620-49d8-83b2-313552491e09 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:38.857322 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf3865-d419-4b07-9ac7-1c1c029c8607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:38.858132 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf3865-d419-4b07-9ac7-1c1c029c8607 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:38.858441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf3865-d419-4b07-9ac7-1c1c029c8607 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:38.858721 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf3865-d419-4b07-9ac7-1c1c029c8607 None admin] PATH_INFO: `/v3/projects/9c02016426a24337ae797364e59b794d` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:38.859999 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf3865-d419-4b07-9ac7-1c1c029c8607 None admin] RBAC: Authorizing `identity:get_project(project_id=9c02016426a24337ae797364e59b794d)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:38.861497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf3865-d419-4b07-9ac7-1c1c029c8607 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:38.863648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 50/96] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:38 2022] GET /identity/v3/projects/9c02016426a24337ae797364e59b794d => generated 312 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:38.869808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af356c-93ac-479e-824f-5cd998dce77b None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:38.928307 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:39.031606 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af356c-93ac-479e-824f-5cd998dce77b None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:39.073831 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbffd2-ca2f-4c30-ab92-53787d3c5458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:39.074780 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbffd2-ca2f-4c30-ab92-53787d3c5458 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:39.075068 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbffd2-ca2f-4c30-ab92-53787d3c5458 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:39.075346 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbffd2-ca2f-4c30-ab92-53787d3c5458 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:39.076196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dbffd2-ca2f-4c30-ab92-53787d3c5458 None admin] RBAC: Adding query filter params (role.id=6aee6f5679c84fc1be21561ddb7d1b56, scope.project.id=9c02016426a24337ae797364e59b794d, user.id=f1a2f68967994de49bab3d1e94aa44dc) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:39.076506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dbffd2-ca2f-4c30-ab92-53787d3c5458 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:39.077972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dbffd2-ca2f-4c30-ab92-53787d3c5458 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:39.087306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 47/97] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:03:38 2022] GET /identity/v3/role_assignments?user.id=f1a2f68967994de49bab3d1e94aa44dc&scope.project.id=9c02016426a24337ae797364e59b794d&role.id=6aee6f5679c84fc1be21561ddb7d1b56 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:40.127463 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbf2b1-90c6-4133-9524-1df2956f5e60 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:40.128029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbf2b1-90c6-4133-9524-1df2956f5e60 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:40.128029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbf2b1-90c6-4133-9524-1df2956f5e60 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:40.129401 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 51/98] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:40.137287 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c36cdd-f52a-4b68-9aa8-5eac5f3d443b None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:40.137798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c36cdd-f52a-4b68-9aa8-5eac5f3d443b None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:40.137889 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c36cdd-f52a-4b68-9aa8-5eac5f3d443b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:40.159772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-09c36cdd-f52a-4b68-9aa8-5eac5f3d443b None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:40.161064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-09c36cdd-f52a-4b68-9aa8-5eac5f3d443b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:40.241351 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 48/99] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:40.280844 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbf2b1-90c6-4133-9524-1df2956f5e60 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:40.411138 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbf2b1-90c6-4133-9524-1df2956f5e60 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:40.451107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407cc28-d4e3-485e-b4c2-ad144b0e89e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:40.451937 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407cc28-d4e3-485e-b4c2-ad144b0e89e8 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:40.452210 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407cc28-d4e3-485e-b4c2-ad144b0e89e8 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:40.452484 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407cc28-d4e3-485e-b4c2-ad144b0e89e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:40.455499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:40.461457 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db913a-57e5-4448-b328-717e9df9289c None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:40.461740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db913a-57e5-4448-b328-717e9df9289c None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:40.462017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db913a-57e5-4448-b328-717e9df9289c None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:40.463629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 49/100] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:40.464994 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407cc28-d4e3-485e-b4c2-ad144b0e89e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:40.469675 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8da82e-2154-417c-855c-e6b6e6675844 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:40.469964 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8da82e-2154-417c-855c-e6b6e6675844 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:40.470245 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8da82e-2154-417c-855c-e6b6e6675844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-5407cc28-d4e3-485e-b4c2-ad144b0e89e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:40.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:40.473346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:40.474319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:40.474319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:40.474319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:40.474319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:40.474319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:40.474319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:40.474319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:40.474319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:40.474319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:40.474319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:40.474319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:03:40.474319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:03:40.474319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 52/101] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:40.489359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407cc28-d4e3-485e-b4c2-ad144b0e89e8 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:40.500879 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-7e8da82e-2154-417c-855c-e6b6e6675844 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:40.502398 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-7e8da82e-2154-417c-855c-e6b6e6675844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:40.550663 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:40.602934 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407cc28-d4e3-485e-b4c2-ad144b0e89e8 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:40.626598 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 50/102] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:40.628040 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb0b96-5659-4cee-bef1-2715101f9af4 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:40.628328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb0b96-5659-4cee-bef1-2715101f9af4 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:40.628603 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb0b96-5659-4cee-bef1-2715101f9af4 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:40.630166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 51/103] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:40.635598 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873611c-5779-4f51-a757-a0cd0ab9ed9d None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:40.635984 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873611c-5779-4f51-a757-a0cd0ab9ed9d None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:40.636402 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873611c-5779-4f51-a757-a0cd0ab9ed9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:40.637901 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191d501-77f9-4938-9e5c-6cfc949b0da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:40.638624 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191d501-77f9-4938-9e5c-6cfc949b0da7 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:40.638906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191d501-77f9-4938-9e5c-6cfc949b0da7 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:40.639189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191d501-77f9-4938-9e5c-6cfc949b0da7 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:40.639944 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191d501-77f9-4938-9e5c-6cfc949b0da7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:40.640252 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191d501-77f9-4938-9e5c-6cfc949b0da7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:40.641646 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191d501-77f9-4938-9e5c-6cfc949b0da7 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:40.650704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 53/104] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:03:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:40.656085 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191d501-77f9-4938-9e5c-6cfc949b0da7 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:40.670976 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-6873611c-5779-4f51-a757-a0cd0ab9ed9d None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:40.672154 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-6873611c-5779-4f51-a757-a0cd0ab9ed9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:40.716038 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:40.759874 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191d501-77f9-4938-9e5c-6cfc949b0da7 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:40.789871 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 52/105] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:40.790768 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873611c-5779-4f51-a757-a0cd0ab9ed9d None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:40.824438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b9521-6932-4ef0-9ad9-a873031fae51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:40.825342 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b9521-6932-4ef0-9ad9-a873031fae51 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:40.825619 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b9521-6932-4ef0-9ad9-a873031fae51 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:40.825895 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b9521-6932-4ef0-9ad9-a873031fae51 None admin] PATH_INFO: `/v3/users/125eb65567a84b35b5abca76fa44eca5` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:40.852286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b9521-6932-4ef0-9ad9-a873031fae51 None admin] RBAC: Authorizing `identity:get_user(user_id=125eb65567a84b35b5abca76fa44eca5)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:40.853778 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b9521-6932-4ef0-9ad9-a873031fae51 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:40.856692 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 54/106] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:40 2022] GET /identity/v3/users/125eb65567a84b35b5abca76fa44eca5 => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:40.864160 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b9521-6932-4ef0-9ad9-a873031fae51 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:40.958752 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:40.989804 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873611c-5779-4f51-a757-a0cd0ab9ed9d None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:41.054863 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd03c2-5461-47bb-8bb2-cb96f012e8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:41.055681 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd03c2-5461-47bb-8bb2-cb96f012e8a6 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:41.055968 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd03c2-5461-47bb-8bb2-cb96f012e8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:41.056241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd03c2-5461-47bb-8bb2-cb96f012e8a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:41.060483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd03c2-5461-47bb-8bb2-cb96f012e8a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:41.061967 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd03c2-5461-47bb-8bb2-cb96f012e8a6 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:41.066489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b9521-6932-4ef0-9ad9-a873031fae51 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:41.068307 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 53/107] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:03:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:41.069099 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd03c2-5461-47bb-8bb2-cb96f012e8a6 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:41.104201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:41.132073 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d72b5-abd3-4cc5-b5d5-13aff3f2440e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:41.132892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d72b5-abd3-4cc5-b5d5-13aff3f2440e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:41.133175 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d72b5-abd3-4cc5-b5d5-13aff3f2440e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:41.133452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d72b5-abd3-4cc5-b5d5-13aff3f2440e None admin] PATH_INFO: `/v3/roles/member` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:41.157044 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d72b5-abd3-4cc5-b5d5-13aff3f2440e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-447d72b5-abd3-4cc5-b5d5-13aff3f2440e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:41.166450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:41.169007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:41.171402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:41.171402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:41.171402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:41.171402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:41.171402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:41.171402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:41.171402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:41.171402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:41.171402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:41.171402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:41.171402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:03:41.171402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:03:41.171402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 55/108] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:41.171402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d72b5-abd3-4cc5-b5d5-13aff3f2440e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:41.252292 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd03c2-5461-47bb-8bb2-cb96f012e8a6 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:41.290038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bfb10-1b95-4e0a-a60d-e20001678b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:41.290865 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bfb10-1b95-4e0a-a60d-e20001678b9f None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:41.291152 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bfb10-1b95-4e0a-a60d-e20001678b9f None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:41.291429 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bfb10-1b95-4e0a-a60d-e20001678b9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:41.330427 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:41.365079 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d72b5-abd3-4cc5-b5d5-13aff3f2440e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:41.395195 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bfb10-1b95-4e0a-a60d-e20001678b9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:41.397296 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bfb10-1b95-4e0a-a60d-e20001678b9f None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:41.400014 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4eb1fa-ca6e-4e1c-819b-533f27b8e6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:41.400854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4eb1fa-ca6e-4e1c-819b-533f27b8e6c4 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:41.401921 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4eb1fa-ca6e-4e1c-819b-533f27b8e6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:41.401921 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4eb1fa-ca6e-4e1c-819b-533f27b8e6c4 None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:41.402621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4eb1fa-ca6e-4e1c-819b-533f27b8e6c4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:41.404392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4eb1fa-ca6e-4e1c-819b-533f27b8e6c4 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:41.433294 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 56/109] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:03:41 2022] POST /identity/v3/projects => generated 302 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:03:41.438584 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4eb1fa-ca6e-4e1c-819b-533f27b8e6c4 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:41.489747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:41.533578 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 54/110] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:03:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:03:41.535539 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054432f-bbbb-4d47-b64d-ff9b972763f9 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:41.535935 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054432f-bbbb-4d47-b64d-ff9b972763f9 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:41.536315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054432f-bbbb-4d47-b64d-ff9b972763f9 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:41.538032 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 55/111] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:41.548737 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054432f-bbbb-4d47-b64d-ff9b972763f9 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:41.618677 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:41.632897 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4eb1fa-ca6e-4e1c-819b-533f27b8e6c4 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:41.663172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054432f-bbbb-4d47-b64d-ff9b972763f9 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:41.693734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878548e-3928-497d-8f94-24d6d14174cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:41.694475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878548e-3928-497d-8f94-24d6d14174cc None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:41.694759 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878548e-3928-497d-8f94-24d6d14174cc None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:41.695036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878548e-3928-497d-8f94-24d6d14174cc None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:41.695811 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878548e-3928-497d-8f94-24d6d14174cc None admin] RBAC: Adding query filter params (name=member) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:41.696123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878548e-3928-497d-8f94-24d6d14174cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:41.697544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878548e-3928-497d-8f94-24d6d14174cc None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:41.701228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb03fc2-62ae-4568-9c7e-4a9de5950b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:41.702255 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb03fc2-62ae-4568-9c7e-4a9de5950b16 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:41.702661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb03fc2-62ae-4568-9c7e-4a9de5950b16 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:41.703478 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb03fc2-62ae-4568-9c7e-4a9de5950b16 None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:41.704830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb03fc2-62ae-4568-9c7e-4a9de5950b16 None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:41.706462 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb03fc2-62ae-4568-9c7e-4a9de5950b16 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:41.707324 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 57/112] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:03:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:41.708648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb66eec-9079-4aa4-803e-c9162130b4d7 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:41.708779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 56/113] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:41 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:41.708933 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb66eec-9079-4aa4-803e-c9162130b4d7 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:41.709402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb66eec-9079-4aa4-803e-c9162130b4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:41.718283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb03fc2-62ae-4568-9c7e-4a9de5950b16 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:41.752223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-adb66eec-9079-4aa4-803e-c9162130b4d7 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:41.755247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-adb66eec-9079-4aa4-803e-c9162130b4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:41.849102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 58/114] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:41.849753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb66eec-9079-4aa4-803e-c9162130b4d7 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:41.866995 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb03fc2-62ae-4568-9c7e-4a9de5950b16 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:41.882083 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:41.900885 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:41.920143 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792d2be-7238-4cb6-88df-192366855c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:41.921207 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792d2be-7238-4cb6-88df-192366855c5c None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:41.921207 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792d2be-7238-4cb6-88df-192366855c5c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:41.921471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792d2be-7238-4cb6-88df-192366855c5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:41.923204 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792d2be-7238-4cb6-88df-192366855c5c None admin] RBAC: Adding query filter params (role.id=609bb86f2a5b44b0b244b9bfe7c073de, scope.project.id=e490252619914b5d80405cb54917237f, user.id=125eb65567a84b35b5abca76fa44eca5) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:41.923445 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792d2be-7238-4cb6-88df-192366855c5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:41.924999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792d2be-7238-4cb6-88df-192366855c5c None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:41.931350 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 57/115] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:03:41 2022] GET /identity/v3/role_assignments?user.id=125eb65567a84b35b5abca76fa44eca5&scope.project.id=e490252619914b5d80405cb54917237f&role.id=609bb86f2a5b44b0b244b9bfe7c073de => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:41.932559 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792d2be-7238-4cb6-88df-192366855c5c None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:41.992107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb66eec-9079-4aa4-803e-c9162130b4d7 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:42.035970 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb49070-e57f-4898-8431-1a3b5808a50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:42.037252 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb49070-e57f-4898-8431-1a3b5808a50d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:42.037760 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb49070-e57f-4898-8431-1a3b5808a50d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:42.038240 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb49070-e57f-4898-8431-1a3b5808a50d None admin] PATH_INFO: `/v3/users/f1a2f68967994de49bab3d1e94aa44dc` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:42.040232 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb49070-e57f-4898-8431-1a3b5808a50d None admin] RBAC: Authorizing `identity:get_user(user_id=f1a2f68967994de49bab3d1e94aa44dc)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:42.042208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb49070-e57f-4898-8431-1a3b5808a50d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:42.044989 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 59/116] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:41 2022] GET /identity/v3/users/f1a2f68967994de49bab3d1e94aa44dc => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:42.051249 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb49070-e57f-4898-8431-1a3b5808a50d None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:42.053035 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792d2be-7238-4cb6-88df-192366855c5c None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:42.094520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191acf7a-8b9f-40f6-97a1-78f928b94312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:42.095379 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191acf7a-8b9f-40f6-97a1-78f928b94312 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:42.095666 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191acf7a-8b9f-40f6-97a1-78f928b94312 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:42.095949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191acf7a-8b9f-40f6-97a1-78f928b94312 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:42.112473 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191acf7a-8b9f-40f6-97a1-78f928b94312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-191acf7a-8b9f-40f6-97a1-78f928b94312 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:42.122803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:42.124027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:42.124987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:42.124987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:42.124987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:42.124987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:42.124987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:42.124987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:42.124987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:42.124987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:42.124987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:42.124987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:42.124987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:42.124987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:42.124987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 58/117] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:03:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:42.132014 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191acf7a-8b9f-40f6-97a1-78f928b94312 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:42.166300 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:42.182882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb49070-e57f-4898-8431-1a3b5808a50d None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:42.216043 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:42.225642 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfeecf0-2c71-4f04-9c20-6fef7b0b090c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:42.226903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfeecf0-2c71-4f04-9c20-6fef7b0b090c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:42.227355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfeecf0-2c71-4f04-9c20-6fef7b0b090c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:42.227810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfeecf0-2c71-4f04-9c20-6fef7b0b090c None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:42.229301 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfeecf0-2c71-4f04-9c20-6fef7b0b090c None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:42.231355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfeecf0-2c71-4f04-9c20-6fef7b0b090c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:42.233466 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 60/118] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:42 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:42.240485 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfeecf0-2c71-4f04-9c20-6fef7b0b090c None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:42.336270 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191acf7a-8b9f-40f6-97a1-78f928b94312 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:42.357547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfeecf0-2c71-4f04-9c20-6fef7b0b090c None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:42.380557 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af1dd7-4c77-4d42-a08f-b82815d42d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:42.381165 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af1dd7-4c77-4d42-a08f-b82815d42d44 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:42.381461 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af1dd7-4c77-4d42-a08f-b82815d42d44 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:42.381992 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af1dd7-4c77-4d42-a08f-b82815d42d44 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:42.382790 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af1dd7-4c77-4d42-a08f-b82815d42d44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:42.383101 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af1dd7-4c77-4d42-a08f-b82815d42d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:42.385469 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af1dd7-4c77-4d42-a08f-b82815d42d44 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:42.395735 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10eedec-7eae-48b1-9cf4-e4c1473e746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:42.395943 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 59/119] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:42.396981 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10eedec-7eae-48b1-9cf4-e4c1473e746f None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:42.397115 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e513b5b-772c-4280-bf30-e3045e46b70e None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:42.397356 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e513b5b-772c-4280-bf30-e3045e46b70e None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:42.397637 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e513b5b-772c-4280-bf30-e3045e46b70e None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:42.397860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10eedec-7eae-48b1-9cf4-e4c1473e746f None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:42.398308 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10eedec-7eae-48b1-9cf4-e4c1473e746f None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f/users/f1a2f68967994de49bab3d1e94aa44dc/roles/6aee6f5679c84fc1be21561ddb7d1b56` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:42.398887 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 60/120] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:42.399950 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0eec55-22b4-4834-aa1b-75df3907c1fa None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:42.400230 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0eec55-22b4-4834-aa1b-75df3907c1fa None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:42.400510 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0eec55-22b4-4834-aa1b-75df3907c1fa None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:42.401120 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10eedec-7eae-48b1-9cf4-e4c1473e746f None admin] RBAC: Authorizing `identity:create_grant(project_id=e490252619914b5d80405cb54917237f, user_id=f1a2f68967994de49bab3d1e94aa44dc, role_id=6aee6f5679c84fc1be21561ddb7d1b56)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:42.401698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 61/121] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:42.403275 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10eedec-7eae-48b1-9cf4-e4c1473e746f None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:42.407852 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0eec55-22b4-4834-aa1b-75df3907c1fa None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:42.424953 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 61/122] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:03:42 2022] PUT /identity/v3/projects/e490252619914b5d80405cb54917237f/users/f1a2f68967994de49bab3d1e94aa44dc/roles/6aee6f5679c84fc1be21561ddb7d1b56 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:03:42.426545 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125dc92-4f20-45d5-86ac-8ba6235e291f None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:42.426991 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125dc92-4f20-45d5-86ac-8ba6235e291f None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:42.427430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125dc92-4f20-45d5-86ac-8ba6235e291f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:42.460603 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-9125dc92-4f20-45d5-86ac-8ba6235e291f None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:42.463351 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-9125dc92-4f20-45d5-86ac-8ba6235e291f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:42.480889 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:42.580992 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 62/123] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:42.582161 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38f783-945e-4907-a0c5-50a7e60f19c0 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:42.582474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38f783-945e-4907-a0c5-50a7e60f19c0 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:42.582859 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38f783-945e-4907-a0c5-50a7e60f19c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:42.612385 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0eec55-22b4-4834-aa1b-75df3907c1fa None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:42.619277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-fe38f783-945e-4907-a0c5-50a7e60f19c0 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:42.620477 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-fe38f783-945e-4907-a0c5-50a7e60f19c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:42.621654 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:42.675450 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5cdac-c425-4418-b8db-5afbdec8bf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:42.676268 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5cdac-c425-4418-b8db-5afbdec8bf12 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:42.676556 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5cdac-c425-4418-b8db-5afbdec8bf12 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:42.676839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5cdac-c425-4418-b8db-5afbdec8bf12 None admin] PATH_INFO: `/v3/users/def9ab1b20e24b65870b4a851d7650b7` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:42.678139 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5cdac-c425-4418-b8db-5afbdec8bf12 None admin] RBAC: Authorizing `identity:get_user(user_id=def9ab1b20e24b65870b4a851d7650b7)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:42.679770 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5cdac-c425-4418-b8db-5afbdec8bf12 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:42.681891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 62/124] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:42 2022] GET /identity/v3/users/def9ab1b20e24b65870b4a851d7650b7 => generated 247 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:42.682847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5cdac-c425-4418-b8db-5afbdec8bf12 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:42.732190 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 63/125] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:42.733268 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38f783-945e-4907-a0c5-50a7e60f19c0 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:42.806423 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5cdac-c425-4418-b8db-5afbdec8bf12 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:42.830682 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:42.843100 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f7784-5bf3-49cc-a42c-00f65ecf211a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:42.844060 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38f783-945e-4907-a0c5-50a7e60f19c0 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:42.844596 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f7784-5bf3-49cc-a42c-00f65ecf211a None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:42.845103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f7784-5bf3-49cc-a42c-00f65ecf211a None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:42.845590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f7784-5bf3-49cc-a42c-00f65ecf211a None admin] PATH_INFO: `/v3/roles/member` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:42.857656 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f7784-5bf3-49cc-a42c-00f65ecf211a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-992f7784-5bf3-49cc-a42c-00f65ecf211a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:42.865629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:42.866836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:42.867913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:42.867913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:42.867913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:42.867913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:42.867913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:42.867913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:42.867913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:42.867913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:42.867913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:42.867913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:42.867913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:03:42.867913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:42.869974 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 63/126] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:42.870726 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f7784-5bf3-49cc-a42c-00f65ecf211a None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:42.884289 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c25a5-5982-43c0-b016-5d8eae4ae0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:42.885240 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c25a5-5982-43c0-b016-5d8eae4ae0ea None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:42.885589 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c25a5-5982-43c0-b016-5d8eae4ae0ea None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:42.885879 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c25a5-5982-43c0-b016-5d8eae4ae0ea None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:42.888122 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c25a5-5982-43c0-b016-5d8eae4ae0ea None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:42.889680 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c25a5-5982-43c0-b016-5d8eae4ae0ea None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:42.891692 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 64/127] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:42 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:42.892563 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c25a5-5982-43c0-b016-5d8eae4ae0ea None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:42.909095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:42.978523 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f7784-5bf3-49cc-a42c-00f65ecf211a None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:43.011316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c25a5-5982-43c0-b016-5d8eae4ae0ea None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:43.013152 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc183d2-8f45-412c-80c7-66ee3864e40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:43.014064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc183d2-8f45-412c-80c7-66ee3864e40a None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:43.014444 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc183d2-8f45-412c-80c7-66ee3864e40a None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:43.014794 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc183d2-8f45-412c-80c7-66ee3864e40a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:43.025634 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc183d2-8f45-412c-80c7-66ee3864e40a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-ffc183d2-8f45-412c-80c7-66ee3864e40a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:43.034630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:43.035617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:43.036480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:43.036480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:43.036480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:43.036480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:43.036480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:43.036480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:43.036480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:43.036480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:43.036480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:43.036480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:43.036480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:03:43.036480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:43.036480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 64/128] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:03:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:43.037222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc183d2-8f45-412c-80c7-66ee3864e40a None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:43.058865 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b440c-7fcd-4897-9734-b3249a7eaeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:43.059694 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b440c-7fcd-4897-9734-b3249a7eaeab None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:43.059986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b440c-7fcd-4897-9734-b3249a7eaeab None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:43.060269 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b440c-7fcd-4897-9734-b3249a7eaeab None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:43.061150 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b440c-7fcd-4897-9734-b3249a7eaeab None admin] RBAC: Adding query filter params (name=member) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:43.061467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b440c-7fcd-4897-9734-b3249a7eaeab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:43.062984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b440c-7fcd-4897-9734-b3249a7eaeab None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:43.082081 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 65/129] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:03:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:43.083190 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b440c-7fcd-4897-9734-b3249a7eaeab None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:43.126628 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:43.156728 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:43.172508 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc183d2-8f45-412c-80c7-66ee3864e40a None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:43.213702 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b440c-7fcd-4897-9734-b3249a7eaeab None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:43.251738 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b93020-e608-41d5-9fa0-43e4e156af37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:43.252684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b93020-e608-41d5-9fa0-43e4e156af37 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:43.253173 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b93020-e608-41d5-9fa0-43e4e156af37 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:43.253777 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b93020-e608-41d5-9fa0-43e4e156af37 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:43.254812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b93020-e608-41d5-9fa0-43e4e156af37 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:43.255276 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b93020-e608-41d5-9fa0-43e4e156af37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:43.256966 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b93020-e608-41d5-9fa0-43e4e156af37 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:43.266152 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 66/130] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:03:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:43.267312 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b93020-e608-41d5-9fa0-43e4e156af37 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:43.275551 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e1781-769f-49a7-9b78-326a37b9a03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:43.276460 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e1781-769f-49a7-9b78-326a37b9a03c None admin] REQUEST_METHOD: `PUT` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:43.276742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e1781-769f-49a7-9b78-326a37b9a03c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:43.277048 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e1781-769f-49a7-9b78-326a37b9a03c None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f/users/def9ab1b20e24b65870b4a851d7650b7/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:43.283694 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e1781-769f-49a7-9b78-326a37b9a03c None admin] RBAC: Authorizing `identity:create_grant(project_id=e490252619914b5d80405cb54917237f, user_id=def9ab1b20e24b65870b4a851d7650b7, role_id=b34e4ad4fe7b4a60bd2fc669e2b31a66)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:43.285111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e1781-769f-49a7-9b78-326a37b9a03c None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:43.321335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 65/131] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:03:43 2022] PUT /identity/v3/projects/e490252619914b5d80405cb54917237f/users/def9ab1b20e24b65870b4a851d7650b7/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:03:43.322220 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e1781-769f-49a7-9b78-326a37b9a03c None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:43.345582 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:43.388297 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b93020-e608-41d5-9fa0-43e4e156af37 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:43.427153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37334b-3681-4d2d-909a-705d105be179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:43.428327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37334b-3681-4d2d-909a-705d105be179 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:43.428792 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37334b-3681-4d2d-909a-705d105be179 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:43.429259 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37334b-3681-4d2d-909a-705d105be179 None admin] PATH_INFO: `/v3/users/f1a2f68967994de49bab3d1e94aa44dc` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:43.430786 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37334b-3681-4d2d-909a-705d105be179 None admin] RBAC: Authorizing `identity:get_user(user_id=f1a2f68967994de49bab3d1e94aa44dc)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:43.432610 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37334b-3681-4d2d-909a-705d105be179 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:43.434644 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 67/132] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:43 2022] GET /identity/v3/users/f1a2f68967994de49bab3d1e94aa44dc => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:43.443937 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37334b-3681-4d2d-909a-705d105be179 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:43.540527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e1781-769f-49a7-9b78-326a37b9a03c None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:43.579986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da22e3-cf7f-4013-b332-30f32f9782a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:43.581029 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da22e3-cf7f-4013-b332-30f32f9782a5 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:43.581331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da22e3-cf7f-4013-b332-30f32f9782a5 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:43.581610 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da22e3-cf7f-4013-b332-30f32f9782a5 None admin] PATH_INFO: `/v3/users/f1a2f68967994de49bab3d1e94aa44dc` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:43.583263 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da22e3-cf7f-4013-b332-30f32f9782a5 None admin] RBAC: Authorizing `identity:get_user(user_id=f1a2f68967994de49bab3d1e94aa44dc)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:43.584728 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da22e3-cf7f-4013-b332-30f32f9782a5 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:43.586411 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 66/133] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:43 2022] GET /identity/v3/users/f1a2f68967994de49bab3d1e94aa44dc => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:43.592736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37334b-3681-4d2d-909a-705d105be179 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:43.595581 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da22e3-cf7f-4013-b332-30f32f9782a5 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:43.637936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:43.646089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed00f3-695a-47e1-93ba-90cebc44f4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:43.647014 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed00f3-695a-47e1-93ba-90cebc44f4b0 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:43.647296 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed00f3-695a-47e1-93ba-90cebc44f4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:43.647573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed00f3-695a-47e1-93ba-90cebc44f4b0 None admin] PATH_INFO: `/v3/projects/9c02016426a24337ae797364e59b794d` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:43.648846 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed00f3-695a-47e1-93ba-90cebc44f4b0 None admin] RBAC: Authorizing `identity:get_project(project_id=9c02016426a24337ae797364e59b794d)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:43.650413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed00f3-695a-47e1-93ba-90cebc44f4b0 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:43.652367 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 68/134] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:43 2022] GET /identity/v3/projects/9c02016426a24337ae797364e59b794d => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:43.657950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed00f3-695a-47e1-93ba-90cebc44f4b0 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:43.662058 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:43.728715 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da22e3-cf7f-4013-b332-30f32f9782a5 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:43.767743 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e87b41-5a3f-49a0-b54f-df36d53ab030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:43.769018 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e87b41-5a3f-49a0-b54f-df36d53ab030 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:43.769551 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e87b41-5a3f-49a0-b54f-df36d53ab030 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:43.770131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e87b41-5a3f-49a0-b54f-df36d53ab030 None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:43.772336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e87b41-5a3f-49a0-b54f-df36d53ab030 None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:43.774053 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e87b41-5a3f-49a0-b54f-df36d53ab030 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:43.776100 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 67/135] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:43 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:43.781880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e87b41-5a3f-49a0-b54f-df36d53ab030 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:43.810374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed00f3-695a-47e1-93ba-90cebc44f4b0 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:43.845321 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343c550-c6b1-40d0-b1b7-67366bb86054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:43.846175 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343c550-c6b1-40d0-b1b7-67366bb86054 None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:43.846492 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343c550-c6b1-40d0-b1b7-67366bb86054 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:43.846768 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343c550-c6b1-40d0-b1b7-67366bb86054 None admin] PATH_INFO: `/v3/projects/9c02016426a24337ae797364e59b794d/users/f1a2f68967994de49bab3d1e94aa44dc/roles/6aee6f5679c84fc1be21561ddb7d1b56` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:43.848933 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4343c550-c6b1-40d0-b1b7-67366bb86054 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c02016426a24337ae797364e59b794d, user_id=f1a2f68967994de49bab3d1e94aa44dc, role_id=6aee6f5679c84fc1be21561ddb7d1b56)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:43.850468 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4343c550-c6b1-40d0-b1b7-67366bb86054 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:43.866767 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 69/136] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:03:43 2022] PUT /identity/v3/projects/9c02016426a24337ae797364e59b794d/users/f1a2f68967994de49bab3d1e94aa44dc/roles/6aee6f5679c84fc1be21561ddb7d1b56 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:03:43.921565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:43.939003 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e87b41-5a3f-49a0-b54f-df36d53ab030 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:43.982644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a14124-194b-426e-9266-797dfc09e1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:43.983573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a14124-194b-426e-9266-797dfc09e1a2 None admin] REQUEST_METHOD: `PUT` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:43.983859 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a14124-194b-426e-9266-797dfc09e1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:43.984141 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a14124-194b-426e-9266-797dfc09e1a2 None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f/users/f1a2f68967994de49bab3d1e94aa44dc/roles/b21c1004651d40f1b44fb0b7becbe3c8` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:43.990611 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a14124-194b-426e-9266-797dfc09e1a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e490252619914b5d80405cb54917237f, user_id=f1a2f68967994de49bab3d1e94aa44dc, role_id=b21c1004651d40f1b44fb0b7becbe3c8)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:43.992211 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a14124-194b-426e-9266-797dfc09e1a2 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:44.014992 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 68/137] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:03:43 2022] PUT /identity/v3/projects/e490252619914b5d80405cb54917237f/users/f1a2f68967994de49bab3d1e94aa44dc/roles/b21c1004651d40f1b44fb0b7becbe3c8 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:03:44.930428 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50c7bb-8905-4f56-a1f4-d3ad9a26a9f4 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:44.930938 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50c7bb-8905-4f56-a1f4-d3ad9a26a9f4 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:44.930938 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50c7bb-8905-4f56-a1f4-d3ad9a26a9f4 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:44.932269 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 70/138] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:44.937721 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389c97e-cf7e-4469-8225-6d21405159e5 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:44.938134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389c97e-cf7e-4469-8225-6d21405159e5 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:44.938428 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389c97e-cf7e-4469-8225-6d21405159e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:44.956070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-2389c97e-cf7e-4469-8225-6d21405159e5 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:44.957259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-2389c97e-cf7e-4469-8225-6d21405159e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:45.036279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 69/139] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:03:45.084969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50c7bb-8905-4f56-a1f4-d3ad9a26a9f4 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:45.102872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff0e3d-3181-41e1-bda8-0af086205a28 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:45.103191 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff0e3d-3181-41e1-bda8-0af086205a28 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:45.103436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff0e3d-3181-41e1-bda8-0af086205a28 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:45.104849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 70/140] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:03:45.110443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f8127-6a75-44c4-b868-c007ca7660e7 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:45.110752 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f8127-6a75-44c4-b868-c007ca7660e7 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:45.111004 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f8127-6a75-44c4-b868-c007ca7660e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:45.151931 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-b65f8127-6a75-44c4-b868-c007ca7660e7 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:45.153213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-b65f8127-6a75-44c4-b868-c007ca7660e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:45.235083 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:45.309452 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 71/141] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:03:45.319913 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50c7bb-8905-4f56-a1f4-d3ad9a26a9f4 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:45.369840 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2885a-d1d2-4d7d-b8f5-00aa1adc9889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:45.370727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2885a-d1d2-4d7d-b8f5-00aa1adc9889 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:45.371259 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2885a-d1d2-4d7d-b8f5-00aa1adc9889 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:45.371576 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2885a-d1d2-4d7d-b8f5-00aa1adc9889 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:45.377704 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f8127-6a75-44c4-b868-c007ca7660e7 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:45.381214 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2885a-d1d2-4d7d-b8f5-00aa1adc9889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:45.387419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2885a-d1d2-4d7d-b8f5-00aa1adc9889 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:45.389121 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 71/142] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:03:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:45.395995 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2885a-d1d2-4d7d-b8f5-00aa1adc9889 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:45.396398 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:45.617447 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2885a-d1d2-4d7d-b8f5-00aa1adc9889 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:45.620625 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f8127-6a75-44c4-b868-c007ca7660e7 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:45.676724 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:45.683227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5bc5a-628d-4057-affd-e95d8817bb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:45.683949 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5bc5a-628d-4057-affd-e95d8817bb1f None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:45.684233 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5bc5a-628d-4057-affd-e95d8817bb1f None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:45.684508 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5bc5a-628d-4057-affd-e95d8817bb1f None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:45.685242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5bc5a-628d-4057-affd-e95d8817bb1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:45.686762 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5bc5a-628d-4057-affd-e95d8817bb1f None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:45.730149 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 72/143] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:03:45 2022] POST /identity/v3/users => generated 283 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:03:45.731908 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf9b50-92f1-4abd-be66-d283572d7d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:45.732748 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf9b50-92f1-4abd-be66-d283572d7d41 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:45.733032 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf9b50-92f1-4abd-be66-d283572d7d41 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:45.733308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf9b50-92f1-4abd-be66-d283572d7d41 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:45.759833 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf9b50-92f1-4abd-be66-d283572d7d41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-d9bf9b50-92f1-4abd-be66-d283572d7d41 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:45.768167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:45.769238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:45.770151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:45.770151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:45.770151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:45.770151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:45.770151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:45.770151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:45.770151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:45.770151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:45.770151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:45.770151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:45.770151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:03:45.770151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:45.770151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 72/144] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:03:45.779463 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5bc5a-628d-4057-affd-e95d8817bb1f None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:45.928447 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5bc5a-628d-4057-affd-e95d8817bb1f None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:45.948459 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598be43-d975-4639-822c-c884a95bab9c None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:45.949096 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598be43-d975-4639-822c-c884a95bab9c None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:45.949096 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598be43-d975-4639-822c-c884a95bab9c None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:45.950822 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 73/145] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:45.959398 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58857e68-706b-4a8c-aae7-3c60fde3873b None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:45.959730 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58857e68-706b-4a8c-aae7-3c60fde3873b None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:45.959972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58857e68-706b-4a8c-aae7-3c60fde3873b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:45.984006 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13388e-7df3-49be-84a7-3a7504f19746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:45.984731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13388e-7df3-49be-84a7-3a7504f19746 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:45.984992 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13388e-7df3-49be-84a7-3a7504f19746 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:45.985270 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13388e-7df3-49be-84a7-3a7504f19746 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:45.986038 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13388e-7df3-49be-84a7-3a7504f19746 None admin] RBAC: Adding query filter params (name=reader) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:45.986379 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13388e-7df3-49be-84a7-3a7504f19746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:45.987798 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13388e-7df3-49be-84a7-3a7504f19746 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:45.996686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-58857e68-706b-4a8c-aae7-3c60fde3873b None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:45.997285 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-58857e68-706b-4a8c-aae7-3c60fde3873b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:46.008886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 73/146] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:03:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:46.033910 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13388e-7df3-49be-84a7-3a7504f19746 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:46.135118 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:46.137195 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:46.145484 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 74/147] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:46.173725 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58857e68-706b-4a8c-aae7-3c60fde3873b None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:46.243761 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13388e-7df3-49be-84a7-3a7504f19746 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:46.322233 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58857e68-706b-4a8c-aae7-3c60fde3873b None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:46.331008 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9840e1-7e22-4a78-9fb1-1b84a54eb10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:46.331928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9840e1-7e22-4a78-9fb1-1b84a54eb10f None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:46.332216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9840e1-7e22-4a78-9fb1-1b84a54eb10f None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:46.332498 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9840e1-7e22-4a78-9fb1-1b84a54eb10f None admin] PATH_INFO: `/v3/users/125eb65567a84b35b5abca76fa44eca5` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:46.334122 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9840e1-7e22-4a78-9fb1-1b84a54eb10f None admin] RBAC: Authorizing `identity:get_user(user_id=125eb65567a84b35b5abca76fa44eca5)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:46.335673 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9840e1-7e22-4a78-9fb1-1b84a54eb10f None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:46.337725 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 74/148] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:46 2022] GET /identity/v3/users/125eb65567a84b35b5abca76fa44eca5 => generated 289 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:46.343330 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9840e1-7e22-4a78-9fb1-1b84a54eb10f None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:46.371820 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e031a1e-0cc6-4707-ba35-5306928a08b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:46.372921 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e031a1e-0cc6-4707-ba35-5306928a08b1 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:46.373205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e031a1e-0cc6-4707-ba35-5306928a08b1 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:46.373488 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e031a1e-0cc6-4707-ba35-5306928a08b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:46.397774 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e031a1e-0cc6-4707-ba35-5306928a08b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-2e031a1e-0cc6-4707-ba35-5306928a08b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:46.419449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:46.420519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:46.421409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:46.421409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:46.421409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:46.421409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:46.421409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:46.421409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:46.421409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:46.421409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:46.421409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:46.421409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:46.421409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:03:46.421409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:46.422069 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 75/149] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:03:46.432892 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e031a1e-0cc6-4707-ba35-5306928a08b1 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:46.518835 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:46.535550 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9840e1-7e22-4a78-9fb1-1b84a54eb10f None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:46.558584 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:46.581287 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e031a1e-0cc6-4707-ba35-5306928a08b1 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:46.592027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3c4c2-a18b-4573-81cd-e4e67a9dc0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:46.593211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3c4c2-a18b-4573-81cd-e4e67a9dc0f3 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:46.593670 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3c4c2-a18b-4573-81cd-e4e67a9dc0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:46.594115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3c4c2-a18b-4573-81cd-e4e67a9dc0f3 None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:46.595901 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3c4c2-a18b-4573-81cd-e4e67a9dc0f3 None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:46.597574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3c4c2-a18b-4573-81cd-e4e67a9dc0f3 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:46.600381 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 75/150] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:46 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:46.601807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230d78c-877b-4bd8-af11-f1b438098c0a None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:46.602330 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230d78c-877b-4bd8-af11-f1b438098c0a None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:46.602794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230d78c-877b-4bd8-af11-f1b438098c0a None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:46.609954 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 76/151] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:46.611044 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230d78c-877b-4bd8-af11-f1b438098c0a None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:46.629271 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061fcdf-eb24-494e-970a-25f73a189744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:46.629968 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061fcdf-eb24-494e-970a-25f73a189744 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:46.630248 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061fcdf-eb24-494e-970a-25f73a189744 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:46.630952 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061fcdf-eb24-494e-970a-25f73a189744 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:46.631723 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061fcdf-eb24-494e-970a-25f73a189744 None admin] RBAC: Adding query filter params (name=member) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:46.632086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061fcdf-eb24-494e-970a-25f73a189744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:46.635659 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061fcdf-eb24-494e-970a-25f73a189744 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:46.645106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 76/152] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:03:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:46.647128 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5e31b-7165-44a1-bae3-2e5d38920b81 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:46.647418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5e31b-7165-44a1-bae3-2e5d38920b81 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:46.647691 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5e31b-7165-44a1-bae3-2e5d38920b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:46.669928 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-c7f5e31b-7165-44a1-bae3-2e5d38920b81 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:46.671468 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-c7f5e31b-7165-44a1-bae3-2e5d38920b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:46.739332 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230d78c-877b-4bd8-af11-f1b438098c0a None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:46.755288 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 77/153] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:46.763155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5e31b-7165-44a1-bae3-2e5d38920b81 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:46.777675 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422d2b4-d16a-4617-b079-e8ce1e48777e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:46.779097 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422d2b4-d16a-4617-b079-e8ce1e48777e None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:46.779589 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422d2b4-d16a-4617-b079-e8ce1e48777e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:46.780070 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422d2b4-d16a-4617-b079-e8ce1e48777e None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f/users/125eb65567a84b35b5abca76fa44eca5/roles/609bb86f2a5b44b0b244b9bfe7c073de` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:46.782307 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422d2b4-d16a-4617-b079-e8ce1e48777e None admin] RBAC: Authorizing `identity:create_grant(project_id=e490252619914b5d80405cb54917237f, user_id=125eb65567a84b35b5abca76fa44eca5, role_id=609bb86f2a5b44b0b244b9bfe7c073de)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:46.783985 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422d2b4-d16a-4617-b079-e8ce1e48777e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:46.801240 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 77/154] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:03:46 2022] PUT /identity/v3/projects/e490252619914b5d80405cb54917237f/users/125eb65567a84b35b5abca76fa44eca5/roles/609bb86f2a5b44b0b244b9bfe7c073de => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:03:46.815919 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422d2b4-d16a-4617-b079-e8ce1e48777e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:46.837928 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:46.889767 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:46.898183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5e31b-7165-44a1-bae3-2e5d38920b81 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:46.936261 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510281a-e8fc-44b8-b4f1-ce4abcd63016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:46.937224 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510281a-e8fc-44b8-b4f1-ce4abcd63016 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:46.937512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510281a-e8fc-44b8-b4f1-ce4abcd63016 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:46.938486 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510281a-e8fc-44b8-b4f1-ce4abcd63016 None admin] PATH_INFO: `/v3/users/f1a2f68967994de49bab3d1e94aa44dc` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:46.940234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510281a-e8fc-44b8-b4f1-ce4abcd63016 None admin] RBAC: Authorizing `identity:get_user(user_id=f1a2f68967994de49bab3d1e94aa44dc)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:46.941870 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510281a-e8fc-44b8-b4f1-ce4abcd63016 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:46.943672 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 78/155] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:46 2022] GET /identity/v3/users/f1a2f68967994de49bab3d1e94aa44dc => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:46.949970 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510281a-e8fc-44b8-b4f1-ce4abcd63016 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:46.979798 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422d2b4-d16a-4617-b079-e8ce1e48777e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:47.035873 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3fc05-f70c-4f8a-85cd-f8f6afd12c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:47.037529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3fc05-f70c-4f8a-85cd-f8f6afd12c9e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:47.038005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3fc05-f70c-4f8a-85cd-f8f6afd12c9e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:47.038507 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3fc05-f70c-4f8a-85cd-f8f6afd12c9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:47.060581 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3fc05-f70c-4f8a-85cd-f8f6afd12c9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:47.068151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510281a-e8fc-44b8-b4f1-ce4abcd63016 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-71d3fc05-f70c-4f8a-85cd-f8f6afd12c9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:47.077389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:47.078601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:47.079486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:47.079486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:47.079486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:47.079486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:47.079486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:47.079486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:47.079486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:47.079486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:47.079486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:47.079486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:47.079486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:47.079486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:03:47.081557 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 78/156] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:03:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:03:47.095890 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3fc05-f70c-4f8a-85cd-f8f6afd12c9e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:47.108130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:47.187138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd0842-2842-4539-8dca-b7175cd973a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:47.187831 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd0842-2842-4539-8dca-b7175cd973a4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:47.188109 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd0842-2842-4539-8dca-b7175cd973a4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:47.188383 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd0842-2842-4539-8dca-b7175cd973a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:47.263692 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3fc05-f70c-4f8a-85cd-f8f6afd12c9e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:47.278388 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:47.320520 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc7677-93ad-40df-b971-6e24cd7c156e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:47.321132 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc7677-93ad-40df-b971-6e24cd7c156e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:47.321409 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc7677-93ad-40df-b971-6e24cd7c156e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:47.321680 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc7677-93ad-40df-b971-6e24cd7c156e None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:47.322513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc7677-93ad-40df-b971-6e24cd7c156e None admin] RBAC: Adding query filter params (name=admin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:47.323283 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc7677-93ad-40df-b971-6e24cd7c156e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:47.324885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc7677-93ad-40df-b971-6e24cd7c156e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:47.336282 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 79/157] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:47.339322 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8ab16-6fe6-40a5-bd1c-9de08bf6c728 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:47.339661 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8ab16-6fe6-40a5-bd1c-9de08bf6c728 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:47.339947 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8ab16-6fe6-40a5-bd1c-9de08bf6c728 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:47.342895 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 80/158] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:47.346848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8ab16-6fe6-40a5-bd1c-9de08bf6c728 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:47.354283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd0842-2842-4539-8dca-b7175cd973a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:47.356329 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd0842-2842-4539-8dca-b7175cd973a4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:47.510953 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8ab16-6fe6-40a5-bd1c-9de08bf6c728 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:47.571069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cc0d6-e202-4f5c-a7f0-1cf2077b35de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:47.572432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cc0d6-e202-4f5c-a7f0-1cf2077b35de None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:47.572713 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cc0d6-e202-4f5c-a7f0-1cf2077b35de None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:47.573030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cc0d6-e202-4f5c-a7f0-1cf2077b35de None admin] PATH_INFO: `/v3/users/def9ab1b20e24b65870b4a851d7650b7` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:47.579252 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cc0d6-e202-4f5c-a7f0-1cf2077b35de None admin] RBAC: Authorizing `identity:get_user(user_id=def9ab1b20e24b65870b4a851d7650b7)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:47.580754 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cc0d6-e202-4f5c-a7f0-1cf2077b35de None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:47.582481 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 81/159] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:47 2022] GET /identity/v3/users/def9ab1b20e24b65870b4a851d7650b7 => generated 247 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:47.583952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec73906-24e0-4adf-884e-ebaa3a9a0f15 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:47.584236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec73906-24e0-4adf-884e-ebaa3a9a0f15 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:47.584507 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec73906-24e0-4adf-884e-ebaa3a9a0f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:47.598306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 79/160] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:03:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:03:47.613112 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd0842-2842-4539-8dca-b7175cd973a4 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:47.615302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-9ec73906-24e0-4adf-884e-ebaa3a9a0f15 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:47.616773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-9ec73906-24e0-4adf-884e-ebaa3a9a0f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:47.658444 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:47.669764 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:47.738628 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 82/161] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:47.739597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec73906-24e0-4adf-884e-ebaa3a9a0f15 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:47.872587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd0842-2842-4539-8dca-b7175cd973a4 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:47.920000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec73906-24e0-4adf-884e-ebaa3a9a0f15 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:47.934794 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f9f28-31b7-4706-8339-b52e39925911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:47.935465 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f9f28-31b7-4706-8339-b52e39925911 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:47.935741 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f9f28-31b7-4706-8339-b52e39925911 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:47.936013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f9f28-31b7-4706-8339-b52e39925911 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:47.990583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ebd6e-f001-4260-bd7d-4218b4743344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:47.991486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ebd6e-f001-4260-bd7d-4218b4743344 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:47.991760 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ebd6e-f001-4260-bd7d-4218b4743344 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:47.992030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ebd6e-f001-4260-bd7d-4218b4743344 None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:47.994812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ebd6e-f001-4260-bd7d-4218b4743344 None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:47.996385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ebd6e-f001-4260-bd7d-4218b4743344 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:47.998290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 83/162] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:47 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:48.002444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ebd6e-f001-4260-bd7d-4218b4743344 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:48.066528 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:48.081885 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f9f28-31b7-4706-8339-b52e39925911 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:48.083539 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f9f28-31b7-4706-8339-b52e39925911 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:48.116821 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:48.157742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ebd6e-f001-4260-bd7d-4218b4743344 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:48.227551 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41511e2-fc8f-4eca-aef9-392510764b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:48.228797 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41511e2-fc8f-4eca-aef9-392510764b52 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:48.229313 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41511e2-fc8f-4eca-aef9-392510764b52 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:48.229787 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41511e2-fc8f-4eca-aef9-392510764b52 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:48.237472 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 80/163] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:03:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:03:48.238363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f9f28-31b7-4706-8339-b52e39925911 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:48.248865 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41511e2-fc8f-4eca-aef9-392510764b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-b41511e2-fc8f-4eca-aef9-392510764b52 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:48.256950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:48.257952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:48.258862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:48.258862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:48.258862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:48.258862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:48.258862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:48.258862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:48.258862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:48.258862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:48.258862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:48.258862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:48.258862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:03:48.258862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:03:48.261804 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 84/164] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:48.263008 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41511e2-fc8f-4eca-aef9-392510764b52 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:48.407680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f9f28-31b7-4706-8339-b52e39925911 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:48.414737 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:48.438719 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41511e2-fc8f-4eca-aef9-392510764b52 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:48.453507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:48.505076 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f554e-e91d-4543-b007-cbf50317f768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:48.505756 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f554e-e91d-4543-b007-cbf50317f768 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:48.506041 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f554e-e91d-4543-b007-cbf50317f768 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:48.506341 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f554e-e91d-4543-b007-cbf50317f768 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:48.507197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f554e-e91d-4543-b007-cbf50317f768 None admin] RBAC: Adding query filter params (role.id=6aee6f5679c84fc1be21561ddb7d1b56, scope.project.id=e490252619914b5d80405cb54917237f, user.id=f1a2f68967994de49bab3d1e94aa44dc) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:48.507511 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f554e-e91d-4543-b007-cbf50317f768 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:48.509093 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f554e-e91d-4543-b007-cbf50317f768 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:48.521651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9accadd3-908b-41d7-8dfb-c5093060c54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:48.522932 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accadd3-908b-41d7-8dfb-c5093060c54c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:48.523412 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 81/165] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:03:48 2022] GET /identity/v3/role_assignments?user.id=f1a2f68967994de49bab3d1e94aa44dc&scope.project.id=e490252619914b5d80405cb54917237f&role.id=6aee6f5679c84fc1be21561ddb7d1b56 => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:48.523931 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accadd3-908b-41d7-8dfb-c5093060c54c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:48.524056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f554e-e91d-4543-b007-cbf50317f768 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:48.524527 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accadd3-908b-41d7-8dfb-c5093060c54c None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:48.527015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9accadd3-908b-41d7-8dfb-c5093060c54c None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:48.528856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9accadd3-908b-41d7-8dfb-c5093060c54c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:48.536675 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 85/166] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:48 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:48.538294 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed96f71-ae44-4958-9779-a9e3ee8a0afe None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:48.538798 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed96f71-ae44-4958-9779-a9e3ee8a0afe None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:48.539387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed96f71-ae44-4958-9779-a9e3ee8a0afe None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:48.540904 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 86/167] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:48.551162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed96f71-ae44-4958-9779-a9e3ee8a0afe None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:48.760020 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed96f71-ae44-4958-9779-a9e3ee8a0afe None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:48.761128 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f554e-e91d-4543-b007-cbf50317f768 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:48.836846 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34cd26c-4a48-4e66-aba9-97c181484cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:48.837645 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cd26c-4a48-4e66-aba9-97c181484cd8 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:48.837911 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cd26c-4a48-4e66-aba9-97c181484cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:48.838187 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cd26c-4a48-4e66-aba9-97c181484cd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:48.839067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34cd26c-4a48-4e66-aba9-97c181484cd8 None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=e490252619914b5d80405cb54917237f, user.id=def9ab1b20e24b65870b4a851d7650b7) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:48.839401 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34cd26c-4a48-4e66-aba9-97c181484cd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:48.840979 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34cd26c-4a48-4e66-aba9-97c181484cd8 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:48.852378 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ed93b-660c-4792-a10d-2c8a9e1defd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:48.853088 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ed93b-660c-4792-a10d-2c8a9e1defd9 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:48.853365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ed93b-660c-4792-a10d-2c8a9e1defd9 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:48.853644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ed93b-660c-4792-a10d-2c8a9e1defd9 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:48.854440 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ed93b-660c-4792-a10d-2c8a9e1defd9 None admin] RBAC: Adding query filter params (name=member) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:48.854750 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ed93b-660c-4792-a10d-2c8a9e1defd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:48.856476 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ed93b-660c-4792-a10d-2c8a9e1defd9 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:48.857230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 87/168] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:03:48 2022] GET /identity/v3/role_assignments?user.id=def9ab1b20e24b65870b4a851d7650b7&scope.project.id=e490252619914b5d80405cb54917237f&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 608 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:48.858662 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e12ac-d4de-4e2f-86be-1718778a51c1 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:48.858950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e12ac-d4de-4e2f-86be-1718778a51c1 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:48.859225 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e12ac-d4de-4e2f-86be-1718778a51c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:48.878864 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 82/169] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:03:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:48.887696 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ed93b-660c-4792-a10d-2c8a9e1defd9 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:48.892639 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-811e12ac-d4de-4e2f-86be-1718778a51c1 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:48.893843 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-811e12ac-d4de-4e2f-86be-1718778a51c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:48.942690 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:49.025561 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:49.030729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 88/170] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:49.074779 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e12ac-d4de-4e2f-86be-1718778a51c1 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:49.158791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ed93b-660c-4792-a10d-2c8a9e1defd9 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:49.267036 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc61b0-472a-4814-85b9-8b4a7fb55bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:49.267978 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc61b0-472a-4814-85b9-8b4a7fb55bf2 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:49.268259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc61b0-472a-4814-85b9-8b4a7fb55bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:49.268531 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc61b0-472a-4814-85b9-8b4a7fb55bf2 None admin] PATH_INFO: `/v3/users/f1a2f68967994de49bab3d1e94aa44dc` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:49.270367 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc61b0-472a-4814-85b9-8b4a7fb55bf2 None admin] RBAC: Authorizing `identity:get_user(user_id=f1a2f68967994de49bab3d1e94aa44dc)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:49.271344 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc61b0-472a-4814-85b9-8b4a7fb55bf2 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:49.273203 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 83/171] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:48 2022] GET /identity/v3/users/f1a2f68967994de49bab3d1e94aa44dc => generated 275 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:49.278813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc61b0-472a-4814-85b9-8b4a7fb55bf2 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:49.284916 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e12ac-d4de-4e2f-86be-1718778a51c1 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:49.344083 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fefb49-b3ae-439a-8c36-0f95d9b27e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:49.347442 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fefb49-b3ae-439a-8c36-0f95d9b27e26 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:49.347890 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fefb49-b3ae-439a-8c36-0f95d9b27e26 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:49.349266 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fefb49-b3ae-439a-8c36-0f95d9b27e26 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:49.375753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fefb49-b3ae-439a-8c36-0f95d9b27e26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-a3fefb49-b3ae-439a-8c36-0f95d9b27e26 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:49.394077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:49.395306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:49.396208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:49.396208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:49.396208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:49.396208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:49.396208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:49.396208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:49.396208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:49.396208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:49.396208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:49.396208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:49.396208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:03:49.396208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:03:49.399090 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 89/172] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:03:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:49.406629 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fefb49-b3ae-439a-8c36-0f95d9b27e26 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:49.446650 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:49.487551 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc61b0-472a-4814-85b9-8b4a7fb55bf2 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:49.511274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:49.549700 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4df23-a5bc-4b0e-8dc8-9902ae61e71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:49.550827 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4df23-a5bc-4b0e-8dc8-9902ae61e71a None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:49.551317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4df23-a5bc-4b0e-8dc8-9902ae61e71a None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:49.551793 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4df23-a5bc-4b0e-8dc8-9902ae61e71a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:49.592521 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fefb49-b3ae-439a-8c36-0f95d9b27e26 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:49.658209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1d8cb-f448-420d-866a-1a6211cb0031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:49.659128 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1d8cb-f448-420d-866a-1a6211cb0031 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:49.659408 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1d8cb-f448-420d-866a-1a6211cb0031 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:49.659692 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1d8cb-f448-420d-866a-1a6211cb0031 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:49.660967 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1d8cb-f448-420d-866a-1a6211cb0031 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:49.661272 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1d8cb-f448-420d-866a-1a6211cb0031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:49.663110 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1d8cb-f448-420d-866a-1a6211cb0031 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:49.682214 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 90/173] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:03:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:49.692651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1d8cb-f448-420d-866a-1a6211cb0031 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:49.708524 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4df23-a5bc-4b0e-8dc8-9902ae61e71a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:49.710364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4df23-a5bc-4b0e-8dc8-9902ae61e71a None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:49.834113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:49.853520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 84/174] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:03:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:03:49.859574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4df23-a5bc-4b0e-8dc8-9902ae61e71a None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:49.868485 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1d8cb-f448-420d-866a-1a6211cb0031 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:49.884484 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:49.908401 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e67073-db49-451f-85fe-83b01661962d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:49.910067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e67073-db49-451f-85fe-83b01661962d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:49.910539 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e67073-db49-451f-85fe-83b01661962d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:49.911219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e67073-db49-451f-85fe-83b01661962d None admin] PATH_INFO: `/v3/users/f1a2f68967994de49bab3d1e94aa44dc` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:49.912847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e67073-db49-451f-85fe-83b01661962d None admin] RBAC: Authorizing `identity:get_user(user_id=f1a2f68967994de49bab3d1e94aa44dc)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:49.915231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e67073-db49-451f-85fe-83b01661962d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:49.917932 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 91/175] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:49 2022] GET /identity/v3/users/f1a2f68967994de49bab3d1e94aa44dc => generated 275 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:49.924120 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e67073-db49-451f-85fe-83b01661962d None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:50.063556 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4df23-a5bc-4b0e-8dc8-9902ae61e71a None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:50.091773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e67073-db49-451f-85fe-83b01661962d None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:50.140471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df251a-f38e-44f7-846b-9ca8d2c8e9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:50.149940 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df251a-f38e-44f7-846b-9ca8d2c8e9ff None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:50.150703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df251a-f38e-44f7-846b-9ca8d2c8e9ff None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:50.151245 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df251a-f38e-44f7-846b-9ca8d2c8e9ff None admin] PATH_INFO: `/v3/projects/9c02016426a24337ae797364e59b794d` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:50.153047 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df251a-f38e-44f7-846b-9ca8d2c8e9ff None admin] RBAC: Authorizing `identity:get_project(project_id=9c02016426a24337ae797364e59b794d)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:50.154808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df251a-f38e-44f7-846b-9ca8d2c8e9ff None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:50.157298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 85/176] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:49 2022] GET /identity/v3/projects/9c02016426a24337ae797364e59b794d => generated 312 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:50.165513 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df251a-f38e-44f7-846b-9ca8d2c8e9ff None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:50.177543 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ef008-a8be-4798-8589-478b0420c8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:50.180558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ef008-a8be-4798-8589-478b0420c8f6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:50.181084 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ef008-a8be-4798-8589-478b0420c8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:50.181576 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ef008-a8be-4798-8589-478b0420c8f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:50.220448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:50.278647 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:50.299388 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df251a-f38e-44f7-846b-9ca8d2c8e9ff None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:50.334642 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a7166-9564-43ac-8f17-479356703e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:50.335703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a7166-9564-43ac-8f17-479356703e57 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:50.336210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a7166-9564-43ac-8f17-479356703e57 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:50.336692 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a7166-9564-43ac-8f17-479356703e57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:50.337739 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a7166-9564-43ac-8f17-479356703e57 None admin] RBAC: Adding query filter params (role.id=6aee6f5679c84fc1be21561ddb7d1b56, scope.project.id=9c02016426a24337ae797364e59b794d, user.id=f1a2f68967994de49bab3d1e94aa44dc) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:50.338299 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a7166-9564-43ac-8f17-479356703e57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:50.340083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a7166-9564-43ac-8f17-479356703e57 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:50.342252 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ef008-a8be-4798-8589-478b0420c8f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:50.343837 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ef008-a8be-4798-8589-478b0420c8f6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:50.347813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 86/177] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:03:50 2022] GET /identity/v3/role_assignments?user.id=f1a2f68967994de49bab3d1e94aa44dc&scope.project.id=9c02016426a24337ae797364e59b794d&role.id=6aee6f5679c84fc1be21561ddb7d1b56 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:50.349449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382adcb8-3189-485e-98a4-c5bbdbcc1397 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:50.349971 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382adcb8-3189-485e-98a4-c5bbdbcc1397 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:50.350638 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382adcb8-3189-485e-98a4-c5bbdbcc1397 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:50.352209 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 87/178] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:50.358402 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47512b97-cdb2-4b54-b002-4616e8650c53 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:50.359035 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47512b97-cdb2-4b54-b002-4616e8650c53 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:50.359561 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47512b97-cdb2-4b54-b002-4616e8650c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:50.393158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-47512b97-cdb2-4b54-b002-4616e8650c53 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:50.394930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-47512b97-cdb2-4b54-b002-4616e8650c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:50.479011 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 88/179] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:50.560545 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47512b97-cdb2-4b54-b002-4616e8650c53 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:50.591001 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 92/180] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:03:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:03:50.596932 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ef008-a8be-4798-8589-478b0420c8f6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:50.670770 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:50.693317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47512b97-cdb2-4b54-b002-4616e8650c53 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:50.693670 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:50.731186 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca6c1a-23dd-401e-b838-e30017c6b345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:50.732006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca6c1a-23dd-401e-b838-e30017c6b345 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:50.732288 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca6c1a-23dd-401e-b838-e30017c6b345 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:50.732563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca6c1a-23dd-401e-b838-e30017c6b345 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:50.743456 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca6c1a-23dd-401e-b838-e30017c6b345 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-aeca6c1a-23dd-401e-b838-e30017c6b345 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:50.751494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:50.752709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:50.753828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:50.753828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:50.753828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:50.753828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:50.753828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:50.753828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:50.753828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:50.753828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:50.753828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:50.753828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:50.753828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:50.753828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:50.753828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 89/181] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:03:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:03:50.754743 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8133dc-7f04-46bf-bdcc-3e8a3bc273b1 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:50.754743 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8133dc-7f04-46bf-bdcc-3e8a3bc273b1 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:50.754743 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8133dc-7f04-46bf-bdcc-3e8a3bc273b1 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:50.755876 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 90/182] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:50.757026 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ef008-a8be-4798-8589-478b0420c8f6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:50.763614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378331d-07c8-4ca4-9bf5-f028b3a87e6a None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:50.764264 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378331d-07c8-4ca4-9bf5-f028b3a87e6a None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:50.764816 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378331d-07c8-4ca4-9bf5-f028b3a87e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:50.784061 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-6378331d-07c8-4ca4-9bf5-f028b3a87e6a None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:50.785634 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-6378331d-07c8-4ca4-9bf5-f028b3a87e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:50.798492 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faaefea-4dbd-492f-b2e4-5ed1cd9d8ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:50.799366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faaefea-4dbd-492f-b2e4-5ed1cd9d8ee9 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:50.799642 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faaefea-4dbd-492f-b2e4-5ed1cd9d8ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:50.799913 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faaefea-4dbd-492f-b2e4-5ed1cd9d8ee9 None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:50.804256 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faaefea-4dbd-492f-b2e4-5ed1cd9d8ee9 None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:50.806206 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faaefea-4dbd-492f-b2e4-5ed1cd9d8ee9 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:50.808264 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 93/183] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:50 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:50.809250 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faaefea-4dbd-492f-b2e4-5ed1cd9d8ee9 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:50.865938 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 91/184] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:50.866680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378331d-07c8-4ca4-9bf5-f028b3a87e6a None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:50.944836 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faaefea-4dbd-492f-b2e4-5ed1cd9d8ee9 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:50.945186 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:50.960598 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:50.983049 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378331d-07c8-4ca4-9bf5-f028b3a87e6a None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:50.985354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87721488-667a-4328-ba5f-ec68fa7551da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:50.986108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87721488-667a-4328-ba5f-ec68fa7551da None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:50.986489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87721488-667a-4328-ba5f-ec68fa7551da None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:50.986776 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87721488-667a-4328-ba5f-ec68fa7551da None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:50.987538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87721488-667a-4328-ba5f-ec68fa7551da None admin] RBAC: Adding query filter params (name=admin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:50.987848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87721488-667a-4328-ba5f-ec68fa7551da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:50.989264 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87721488-667a-4328-ba5f-ec68fa7551da None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:51.002355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 94/185] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:51.007245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d2775-c090-4646-b787-f787d1a87964 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:51.007245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d2775-c090-4646-b787-f787d1a87964 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:51.007245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d2775-c090-4646-b787-f787d1a87964 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:51.009682 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 95/186] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:51.010497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d2775-c090-4646-b787-f787d1a87964 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:51.025274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e943f91-d9ba-4409-af0a-ab06264ebab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:51.026005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e943f91-d9ba-4409-af0a-ab06264ebab2 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:51.026304 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e943f91-d9ba-4409-af0a-ab06264ebab2 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:51.026543 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e943f91-d9ba-4409-af0a-ab06264ebab2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:51.027959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e943f91-d9ba-4409-af0a-ab06264ebab2 None admin] RBAC: Adding query filter params (role.id=b21c1004651d40f1b44fb0b7becbe3c8, scope.project.id=e490252619914b5d80405cb54917237f, user.id=f1a2f68967994de49bab3d1e94aa44dc) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:51.028282 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e943f91-d9ba-4409-af0a-ab06264ebab2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:51.029751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e943f91-d9ba-4409-af0a-ab06264ebab2 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:51.046290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 92/187] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:03:50 2022] GET /identity/v3/role_assignments?user.id=f1a2f68967994de49bab3d1e94aa44dc&scope.project.id=e490252619914b5d80405cb54917237f&role.id=b21c1004651d40f1b44fb0b7becbe3c8 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:51.046290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e943f91-d9ba-4409-af0a-ab06264ebab2 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:51.155063 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d2775-c090-4646-b787-f787d1a87964 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:51.173227 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e943f91-d9ba-4409-af0a-ab06264ebab2 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:51.197023 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad30406-04a4-4036-a0a1-dfbf71e76f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:51.197851 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad30406-04a4-4036-a0a1-dfbf71e76f77 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:51.198132 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad30406-04a4-4036-a0a1-dfbf71e76f77 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:51.198438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad30406-04a4-4036-a0a1-dfbf71e76f77 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:51.204639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:51.210564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad30406-04a4-4036-a0a1-dfbf71e76f77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:51.216513 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cd9a4-ffe9-4584-9909-9c17cfa9387e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:51.217418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cd9a4-ffe9-4584-9909-9c17cfa9387e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:51.217698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cd9a4-ffe9-4584-9909-9c17cfa9387e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:51.217968 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cd9a4-ffe9-4584-9909-9c17cfa9387e None admin] PATH_INFO: `/v3/users/b68ff2aacdf048ccb1e53d0b9e33de00` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-2ad30406-04a4-4036-a0a1-dfbf71e76f77 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:51.218856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:51.220231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:51.222102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:51.222102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:51.222102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:51.222102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:51.222102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:51.222102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:51.222102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:51.222102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:51.222102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:51.222102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:51.222102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:03:51.222102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:03:51.222102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 96/188] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:51.222102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bcc77-800d-45e4-9a23-029fa8ab7e69 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:51.223998 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bcc77-800d-45e4-9a23-029fa8ab7e69 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:51.223998 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bcc77-800d-45e4-9a23-029fa8ab7e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:51.236512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6cd9a4-ffe9-4584-9909-9c17cfa9387e None admin] RBAC: Authorizing `identity:get_user(user_id=b68ff2aacdf048ccb1e53d0b9e33de00)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:51.238009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6cd9a4-ffe9-4584-9909-9c17cfa9387e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:51.239962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 93/189] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:51 2022] GET /identity/v3/users/b68ff2aacdf048ccb1e53d0b9e33de00 => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:51.240156 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-7c8bcc77-800d-45e4-9a23-029fa8ab7e69 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:51.241062 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fc229-102b-448f-a8e1-8961756f939a None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:51.241343 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-7c8bcc77-800d-45e4-9a23-029fa8ab7e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:51.241454 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fc229-102b-448f-a8e1-8961756f939a None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:51.241612 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fc229-102b-448f-a8e1-8961756f939a None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:51.242988 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 94/190] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:51.243560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fc229-102b-448f-a8e1-8961756f939a None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:51.276535 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:51.321162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 97/191] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:51.322113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bcc77-800d-45e4-9a23-029fa8ab7e69 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:51.364744 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fc229-102b-448f-a8e1-8961756f939a None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:51.397915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ff4c3-8e6f-4282-932d-12b410e3c1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:51.398615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ff4c3-8e6f-4282-932d-12b410e3c1f4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:51.398885 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ff4c3-8e6f-4282-932d-12b410e3c1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:51.399156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ff4c3-8e6f-4282-932d-12b410e3c1f4 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:51.399915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ff4c3-8e6f-4282-932d-12b410e3c1f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:51.400221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ff4c3-8e6f-4282-932d-12b410e3c1f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:51.401600 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ff4c3-8e6f-4282-932d-12b410e3c1f4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:51.409732 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 95/192] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:03:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:51.410972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886c4c6-993b-4fce-ba67-4079095656cd None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:51.411257 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886c4c6-993b-4fce-ba67-4079095656cd None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:51.411532 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886c4c6-993b-4fce-ba67-4079095656cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:51.426462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bcc77-800d-45e4-9a23-029fa8ab7e69 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:51.427222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-6886c4c6-993b-4fce-ba67-4079095656cd None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:51.428405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-6886c4c6-993b-4fce-ba67-4079095656cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:51.461272 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892448e-a6aa-4f93-8739-4fa046d64487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:51.461919 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892448e-a6aa-4f93-8739-4fa046d64487 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:51.462204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892448e-a6aa-4f93-8739-4fa046d64487 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:51.462505 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892448e-a6aa-4f93-8739-4fa046d64487 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:51.474248 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:51.496994 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 96/193] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:51.497814 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886c4c6-993b-4fce-ba67-4079095656cd None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:51.499202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:51.541430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892448e-a6aa-4f93-8739-4fa046d64487 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:51.543251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892448e-a6aa-4f93-8739-4fa046d64487 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:51.606317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886c4c6-993b-4fce-ba67-4079095656cd None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:51.640831 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461ec47-ec0c-42aa-8e99-e34d0e1b13fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:51.641650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461ec47-ec0c-42aa-8e99-e34d0e1b13fb None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:51.641932 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461ec47-ec0c-42aa-8e99-e34d0e1b13fb None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:51.642210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461ec47-ec0c-42aa-8e99-e34d0e1b13fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:51.643442 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 98/194] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:03:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:03:51.644280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892448e-a6aa-4f93-8739-4fa046d64487 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:51.653043 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461ec47-ec0c-42aa-8e99-e34d0e1b13fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-e461ec47-ec0c-42aa-8e99-e34d0e1b13fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:51.661210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:51.662250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:51.663187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:51.663187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:51.663187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:51.663187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:51.663187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:51.663187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:51.663187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:51.663187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:51.663187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:51.663187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:51.663187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:03:51.663187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:51.663187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 97/195] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:03:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:51.663187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461ec47-ec0c-42aa-8e99-e34d0e1b13fb None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:51.714609 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:51.717470 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:51.752744 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892448e-a6aa-4f93-8739-4fa046d64487 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:51.775334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461ec47-ec0c-42aa-8e99-e34d0e1b13fb None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:51.788969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0299a48-ea55-48e0-a3c6-2eeb5e6d4332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:51.789878 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0299a48-ea55-48e0-a3c6-2eeb5e6d4332 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:51.790162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0299a48-ea55-48e0-a3c6-2eeb5e6d4332 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:51.790469 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0299a48-ea55-48e0-a3c6-2eeb5e6d4332 None admin] PATH_INFO: `/v3/users/125eb65567a84b35b5abca76fa44eca5` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:51.791717 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0299a48-ea55-48e0-a3c6-2eeb5e6d4332 None admin] RBAC: Authorizing `identity:get_user(user_id=125eb65567a84b35b5abca76fa44eca5)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:51.793311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0299a48-ea55-48e0-a3c6-2eeb5e6d4332 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:51.794996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 99/196] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:51 2022] GET /identity/v3/users/125eb65567a84b35b5abca76fa44eca5 => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:51.797939 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0299a48-ea55-48e0-a3c6-2eeb5e6d4332 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:51.811554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbc5b5-ff53-46a1-a6c6-884e490de361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:51.812386 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbc5b5-ff53-46a1-a6c6-884e490de361 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:51.812662 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbc5b5-ff53-46a1-a6c6-884e490de361 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:51.812935 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbc5b5-ff53-46a1-a6c6-884e490de361 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:51.814220 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbc5b5-ff53-46a1-a6c6-884e490de361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:51.815760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbc5b5-ff53-46a1-a6c6-884e490de361 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:51.817394 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 98/197] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:03:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:51.818157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbc5b5-ff53-46a1-a6c6-884e490de361 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:51.899046 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0299a48-ea55-48e0-a3c6-2eeb5e6d4332 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:51.923565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbc5b5-ff53-46a1-a6c6-884e490de361 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:51.932620 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb21f2-920a-48ff-9100-559cdb57bee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:51.933915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:51.934066 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb21f2-920a-48ff-9100-559cdb57bee3 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:51.934229 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb21f2-920a-48ff-9100-559cdb57bee3 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:51.934524 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb21f2-920a-48ff-9100-559cdb57bee3 None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:51.935817 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb21f2-920a-48ff-9100-559cdb57bee3 None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:51.937377 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb21f2-920a-48ff-9100-559cdb57bee3 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:51.939366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 100/198] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:51 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:51.940055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb21f2-920a-48ff-9100-559cdb57bee3 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:51.945261 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:51.962072 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ee2c3-6758-4955-aa9a-5f1a895c1299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:51.962794 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ee2c3-6758-4955-aa9a-5f1a895c1299 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:51.963081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ee2c3-6758-4955-aa9a-5f1a895c1299 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:51.963360 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ee2c3-6758-4955-aa9a-5f1a895c1299 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:51.964117 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ee2c3-6758-4955-aa9a-5f1a895c1299 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:51.964676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ee2c3-6758-4955-aa9a-5f1a895c1299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:51.966092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ee2c3-6758-4955-aa9a-5f1a895c1299 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:51.974733 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 99/199] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:03:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:51.975556 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ee2c3-6758-4955-aa9a-5f1a895c1299 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:52.048392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb21f2-920a-48ff-9100-559cdb57bee3 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:52.077988 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ee2c3-6758-4955-aa9a-5f1a895c1299 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:52.083068 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7fd21-5f3c-4f5e-8520-203627f150a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:52.083729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7fd21-5f3c-4f5e-8520-203627f150a9 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:52.084009 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7fd21-5f3c-4f5e-8520-203627f150a9 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:52.084278 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7fd21-5f3c-4f5e-8520-203627f150a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:52.113258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc61d3a-26c7-421e-af43-6a48f873617d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:52.113952 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc61d3a-26c7-421e-af43-6a48f873617d None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:52.114237 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc61d3a-26c7-421e-af43-6a48f873617d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:52.114543 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc61d3a-26c7-421e-af43-6a48f873617d None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:52.115273 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc61d3a-26c7-421e-af43-6a48f873617d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:52.116755 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc61d3a-26c7-421e-af43-6a48f873617d None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:52.137313 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 100/200] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:03:51 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:03:52.138142 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc61d3a-26c7-421e-af43-6a48f873617d None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:52.158586 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7fd21-5f3c-4f5e-8520-203627f150a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:52.160159 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7fd21-5f3c-4f5e-8520-203627f150a9 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:52.162838 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:52.173825 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:52.259072 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc61d3a-26c7-421e-af43-6a48f873617d None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:52.279446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 101/201] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:03:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:03:52.285091 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7fd21-5f3c-4f5e-8520-203627f150a9 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:52.299327 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7f354-1f9d-4770-a98b-e1a03cda3a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:52.300007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7f354-1f9d-4770-a98b-e1a03cda3a4e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:52.300293 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7f354-1f9d-4770-a98b-e1a03cda3a4e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:52.300572 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7f354-1f9d-4770-a98b-e1a03cda3a4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:52.301429 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7f354-1f9d-4770-a98b-e1a03cda3a4e None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=bb83c47ec18443db80a5cf2ed51d142e, user.id=b68ff2aacdf048ccb1e53d0b9e33de00) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:52.301746 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7f354-1f9d-4770-a98b-e1a03cda3a4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:52.303270 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7f354-1f9d-4770-a98b-e1a03cda3a4e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:52.309742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 101/202] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:03:52 2022] GET /identity/v3/role_assignments?user.id=b68ff2aacdf048ccb1e53d0b9e33de00&scope.project.id=bb83c47ec18443db80a5cf2ed51d142e&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:52.311251 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7f354-1f9d-4770-a98b-e1a03cda3a4e None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:52.424602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7fd21-5f3c-4f5e-8520-203627f150a9 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:52.429683 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:52.444629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:52.456759 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7f354-1f9d-4770-a98b-e1a03cda3a4e None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:52.486746 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bc32b-e423-49cd-8bcb-ba6a2e48ec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:52.487741 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bc32b-e423-49cd-8bcb-ba6a2e48ec44 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:52.488073 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bc32b-e423-49cd-8bcb-ba6a2e48ec44 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:52.488636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bc32b-e423-49cd-8bcb-ba6a2e48ec44 None admin] PATH_INFO: `/v3/users/b68ff2aacdf048ccb1e53d0b9e33de00` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:52.490823 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bc32b-e423-49cd-8bcb-ba6a2e48ec44 None admin] RBAC: Authorizing `identity:get_user(user_id=b68ff2aacdf048ccb1e53d0b9e33de00)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:52.492336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bc32b-e423-49cd-8bcb-ba6a2e48ec44 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:52.498394 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 102/203] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:52 2022] GET /identity/v3/users/b68ff2aacdf048ccb1e53d0b9e33de00 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:52.500474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bc32b-e423-49cd-8bcb-ba6a2e48ec44 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:52.501414 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4468d9e-38ad-4c05-8cdd-a5e08545eaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:52.502367 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4468d9e-38ad-4c05-8cdd-a5e08545eaed None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:52.502643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4468d9e-38ad-4c05-8cdd-a5e08545eaed None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:52.502921 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4468d9e-38ad-4c05-8cdd-a5e08545eaed None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:52.504537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4468d9e-38ad-4c05-8cdd-a5e08545eaed None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:52.507220 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4468d9e-38ad-4c05-8cdd-a5e08545eaed None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:52.509204 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 102/204] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:52 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:52.517069 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4468d9e-38ad-4c05-8cdd-a5e08545eaed None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:52.640638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bc32b-e423-49cd-8bcb-ba6a2e48ec44 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:52.655318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4468d9e-38ad-4c05-8cdd-a5e08545eaed None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:52.699932 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e6caa-62c4-4b58-9311-2f9f4ad739f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:52.700527 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e6caa-62c4-4b58-9311-2f9f4ad739f6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:52.700800 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e6caa-62c4-4b58-9311-2f9f4ad739f6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:52.701071 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e6caa-62c4-4b58-9311-2f9f4ad739f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:52.717544 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b269cca-a004-49e0-8825-c99ef28f237f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:52.718227 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b269cca-a004-49e0-8825-c99ef28f237f None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:52.718544 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b269cca-a004-49e0-8825-c99ef28f237f None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:52.718826 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b269cca-a004-49e0-8825-c99ef28f237f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:52.720003 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b269cca-a004-49e0-8825-c99ef28f237f None admin] RBAC: Adding query filter params (role.id=609bb86f2a5b44b0b244b9bfe7c073de, scope.project.id=e490252619914b5d80405cb54917237f, user.id=125eb65567a84b35b5abca76fa44eca5) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:52.720323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b269cca-a004-49e0-8825-c99ef28f237f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:52.721795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b269cca-a004-49e0-8825-c99ef28f237f None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:52.728834 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 103/205] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:03:52 2022] GET /identity/v3/role_assignments?user.id=125eb65567a84b35b5abca76fa44eca5&scope.project.id=e490252619914b5d80405cb54917237f&role.id=609bb86f2a5b44b0b244b9bfe7c073de => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:52.792174 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:52.854429 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e6caa-62c4-4b58-9311-2f9f4ad739f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:52.856283 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e6caa-62c4-4b58-9311-2f9f4ad739f6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:52.995630 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 103/206] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:03:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:03:53.001080 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b269cca-a004-49e0-8825-c99ef28f237f None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:53.031378 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:53.121435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b269cca-a004-49e0-8825-c99ef28f237f None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:53.164738 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72f3e3-4516-4749-8f01-ec2ba5010fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:53.166016 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72f3e3-4516-4749-8f01-ec2ba5010fdf None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:53.166541 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72f3e3-4516-4749-8f01-ec2ba5010fdf None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:53.167053 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72f3e3-4516-4749-8f01-ec2ba5010fdf None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:53.168620 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72f3e3-4516-4749-8f01-ec2ba5010fdf None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:53.170350 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72f3e3-4516-4749-8f01-ec2ba5010fdf None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:53.175977 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 104/207] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:53 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:53.179422 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e6caa-62c4-4b58-9311-2f9f4ad739f6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:53.260901 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:53.296312 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e6caa-62c4-4b58-9311-2f9f4ad739f6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:53.334969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4316a-470a-49ff-9d5d-218192d01911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:53.336063 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4316a-470a-49ff-9d5d-218192d01911 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:53.336552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4316a-470a-49ff-9d5d-218192d01911 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:53.337031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4316a-470a-49ff-9d5d-218192d01911 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:53.338097 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4316a-470a-49ff-9d5d-218192d01911 None admin] RBAC: Adding query filter params (role.id=b21c1004651d40f1b44fb0b7becbe3c8, scope.project.id=bb83c47ec18443db80a5cf2ed51d142e, user.id=b68ff2aacdf048ccb1e53d0b9e33de00) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:53.338650 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4316a-470a-49ff-9d5d-218192d01911 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:53.340364 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4316a-470a-49ff-9d5d-218192d01911 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:53.354441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 104/208] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:03:53 2022] GET /identity/v3/role_assignments?user.id=b68ff2aacdf048ccb1e53d0b9e33de00&scope.project.id=bb83c47ec18443db80a5cf2ed51d142e&role.id=b21c1004651d40f1b44fb0b7becbe3c8 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:54.719686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a3a40-ab47-4801-8a90-ce6571d0e163 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:54.720249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a3a40-ab47-4801-8a90-ce6571d0e163 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:54.720249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a3a40-ab47-4801-8a90-ce6571d0e163 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:54.722025 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 105/209] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:54.725176 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aea318-cc57-4890-b84c-333f1ceccd14 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:54.726104 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aea318-cc57-4890-b84c-333f1ceccd14 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:54.726652 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aea318-cc57-4890-b84c-333f1ceccd14 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:54.728546 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa45443-f714-4a4f-9034-ec6a8ec2bdb1 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:54.728805 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa45443-f714-4a4f-9034-ec6a8ec2bdb1 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:54.728911 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 105/210] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:54.729096 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa45443-f714-4a4f-9034-ec6a8ec2bdb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:54.735962 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b76485-bc53-45c2-be35-dcd3bd343dbd None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:54.736617 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b76485-bc53-45c2-be35-dcd3bd343dbd None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:54.736962 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b76485-bc53-45c2-be35-dcd3bd343dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:54.751273 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-5aa45443-f714-4a4f-9034-ec6a8ec2bdb1 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:54.753184 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-5aa45443-f714-4a4f-9034-ec6a8ec2bdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:54.757242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-40b76485-bc53-45c2-be35-dcd3bd343dbd None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:54.757783 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-40b76485-bc53-45c2-be35-dcd3bd343dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:54.836848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 106/211] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:03:54.843798 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 106/212] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:03:54.867307 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa45443-f714-4a4f-9034-ec6a8ec2bdb1 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:54.867885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b76485-bc53-45c2-be35-dcd3bd343dbd None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:54.892161 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:54.943809 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:54.981800 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa45443-f714-4a4f-9034-ec6a8ec2bdb1 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:54.983020 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b76485-bc53-45c2-be35-dcd3bd343dbd None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:55.019293 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6e313-5002-49df-98f3-4cd7ba7dfea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:55.019872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6e313-5002-49df-98f3-4cd7ba7dfea9 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:55.020157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6e313-5002-49df-98f3-4cd7ba7dfea9 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:55.020469 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6e313-5002-49df-98f3-4cd7ba7dfea9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:55.024738 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453b74d-374a-4747-afb8-a4a10ec8f3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:55.025553 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453b74d-374a-4747-afb8-a4a10ec8f3bf None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:55.025834 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453b74d-374a-4747-afb8-a4a10ec8f3bf None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:55.026110 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453b74d-374a-4747-afb8-a4a10ec8f3bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:55.032902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6e313-5002-49df-98f3-4cd7ba7dfea9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:55.037509 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453b74d-374a-4747-afb8-a4a10ec8f3bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-0da6e313-5002-49df-98f3-4cd7ba7dfea9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:55.040574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:55.041525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:55.042590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:55.042590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:55.042590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:55.042590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:55.042590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:55.042590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:55.042590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:55.042590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:55.042590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:55.042590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:55.042590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:55.042590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:55.042590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 107/213] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:03:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:55.043506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c741a0-d9a0-4941-8870-bc3cbbda6335 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:55.044082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c741a0-d9a0-4941-8870-bc3cbbda6335 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:55.044082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c741a0-d9a0-4941-8870-bc3cbbda6335 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:55.045315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 108/214] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-5453b74d-374a-4747-afb8-a4a10ec8f3bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:55.050036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:55.051244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:55.052145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:55.052145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:55.052145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:55.052145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:55.052145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:55.052145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:55.052145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:55.052145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:55.052145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:55.052145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:55.052145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:03:55.052145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:03:55.052821 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c741a0-d9a0-4941-8870-bc3cbbda6335 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:55.052899 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 107/215] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:03:55.052899 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f4e9e-b7b1-4f4c-a4ba-3bc8043b7462 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:55.052899 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f4e9e-b7b1-4f4c-a4ba-3bc8043b7462 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:55.053103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f4e9e-b7b1-4f4c-a4ba-3bc8043b7462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:55.073343 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-fa2f4e9e-b7b1-4f4c-a4ba-3bc8043b7462 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:55.075874 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-fa2f4e9e-b7b1-4f4c-a4ba-3bc8043b7462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:55.143905 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:55.158288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 108/216] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:55.158288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f4e9e-b7b1-4f4c-a4ba-3bc8043b7462 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:55.178127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c741a0-d9a0-4941-8870-bc3cbbda6335 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:55.216549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38724e-2d62-4911-848d-1c026c9718fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:55.217233 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38724e-2d62-4911-848d-1c026c9718fe None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:55.217846 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38724e-2d62-4911-848d-1c026c9718fe None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:55.218057 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38724e-2d62-4911-848d-1c026c9718fe None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:55.218685 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f38724e-2d62-4911-848d-1c026c9718fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:55.219009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f38724e-2d62-4911-848d-1c026c9718fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:55.220866 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:55.221437 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f38724e-2d62-4911-848d-1c026c9718fe None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:55.231077 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 109/217] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:55.231834 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38724e-2d62-4911-848d-1c026c9718fe None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:55.273207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f4e9e-b7b1-4f4c-a4ba-3bc8043b7462 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:55.312228 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005b3e9-6cf8-4875-955a-32a0cccf458e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:55.312905 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005b3e9-6cf8-4875-955a-32a0cccf458e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:55.313186 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005b3e9-6cf8-4875-955a-32a0cccf458e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:55.313459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005b3e9-6cf8-4875-955a-32a0cccf458e None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:55.314223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005b3e9-6cf8-4875-955a-32a0cccf458e None admin] RBAC: Adding query filter params (name=member) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:55.314558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005b3e9-6cf8-4875-955a-32a0cccf458e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:55.315973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005b3e9-6cf8-4875-955a-32a0cccf458e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:55.329180 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 109/218] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:03:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:55.329180 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005b3e9-6cf8-4875-955a-32a0cccf458e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:55.349884 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38724e-2d62-4911-848d-1c026c9718fe None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:55.390253 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb658b-f66c-43cb-9457-aeb8c49a54ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:55.391091 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb658b-f66c-43cb-9457-aeb8c49a54ea None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:55.391378 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb658b-f66c-43cb-9457-aeb8c49a54ea None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:55.391656 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb658b-f66c-43cb-9457-aeb8c49a54ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:55.392862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb658b-f66c-43cb-9457-aeb8c49a54ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:55.394421 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb658b-f66c-43cb-9457-aeb8c49a54ea None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:55.396006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 110/219] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:03:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:55.396767 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb658b-f66c-43cb-9457-aeb8c49a54ea None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:55.413178 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:55.448109 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005b3e9-6cf8-4875-955a-32a0cccf458e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:55.466193 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:55.486552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ade2b8-b0bd-453e-ac96-4214800a1ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:55.487445 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ade2b8-b0bd-453e-ac96-4214800a1ede None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:55.487722 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ade2b8-b0bd-453e-ac96-4214800a1ede None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:55.487991 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ade2b8-b0bd-453e-ac96-4214800a1ede None admin] PATH_INFO: `/v3/users/b68ff2aacdf048ccb1e53d0b9e33de00` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:55.490058 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ade2b8-b0bd-453e-ac96-4214800a1ede None admin] RBAC: Authorizing `identity:get_user(user_id=b68ff2aacdf048ccb1e53d0b9e33de00)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:55.491566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ade2b8-b0bd-453e-ac96-4214800a1ede None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:55.497774 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 110/220] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:55 2022] GET /identity/v3/users/b68ff2aacdf048ccb1e53d0b9e33de00 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:55.497774 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ade2b8-b0bd-453e-ac96-4214800a1ede None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:55.512499 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb658b-f66c-43cb-9457-aeb8c49a54ea None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:55.550434 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bc66f-28e4-4111-b0f7-f1855fe0af1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:55.551332 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bc66f-28e4-4111-b0f7-f1855fe0af1c None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:55.551616 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bc66f-28e4-4111-b0f7-f1855fe0af1c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:55.551890 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bc66f-28e4-4111-b0f7-f1855fe0af1c None admin] PATH_INFO: `/v3/users/28658090be4845cb9ab0220fc87a5812` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:55.565561 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bc66f-28e4-4111-b0f7-f1855fe0af1c None admin] RBAC: Authorizing `identity:get_user(user_id=28658090be4845cb9ab0220fc87a5812)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:55.567098 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bc66f-28e4-4111-b0f7-f1855fe0af1c None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:55.568778 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 111/221] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:55 2022] GET /identity/v3/users/28658090be4845cb9ab0220fc87a5812 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:55.569548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bc66f-28e4-4111-b0f7-f1855fe0af1c None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:55.598487 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ade2b8-b0bd-453e-ac96-4214800a1ede None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:55.631191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032679ad-18dd-46c2-878e-260bb3ee2a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:55.632087 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032679ad-18dd-46c2-878e-260bb3ee2a8b None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:55.632366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032679ad-18dd-46c2-878e-260bb3ee2a8b None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:55.632643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032679ad-18dd-46c2-878e-260bb3ee2a8b None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:55.633362 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032679ad-18dd-46c2-878e-260bb3ee2a8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:55.634747 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:55.634867 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032679ad-18dd-46c2-878e-260bb3ee2a8b None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:55.655610 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 111/222] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:03:55 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:03:55.656720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c1e3c-c9e3-4e50-a249-958b844b3184 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:55.657009 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c1e3c-c9e3-4e50-a249-958b844b3184 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:55.657523 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c1e3c-c9e3-4e50-a249-958b844b3184 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:55.658668 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 112/223] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:55.659488 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c1e3c-c9e3-4e50-a249-958b844b3184 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:55.683796 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bc66f-28e4-4111-b0f7-f1855fe0af1c None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:55.719656 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:55.724083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac845696-004b-4ede-bd0c-97681818d213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:55.724971 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac845696-004b-4ede-bd0c-97681818d213 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:55.725271 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac845696-004b-4ede-bd0c-97681818d213 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:55.725511 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac845696-004b-4ede-bd0c-97681818d213 None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:55.726887 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac845696-004b-4ede-bd0c-97681818d213 None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:55.728399 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac845696-004b-4ede-bd0c-97681818d213 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:55.730432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 112/224] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:55 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:55.736669 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c3f7f-be74-4e33-ae89-ef048ef6c7b4 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:55.737384 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c3f7f-be74-4e33-ae89-ef048ef6c7b4 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:55.737654 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c3f7f-be74-4e33-ae89-ef048ef6c7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:55.755908 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-f88c3f7f-be74-4e33-ae89-ef048ef6c7b4 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:55.757173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-f88c3f7f-be74-4e33-ae89-ef048ef6c7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:55.806185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c1e3c-c9e3-4e50-a249-958b844b3184 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:55.838909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 113/225] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:55.840221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c3f7f-be74-4e33-ae89-ef048ef6c7b4 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:55.852649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a714734-0deb-495b-8b53-df8b89362f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:55.853540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a714734-0deb-495b-8b53-df8b89362f59 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:55.853974 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a714734-0deb-495b-8b53-df8b89362f59 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:55.854432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a714734-0deb-495b-8b53-df8b89362f59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:55.922530 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:55.938367 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a714734-0deb-495b-8b53-df8b89362f59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:55.940377 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a714734-0deb-495b-8b53-df8b89362f59 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:55.974420 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c3f7f-be74-4e33-ae89-ef048ef6c7b4 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:55.997550 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:56.018569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3db4d7-7026-416d-b2cf-703928182af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:56.019827 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3db4d7-7026-416d-b2cf-703928182af2 None admin] REQUEST_METHOD: `PUT` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:56.019827 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3db4d7-7026-416d-b2cf-703928182af2 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:56.020013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3db4d7-7026-416d-b2cf-703928182af2 None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e/users/b68ff2aacdf048ccb1e53d0b9e33de00/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:56.022437 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3db4d7-7026-416d-b2cf-703928182af2 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb83c47ec18443db80a5cf2ed51d142e, user_id=b68ff2aacdf048ccb1e53d0b9e33de00, role_id=b34e4ad4fe7b4a60bd2fc669e2b31a66)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:56.023580 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3db4d7-7026-416d-b2cf-703928182af2 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:56.038936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 114/226] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:03:55 2022] PUT /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e/users/b68ff2aacdf048ccb1e53d0b9e33de00/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:03:56.039964 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3db4d7-7026-416d-b2cf-703928182af2 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:56.076803 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 113/227] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:03:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:03:56.083068 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a714734-0deb-495b-8b53-df8b89362f59 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:56.163035 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3db4d7-7026-416d-b2cf-703928182af2 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:56.192749 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:56.199498 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a714734-0deb-495b-8b53-df8b89362f59 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:56.203232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6ce35-9325-4864-82e5-d64d9840fea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:56.204127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6ce35-9325-4864-82e5-d64d9840fea2 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:56.204708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6ce35-9325-4864-82e5-d64d9840fea2 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:56.205049 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6ce35-9325-4864-82e5-d64d9840fea2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:56.218344 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6ce35-9325-4864-82e5-d64d9840fea2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-3cf6ce35-9325-4864-82e5-d64d9840fea2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:56.228328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:56.229387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:56.230389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:56.230389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:56.230389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:56.230389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:56.230389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:56.230389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:56.230389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:56.230389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:56.230389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:56.230389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:56.230389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:03:56.230389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:56.230389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 115/228] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:03:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:56.239650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6ce35-9325-4864-82e5-d64d9840fea2 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:56.245629 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb252da-9481-4e24-b862-41a0153c0b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:56.246798 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb252da-9481-4e24-b862-41a0153c0b90 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:56.247078 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb252da-9481-4e24-b862-41a0153c0b90 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:56.247351 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb252da-9481-4e24-b862-41a0153c0b90 None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:56.248573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb252da-9481-4e24-b862-41a0153c0b90 None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:56.250116 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb252da-9481-4e24-b862-41a0153c0b90 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:56.256992 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 114/229] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:56 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:56.258236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb252da-9481-4e24-b862-41a0153c0b90 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:56.270886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:56.377775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb252da-9481-4e24-b862-41a0153c0b90 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:56.426914 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6ce35-9325-4864-82e5-d64d9840fea2 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:56.429714 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8469de1-540d-46d3-acd6-6a6073bb73b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:56.430566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8469de1-540d-46d3-acd6-6a6073bb73b3 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:56.430852 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8469de1-540d-46d3-acd6-6a6073bb73b3 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:56.431130 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8469de1-540d-46d3-acd6-6a6073bb73b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:56.431991 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8469de1-540d-46d3-acd6-6a6073bb73b3 None admin] RBAC: Adding query filter params (role.id=6aee6f5679c84fc1be21561ddb7d1b56, scope.project.id=bb83c47ec18443db80a5cf2ed51d142e, user.id=28658090be4845cb9ab0220fc87a5812) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:56.432309 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8469de1-540d-46d3-acd6-6a6073bb73b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:56.435331 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8469de1-540d-46d3-acd6-6a6073bb73b3 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:56.446520 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 115/230] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:03:56 2022] GET /identity/v3/role_assignments?user.id=28658090be4845cb9ab0220fc87a5812&scope.project.id=bb83c47ec18443db80a5cf2ed51d142e&role.id=6aee6f5679c84fc1be21561ddb7d1b56 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:56.466953 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48983ff-74bc-4278-b6cd-d04986c81419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:56.467696 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48983ff-74bc-4278-b6cd-d04986c81419 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:56.468057 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48983ff-74bc-4278-b6cd-d04986c81419 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:56.468339 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48983ff-74bc-4278-b6cd-d04986c81419 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:56.469528 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48983ff-74bc-4278-b6cd-d04986c81419 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:56.469840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48983ff-74bc-4278-b6cd-d04986c81419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:56.471299 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48983ff-74bc-4278-b6cd-d04986c81419 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:56.481691 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 116/231] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:03:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:56.487548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8469de1-540d-46d3-acd6-6a6073bb73b3 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:56.567845 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:56.602557 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8469de1-540d-46d3-acd6-6a6073bb73b3 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:56.638778 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65fe7a-59ab-46c5-9168-4f893d54b6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:56.639688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65fe7a-59ab-46c5-9168-4f893d54b6f6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:56.639967 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65fe7a-59ab-46c5-9168-4f893d54b6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:56.640235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65fe7a-59ab-46c5-9168-4f893d54b6f6 None admin] PATH_INFO: `/v3/users/b68ff2aacdf048ccb1e53d0b9e33de00` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:56.641507 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65fe7a-59ab-46c5-9168-4f893d54b6f6 None admin] RBAC: Authorizing `identity:get_user(user_id=b68ff2aacdf048ccb1e53d0b9e33de00)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:56.643035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65fe7a-59ab-46c5-9168-4f893d54b6f6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:56.645084 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 116/232] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:56 2022] GET /identity/v3/users/b68ff2aacdf048ccb1e53d0b9e33de00 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:56.650705 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48983ff-74bc-4278-b6cd-d04986c81419 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:56.706378 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:56.763686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48983ff-74bc-4278-b6cd-d04986c81419 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:56.798395 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac6ebd-7b67-41bd-9681-5978f5b6c2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:56.799284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac6ebd-7b67-41bd-9681-5978f5b6c2ae None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:56.799560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac6ebd-7b67-41bd-9681-5978f5b6c2ae None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:56.799828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac6ebd-7b67-41bd-9681-5978f5b6c2ae None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:56.801052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac6ebd-7b67-41bd-9681-5978f5b6c2ae None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:56.802544 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac6ebd-7b67-41bd-9681-5978f5b6c2ae None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:56.804567 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 117/233] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:56 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:56.810563 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65fe7a-59ab-46c5-9168-4f893d54b6f6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:56.916047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65fe7a-59ab-46c5-9168-4f893d54b6f6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:56.952797 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914b8f0-3451-4800-9d17-b36610a5a313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:56.953706 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914b8f0-3451-4800-9d17-b36610a5a313 None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:56.953981 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914b8f0-3451-4800-9d17-b36610a5a313 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:56.954253 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914b8f0-3451-4800-9d17-b36610a5a313 None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e/users/b68ff2aacdf048ccb1e53d0b9e33de00/roles/b21c1004651d40f1b44fb0b7becbe3c8` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:56.956561 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914b8f0-3451-4800-9d17-b36610a5a313 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb83c47ec18443db80a5cf2ed51d142e, user_id=b68ff2aacdf048ccb1e53d0b9e33de00, role_id=b21c1004651d40f1b44fb0b7becbe3c8)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:56.957970 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914b8f0-3451-4800-9d17-b36610a5a313 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:56.973650 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:56.979036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 117/234] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:03:56 2022] PUT /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e/users/b68ff2aacdf048ccb1e53d0b9e33de00/roles/b21c1004651d40f1b44fb0b7becbe3c8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:03:58.194486 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21069c69-29f9-48c4-b2b0-537ad5a6a19b None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:58.195063 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21069c69-29f9-48c4-b2b0-537ad5a6a19b None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:58.195063 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21069c69-29f9-48c4-b2b0-537ad5a6a19b None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:58.196419 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 118/235] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:58.201929 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802544e-a425-4ea6-a8a8-c2c02d092d31 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:58.202394 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802544e-a425-4ea6-a8a8-c2c02d092d31 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:58.202498 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802544e-a425-4ea6-a8a8-c2c02d092d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:58.223335 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-1802544e-a425-4ea6-a8a8-c2c02d092d31 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:58.224526 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-1802544e-a425-4ea6-a8a8-c2c02d092d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:58.316204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 118/236] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:58.363830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21069c69-29f9-48c4-b2b0-537ad5a6a19b None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:58.501661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21069c69-29f9-48c4-b2b0-537ad5a6a19b None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:58.522770 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:58.550280 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b86c28-a124-4f6c-95c9-e8b5dd0d80ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:58.551121 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b86c28-a124-4f6c-95c9-e8b5dd0d80ba None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:58.551400 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b86c28-a124-4f6c-95c9-e8b5dd0d80ba None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:58.551671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b86c28-a124-4f6c-95c9-e8b5dd0d80ba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:58.575195 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b86c28-a124-4f6c-95c9-e8b5dd0d80ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-65b86c28-a124-4f6c-95c9-e8b5dd0d80ba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:58.583254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:58.584258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:58.585146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:58.585146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:58.585146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:58.585146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:58.585146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:58.585146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:58.585146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:58.585146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:58.585146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:58.585146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:58.585146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:03:58.585146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:58.587626 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 119/237] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:58.595630 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802544e-a425-4ea6-a8a8-c2c02d092d31 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:58.607257 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be0b11-b077-4cbd-8588-34011c1c721b None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:58.607878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be0b11-b077-4cbd-8588-34011c1c721b None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:58.608381 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be0b11-b077-4cbd-8588-34011c1c721b None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:58.610802 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 120/238] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:03:58.615542 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5367d-ab90-41e1-8005-149688bcf8e5 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:58.616546 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5367d-ab90-41e1-8005-149688bcf8e5 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:58.617050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5367d-ab90-41e1-8005-149688bcf8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:58.649469 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-91a5367d-ab90-41e1-8005-149688bcf8e5 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:58.653067 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-91a5367d-ab90-41e1-8005-149688bcf8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:58.777027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802544e-a425-4ea6-a8a8-c2c02d092d31 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:58.800808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 121/239] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:58.809654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:58.824806 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c36b8-d45b-4424-a9ca-c63d3b36e58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:58.825883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c36b8-d45b-4424-a9ca-c63d3b36e58a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:58.825999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5367d-ab90-41e1-8005-149688bcf8e5 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:58.826552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c36b8-d45b-4424-a9ca-c63d3b36e58a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:58.827061 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c36b8-d45b-4424-a9ca-c63d3b36e58a None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:58.828157 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c36b8-d45b-4424-a9ca-c63d3b36e58a None admin] RBAC: Adding query filter params (name=reader) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:58.828672 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c36b8-d45b-4424-a9ca-c63d3b36e58a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:58.831157 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c36b8-d45b-4424-a9ca-c63d3b36e58a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:58.849805 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 119/240] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:03:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:58.856529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c36b8-d45b-4424-a9ca-c63d3b36e58a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:59.017795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5367d-ab90-41e1-8005-149688bcf8e5 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:59.032396 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:59.050758 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c36b8-d45b-4424-a9ca-c63d3b36e58a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:59.077526 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4c053-fb8b-45f5-84be-98ccb54af50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:59.078624 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4c053-fb8b-45f5-84be-98ccb54af50e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:59.079316 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4c053-fb8b-45f5-84be-98ccb54af50e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:59.079603 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4c053-fb8b-45f5-84be-98ccb54af50e None admin] PATH_INFO: `/v3/domains/default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:59.081320 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4c053-fb8b-45f5-84be-98ccb54af50e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:59.083552 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4c053-fb8b-45f5-84be-98ccb54af50e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:59.085567 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 122/241] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:03:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:59.086877 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e52f76-9473-40a5-bca7-7b1bf1af9d5f None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:59.087170 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e52f76-9473-40a5-bca7-7b1bf1af9d5f None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:59.087451 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e52f76-9473-40a5-bca7-7b1bf1af9d5f None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:59.092479 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc469bb-2408-41b3-8337-096324024700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:59.093455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc469bb-2408-41b3-8337-096324024700 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:59.093646 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 123/242] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:59.093796 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc469bb-2408-41b3-8337-096324024700 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:59.094491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc469bb-2408-41b3-8337-096324024700 None admin] PATH_INFO: `/v3/users/fcf21c2104594c4f85c6557e2a6dafc6` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:59.094783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e52f76-9473-40a5-bca7-7b1bf1af9d5f None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:59.110666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc469bb-2408-41b3-8337-096324024700 None admin] RBAC: Authorizing `identity:get_user(user_id=fcf21c2104594c4f85c6557e2a6dafc6)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:59.112305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc469bb-2408-41b3-8337-096324024700 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:59.114419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 120/243] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:03:58 2022] GET /identity/v3/users/fcf21c2104594c4f85c6557e2a6dafc6 => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:59.115717 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da7f53-c6c9-49a6-9a00-4f899fed4b7e None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:59.120555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da7f53-c6c9-49a6-9a00-4f899fed4b7e None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:59.120927 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da7f53-c6c9-49a6-9a00-4f899fed4b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:59.138434 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-a4da7f53-c6c9-49a6-9a00-4f899fed4b7e None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:59.139700 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-a4da7f53-c6c9-49a6-9a00-4f899fed4b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:59.202461 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:59.225966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e52f76-9473-40a5-bca7-7b1bf1af9d5f None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:59.233753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 121/244] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:59.239131 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da7f53-c6c9-49a6-9a00-4f899fed4b7e None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:59.280968 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae720a01-24ef-40af-8a5a-e88062272662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:59.282182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae720a01-24ef-40af-8a5a-e88062272662 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:59.282717 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae720a01-24ef-40af-8a5a-e88062272662 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:59.283611 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae720a01-24ef-40af-8a5a-e88062272662 None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:59.284897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae720a01-24ef-40af-8a5a-e88062272662 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:59.287281 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae720a01-24ef-40af-8a5a-e88062272662 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:59.315635 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 124/245] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:03:59 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:03:59.323131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae720a01-24ef-40af-8a5a-e88062272662 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:59.387681 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:59.422113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da7f53-c6c9-49a6-9a00-4f899fed4b7e None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:59.489983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae720a01-24ef-40af-8a5a-e88062272662 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:59.490943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ef12e-bdae-415a-a2c7-e035484c07e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:59.491793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ef12e-bdae-415a-a2c7-e035484c07e4 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:59.492317 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ef12e-bdae-415a-a2c7-e035484c07e4 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:59.492929 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ef12e-bdae-415a-a2c7-e035484c07e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:59.538871 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1edb72-021c-409f-a6bc-b78f5076d093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:59.539621 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1edb72-021c-409f-a6bc-b78f5076d093 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:59.539901 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1edb72-021c-409f-a6bc-b78f5076d093 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:59.540176 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1edb72-021c-409f-a6bc-b78f5076d093 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:59.555673 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:59.563444 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1edb72-021c-409f-a6bc-b78f5076d093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-2b1edb72-021c-409f-a6bc-b78f5076d093 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:03:59.573880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:03:59.578902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:03:59.580130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:03:59.580130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:03:59.580130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:03:59.580130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:03:59.580130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:03:59.580130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:03:59.580130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:03:59.580130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:03:59.580130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:03:59.580130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:03:59.580130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:03:59.580130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:03:59.580130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 125/246] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:03:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:03:59.580130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3e8f1-0c2f-44af-8411-4f12d58a2f5d None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:59.581101 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3e8f1-0c2f-44af-8411-4f12d58a2f5d None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:59.581101 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3e8f1-0c2f-44af-8411-4f12d58a2f5d None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:59.581101 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 126/247] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:03:59.584404 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061f9f0-6405-4427-ab2a-2a9f15290360 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:59.584692 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061f9f0-6405-4427-ab2a-2a9f15290360 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:59.584967 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061f9f0-6405-4427-ab2a-2a9f15290360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:59.591137 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ef12e-bdae-415a-a2c7-e035484c07e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:59.593603 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ef12e-bdae-415a-a2c7-e035484c07e4 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:59.606518 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-a061f9f0-6405-4427-ab2a-2a9f15290360 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:03:59.607718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-a061f9f0-6405-4427-ab2a-2a9f15290360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:03:59.705782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 127/248] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:03:59.707303 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061f9f0-6405-4427-ab2a-2a9f15290360 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:59.718442 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 122/249] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:03:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:03:59.725035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ef12e-bdae-415a-a2c7-e035484c07e4 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:59.736769 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:59.834917 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:59.842639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061f9f0-6405-4427-ab2a-2a9f15290360 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:59.861178 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ef12e-bdae-415a-a2c7-e035484c07e4 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:03:59.883124 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4146bc5-17ee-479c-885c-83680b1583c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:59.884236 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4146bc5-17ee-479c-885c-83680b1583c4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:59.884720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4146bc5-17ee-479c-885c-83680b1583c4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:59.885214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4146bc5-17ee-479c-885c-83680b1583c4 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:59.886439 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4146bc5-17ee-479c-885c-83680b1583c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:03:59.886957 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4146bc5-17ee-479c-885c-83680b1583c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:59.888691 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4146bc5-17ee-479c-885c-83680b1583c4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:59.897032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d049ae-30bf-4d0f-8409-77a0cb366bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:03:59.897922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d049ae-30bf-4d0f-8409-77a0cb366bff None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:03:59.898208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d049ae-30bf-4d0f-8409-77a0cb366bff None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:03:59.898505 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d049ae-30bf-4d0f-8409-77a0cb366bff None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:03:59.899613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 128/250] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:03:59.899741 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d049ae-30bf-4d0f-8409-77a0cb366bff None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:03:59.900819 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4146bc5-17ee-479c-885c-83680b1583c4 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:59.901199 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d049ae-30bf-4d0f-8409-77a0cb366bff None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:03:59.903392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 123/251] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:03:59 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:03:59.910040 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d049ae-30bf-4d0f-8409-77a0cb366bff None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:03:59.999759 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:00.020175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4146bc5-17ee-479c-885c-83680b1583c4 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:00.046704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d049ae-30bf-4d0f-8409-77a0cb366bff None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:00.055397 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90159166-84cc-49de-8a6f-1fb8696ca7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:00.056216 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90159166-84cc-49de-8a6f-1fb8696ca7f4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:00.056495 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90159166-84cc-49de-8a6f-1fb8696ca7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:00.056768 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90159166-84cc-49de-8a6f-1fb8696ca7f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:00.068092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90159166-84cc-49de-8a6f-1fb8696ca7f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-90159166-84cc-49de-8a6f-1fb8696ca7f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:00.075813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:00.076832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:00.077708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:00.077708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:00.077708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:00.077708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:00.077708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:00.077708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:00.077708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:00.077708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:00.077708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:00.077708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:00.077708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:00.077708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:00.077708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 129/252] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:03:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:00.084873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90159166-84cc-49de-8a6f-1fb8696ca7f4 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:00.093683 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb036c-4045-402e-b8ef-52ff5757bc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:00.095100 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb036c-4045-402e-b8ef-52ff5757bc5e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:00.095702 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb036c-4045-402e-b8ef-52ff5757bc5e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:00.096153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb036c-4045-402e-b8ef-52ff5757bc5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:00.097151 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb036c-4045-402e-b8ef-52ff5757bc5e None admin] RBAC: Adding query filter params (role.id=609bb86f2a5b44b0b244b9bfe7c073de, scope.project.id=bb83c47ec18443db80a5cf2ed51d142e, user.id=fcf21c2104594c4f85c6557e2a6dafc6) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:00.097615 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb036c-4045-402e-b8ef-52ff5757bc5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:00.099423 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb036c-4045-402e-b8ef-52ff5757bc5e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:00.106343 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 124/253] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:03:59 2022] GET /identity/v3/role_assignments?user.id=fcf21c2104594c4f85c6557e2a6dafc6&scope.project.id=bb83c47ec18443db80a5cf2ed51d142e&role.id=609bb86f2a5b44b0b244b9bfe7c073de => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:00.107650 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e7e42-da66-4b1d-9c3d-f09b0106838e None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:00.107939 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e7e42-da66-4b1d-9c3d-f09b0106838e None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:00.108211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e7e42-da66-4b1d-9c3d-f09b0106838e None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:00.109865 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 125/254] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:00.114891 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e7e42-da66-4b1d-9c3d-f09b0106838e None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:00.132040 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:00.268849 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e7e42-da66-4b1d-9c3d-f09b0106838e None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:00.305994 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90159166-84cc-49de-8a6f-1fb8696ca7f4 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:00.315528 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d2fc8-3514-4d5e-b498-c95684365adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:00.316626 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d2fc8-3514-4d5e-b498-c95684365adf None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:00.317140 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d2fc8-3514-4d5e-b498-c95684365adf None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:00.317611 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d2fc8-3514-4d5e-b498-c95684365adf None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:00.318644 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d2fc8-3514-4d5e-b498-c95684365adf None admin] RBAC: Adding query filter params (name=member) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:00.319198 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d2fc8-3514-4d5e-b498-c95684365adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:00.321470 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d2fc8-3514-4d5e-b498-c95684365adf None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:00.332776 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 126/255] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:00.338090 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e9f49-92bd-48be-b892-21c6383f240f None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:00.338090 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e9f49-92bd-48be-b892-21c6383f240f None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:00.338090 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e9f49-92bd-48be-b892-21c6383f240f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:00.358045 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-e71e9f49-92bd-48be-b892-21c6383f240f None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:00.359743 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-e71e9f49-92bd-48be-b892-21c6383f240f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:00.390873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781a3ed-872e-4536-b648-019866995cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:00.392893 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781a3ed-872e-4536-b648-019866995cbf None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:00.393182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781a3ed-872e-4536-b648-019866995cbf None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:00.393464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781a3ed-872e-4536-b648-019866995cbf None admin] PATH_INFO: `/v3/users/b68ff2aacdf048ccb1e53d0b9e33de00` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:00.398053 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781a3ed-872e-4536-b648-019866995cbf None admin] RBAC: Authorizing `identity:get_user(user_id=b68ff2aacdf048ccb1e53d0b9e33de00)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:00.399691 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781a3ed-872e-4536-b648-019866995cbf None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:00.402847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 130/256] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:04:00 2022] GET /identity/v3/users/b68ff2aacdf048ccb1e53d0b9e33de00 => generated 283 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:00.403106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781a3ed-872e-4536-b648-019866995cbf None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:00.435497 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:00.442462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 127/257] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:00.443598 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e9f49-92bd-48be-b892-21c6383f240f None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:00.448055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:00.557188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e9f49-92bd-48be-b892-21c6383f240f None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:00.596708 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6cb9e-4188-4d6e-8ec3-e43a98349f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:00.597894 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6cb9e-4188-4d6e-8ec3-e43a98349f59 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:00.598423 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6cb9e-4188-4d6e-8ec3-e43a98349f59 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:00.598918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6cb9e-4188-4d6e-8ec3-e43a98349f59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:00.645099 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781a3ed-872e-4536-b648-019866995cbf None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:00.671577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6cb9e-4188-4d6e-8ec3-e43a98349f59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:00.673467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6cb9e-4188-4d6e-8ec3-e43a98349f59 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:00.677103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:00.742985 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bdfb0-1a20-47b4-809c-f4c56f7d2129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:00.743897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bdfb0-1a20-47b4-809c-f4c56f7d2129 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:00.744193 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bdfb0-1a20-47b4-809c-f4c56f7d2129 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:00.744483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bdfb0-1a20-47b4-809c-f4c56f7d2129 None admin] PATH_INFO: `/v3/users/28658090be4845cb9ab0220fc87a5812` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:00.745964 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bdfb0-1a20-47b4-809c-f4c56f7d2129 None admin] RBAC: Authorizing `identity:get_user(user_id=28658090be4845cb9ab0220fc87a5812)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:00.748055 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bdfb0-1a20-47b4-809c-f4c56f7d2129 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:00.755528 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 131/258] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:04:00 2022] GET /identity/v3/users/28658090be4845cb9ab0220fc87a5812 => generated 297 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:00.755528 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bdfb0-1a20-47b4-809c-f4c56f7d2129 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:00.784765 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 128/259] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:00.786024 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6cb9e-4188-4d6e-8ec3-e43a98349f59 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:00.888706 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6cb9e-4188-4d6e-8ec3-e43a98349f59 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:00.910739 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:00.925783 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20b2f7-1000-45eb-a08a-3aecafb7dd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:00.927070 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20b2f7-1000-45eb-a08a-3aecafb7dd92 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:00.927562 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20b2f7-1000-45eb-a08a-3aecafb7dd92 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:00.928151 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20b2f7-1000-45eb-a08a-3aecafb7dd92 None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:00.929560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20b2f7-1000-45eb-a08a-3aecafb7dd92 None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:00.931262 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20b2f7-1000-45eb-a08a-3aecafb7dd92 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:00.933412 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 129/260] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:00 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:00.934476 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20b2f7-1000-45eb-a08a-3aecafb7dd92 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:00.972363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:01.026198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bdfb0-1a20-47b4-809c-f4c56f7d2129 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:01.097670 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a7795-7630-47b8-9961-458c974bbfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:01.098891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a7795-7630-47b8-9961-458c974bbfcd None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:01.100134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a7795-7630-47b8-9961-458c974bbfcd None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:01.100850 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a7795-7630-47b8-9961-458c974bbfcd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:01.112217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20b2f7-1000-45eb-a08a-3aecafb7dd92 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:01.116557 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a7795-7630-47b8-9961-458c974bbfcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-f73a7795-7630-47b8-9961-458c974bbfcd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:01.125966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:01.127024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:01.128007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:01.128007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:01.128007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:01.128007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:01.128007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:01.128007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:01.128007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:01.128007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:01.128007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:01.128007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:01.128007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:01.128007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:01.131250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 132/261] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:04:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:01.132559 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a7795-7630-47b8-9961-458c974bbfcd None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:01.179492 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8674eb5-56ff-4012-b31c-70cae366cd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:01.180851 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8674eb5-56ff-4012-b31c-70cae366cd24 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:01.181357 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8674eb5-56ff-4012-b31c-70cae366cd24 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:01.181864 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8674eb5-56ff-4012-b31c-70cae366cd24 None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:01.183430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8674eb5-56ff-4012-b31c-70cae366cd24 None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:01.185545 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8674eb5-56ff-4012-b31c-70cae366cd24 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:01.187615 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 130/262] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:00 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:01.188652 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8674eb5-56ff-4012-b31c-70cae366cd24 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:01.236686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:01.297411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8674eb5-56ff-4012-b31c-70cae366cd24 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:01.335086 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12331e-9bd6-4da4-815e-0e2d2478fe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:01.335927 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a7795-7630-47b8-9961-458c974bbfcd None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:01.336783 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12331e-9bd6-4da4-815e-0e2d2478fe7a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:01.337371 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12331e-9bd6-4da4-815e-0e2d2478fe7a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:01.337877 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12331e-9bd6-4da4-815e-0e2d2478fe7a None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:01.338877 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12331e-9bd6-4da4-815e-0e2d2478fe7a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:01.339459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12331e-9bd6-4da4-815e-0e2d2478fe7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:01.341101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12331e-9bd6-4da4-815e-0e2d2478fe7a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:01.350912 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 131/263] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:01.352059 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12331e-9bd6-4da4-815e-0e2d2478fe7a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:01.404049 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40861087-09c9-417f-bc88-d7abb18d89ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:01.405059 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40861087-09c9-417f-bc88-d7abb18d89ed None admin] REQUEST_METHOD: `PUT` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:01.405345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40861087-09c9-417f-bc88-d7abb18d89ed None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:01.405620 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40861087-09c9-417f-bc88-d7abb18d89ed None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e/users/28658090be4845cb9ab0220fc87a5812/roles/6aee6f5679c84fc1be21561ddb7d1b56` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:01.409463 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40861087-09c9-417f-bc88-d7abb18d89ed None admin] RBAC: Authorizing `identity:create_grant(project_id=bb83c47ec18443db80a5cf2ed51d142e, user_id=28658090be4845cb9ab0220fc87a5812, role_id=6aee6f5679c84fc1be21561ddb7d1b56)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:01.410962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40861087-09c9-417f-bc88-d7abb18d89ed None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:01.430413 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:01.437741 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 133/264] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:04:01 2022] PUT /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e/users/28658090be4845cb9ab0220fc87a5812/roles/6aee6f5679c84fc1be21561ddb7d1b56 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:04:01.438591 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40861087-09c9-417f-bc88-d7abb18d89ed None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:01.460914 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12331e-9bd6-4da4-815e-0e2d2478fe7a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:01.466357 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:01.527430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b4e5b-0949-499a-8d1e-5382edac10d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:01.528211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b4e5b-0949-499a-8d1e-5382edac10d2 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:01.528499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b4e5b-0949-499a-8d1e-5382edac10d2 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:01.528781 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b4e5b-0949-499a-8d1e-5382edac10d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:01.529715 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b4e5b-0949-499a-8d1e-5382edac10d2 None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=bb83c47ec18443db80a5cf2ed51d142e, user.id=b68ff2aacdf048ccb1e53d0b9e33de00) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:01.530033 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b4e5b-0949-499a-8d1e-5382edac10d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:01.531609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b4e5b-0949-499a-8d1e-5382edac10d2 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:01.555718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 132/265] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:04:01 2022] GET /identity/v3/role_assignments?user.id=b68ff2aacdf048ccb1e53d0b9e33de00&scope.project.id=bb83c47ec18443db80a5cf2ed51d142e&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:01.639292 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40861087-09c9-417f-bc88-d7abb18d89ed None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:01.702441 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28619b01-eec2-4c6e-8d6a-2dc22967065f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:01.703381 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28619b01-eec2-4c6e-8d6a-2dc22967065f None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:01.703674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28619b01-eec2-4c6e-8d6a-2dc22967065f None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:01.703954 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28619b01-eec2-4c6e-8d6a-2dc22967065f None admin] PATH_INFO: `/v3/users/b68ff2aacdf048ccb1e53d0b9e33de00` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:01.705289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28619b01-eec2-4c6e-8d6a-2dc22967065f None admin] RBAC: Authorizing `identity:get_user(user_id=b68ff2aacdf048ccb1e53d0b9e33de00)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:01.706958 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28619b01-eec2-4c6e-8d6a-2dc22967065f None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:01.708678 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 134/266] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:04:01 2022] GET /identity/v3/users/b68ff2aacdf048ccb1e53d0b9e33de00 => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:01.714484 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b4e5b-0949-499a-8d1e-5382edac10d2 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:01.859525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b4e5b-0949-499a-8d1e-5382edac10d2 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:01.915980 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadeea8d-0a11-40e4-a51a-3c94b65abccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:01.916660 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadeea8d-0a11-40e4-a51a-3c94b65abccf None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:01.916944 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadeea8d-0a11-40e4-a51a-3c94b65abccf None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:01.917277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadeea8d-0a11-40e4-a51a-3c94b65abccf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:01.959804 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:02.010276 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadeea8d-0a11-40e4-a51a-3c94b65abccf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:02.012635 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadeea8d-0a11-40e4-a51a-3c94b65abccf None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:02.126331 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 133/267] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:02.132223 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28619b01-eec2-4c6e-8d6a-2dc22967065f None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:02.217475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:02.264347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28619b01-eec2-4c6e-8d6a-2dc22967065f None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:02.307008 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb062b3-afb7-43cb-a9f3-61a1fd396e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:02.307948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb062b3-afb7-43cb-a9f3-61a1fd396e4d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:02.308376 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb062b3-afb7-43cb-a9f3-61a1fd396e4d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:02.308657 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb062b3-afb7-43cb-a9f3-61a1fd396e4d None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:02.315072 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb062b3-afb7-43cb-a9f3-61a1fd396e4d None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:02.316570 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb062b3-afb7-43cb-a9f3-61a1fd396e4d None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:02.318697 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 135/268] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:02 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:02.324543 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadeea8d-0a11-40e4-a51a-3c94b65abccf None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:02.401550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787dd0d-2aa1-40cd-9d3d-5d9a10c823f0 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:02.402149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787dd0d-2aa1-40cd-9d3d-5d9a10c823f0 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:02.402149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787dd0d-2aa1-40cd-9d3d-5d9a10c823f0 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:02.404032 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 136/269] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:02.409235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af475155-1a7d-4101-9a8e-1aeed6976893 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:02.409529 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af475155-1a7d-4101-9a8e-1aeed6976893 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:02.409814 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af475155-1a7d-4101-9a8e-1aeed6976893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:02.439609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:02.441502 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-af475155-1a7d-4101-9a8e-1aeed6976893 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:02.442816 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-af475155-1a7d-4101-9a8e-1aeed6976893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:02.509559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadeea8d-0a11-40e4-a51a-3c94b65abccf None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:02.554801 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 137/270] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:02.568746 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59460a-563f-4a83-b473-1321a611c170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:02.569729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59460a-563f-4a83-b473-1321a611c170 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:02.570245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59460a-563f-4a83-b473-1321a611c170 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:02.570738 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59460a-563f-4a83-b473-1321a611c170 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:02.571963 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59460a-563f-4a83-b473-1321a611c170 None admin] RBAC: Adding query filter params (role.id=b21c1004651d40f1b44fb0b7becbe3c8, scope.project.id=bb83c47ec18443db80a5cf2ed51d142e, user.id=b68ff2aacdf048ccb1e53d0b9e33de00) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:02.572462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59460a-563f-4a83-b473-1321a611c170 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:02.574464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59460a-563f-4a83-b473-1321a611c170 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:02.585775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 134/271] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:04:02 2022] GET /identity/v3/role_assignments?user.id=b68ff2aacdf048ccb1e53d0b9e33de00&scope.project.id=bb83c47ec18443db80a5cf2ed51d142e&role.id=b21c1004651d40f1b44fb0b7becbe3c8 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:02.586947 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af475155-1a7d-4101-9a8e-1aeed6976893 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:02.624267 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfb2ae-85f1-4d6e-949b-7c12cffb2bb0 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:02.624599 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfb2ae-85f1-4d6e-949b-7c12cffb2bb0 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:02.624821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfb2ae-85f1-4d6e-949b-7c12cffb2bb0 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:02.626509 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 135/272] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:04:02.634113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f763be-e462-4826-9b4a-e6bc451ba485 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:02.634453 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f763be-e462-4826-9b4a-e6bc451ba485 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:02.634705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f763be-e462-4826-9b4a-e6bc451ba485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:02.675471 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-65f763be-e462-4826-9b4a-e6bc451ba485 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:02.676379 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:02.676709 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-65f763be-e462-4826-9b4a-e6bc451ba485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:02.725319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af475155-1a7d-4101-9a8e-1aeed6976893 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:02.785309 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113ead7-7a75-479a-bf29-ba8aac128cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:02.788527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113ead7-7a75-479a-bf29-ba8aac128cb0 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:02.789943 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113ead7-7a75-479a-bf29-ba8aac128cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:02.790991 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113ead7-7a75-479a-bf29-ba8aac128cb0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:02.793629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113ead7-7a75-479a-bf29-ba8aac128cb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:02.800095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113ead7-7a75-479a-bf29-ba8aac128cb0 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:02.802289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 138/273] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:02.807479 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113ead7-7a75-479a-bf29-ba8aac128cb0 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:02.821413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 136/274] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:02.862331 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f763be-e462-4826-9b4a-e6bc451ba485 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:02.941432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:02.952846 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113ead7-7a75-479a-bf29-ba8aac128cb0 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:02.991140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:02.997073 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f763be-e462-4826-9b4a-e6bc451ba485 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:03.001330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba0e99-d079-4561-af53-1623af27b7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:03.002028 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba0e99-d079-4561-af53-1623af27b7d1 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:03.002346 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba0e99-d079-4561-af53-1623af27b7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:03.002629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba0e99-d079-4561-af53-1623af27b7d1 None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:03.003365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba0e99-d079-4561-af53-1623af27b7d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:03.004824 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba0e99-d079-4561-af53-1623af27b7d1 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:03.028042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 139/275] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:04:02 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:04:03.029438 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f68454-6233-4d0a-8190-c9867a5bfbc7 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:03.029730 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f68454-6233-4d0a-8190-c9867a5bfbc7 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:03.030068 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f68454-6233-4d0a-8190-c9867a5bfbc7 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:03.034327 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 140/276] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:03.037912 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dab6d-94e4-4cc2-aa22-e9d1e018e183 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:03.038352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045b954-ad5e-47b9-9a8d-de461c91672d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:03.038684 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dab6d-94e4-4cc2-aa22-e9d1e018e183 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:03.039135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dab6d-94e4-4cc2-aa22-e9d1e018e183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:03.039458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045b954-ad5e-47b9-9a8d-de461c91672d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:03.039842 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045b954-ad5e-47b9-9a8d-de461c91672d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:03.040206 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045b954-ad5e-47b9-9a8d-de461c91672d None admin] PATH_INFO: `/v3/roles/member` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:03.052973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045b954-ad5e-47b9-9a8d-de461c91672d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:03.059153 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-150dab6d-94e4-4cc2-aa22-e9d1e018e183 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:03.060804 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-150dab6d-94e4-4cc2-aa22-e9d1e018e183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-7045b954-ad5e-47b9-9a8d-de461c91672d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:03.061374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:03.062419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:03.063352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:03.063352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:03.063352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:03.063352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:03.063352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:03.063352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:03.063352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:03.063352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:03.063352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:03.063352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:03.063352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:03.063352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:03.063352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 137/277] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:03.070436 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045b954-ad5e-47b9-9a8d-de461c91672d None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:03.161909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 141/278] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:03.184650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150dab6d-94e4-4cc2-aa22-e9d1e018e183 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:03.195092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045b954-ad5e-47b9-9a8d-de461c91672d None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:03.225225 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:03.248818 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171599e-e45a-4e1d-9724-4707313dcb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:03.249729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171599e-e45a-4e1d-9724-4707313dcb42 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:03.250072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171599e-e45a-4e1d-9724-4707313dcb42 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:03.250441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171599e-e45a-4e1d-9724-4707313dcb42 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:03.251280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171599e-e45a-4e1d-9724-4707313dcb42 None admin] RBAC: Adding query filter params (name=member) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:03.251656 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171599e-e45a-4e1d-9724-4707313dcb42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:03.253255 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171599e-e45a-4e1d-9724-4707313dcb42 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:03.264357 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 138/279] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:03.269842 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171599e-e45a-4e1d-9724-4707313dcb42 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:03.311609 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150dab6d-94e4-4cc2-aa22-e9d1e018e183 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:03.340385 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:03.367480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b43f9-6fd5-4e01-992c-02acc86b236d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:03.368320 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b43f9-6fd5-4e01-992c-02acc86b236d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:03.368598 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b43f9-6fd5-4e01-992c-02acc86b236d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:03.368872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b43f9-6fd5-4e01-992c-02acc86b236d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:03.380732 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b43f9-6fd5-4e01-992c-02acc86b236d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-c15b43f9-6fd5-4e01-992c-02acc86b236d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:03.388955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:03.389959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:03.391336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:03.391336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:03.391336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:03.391336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:03.391336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:03.391336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:03.391336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:03.391336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:03.391336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:03.391336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:03.391336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:04:03.391336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:03.391336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 142/280] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:03.399422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b43f9-6fd5-4e01-992c-02acc86b236d None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:03.408506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171599e-e45a-4e1d-9724-4707313dcb42 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:03.452469 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee2640-7a80-48e7-aec5-1094b75beea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:03.454178 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee2640-7a80-48e7-aec5-1094b75beea2 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:03.454656 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee2640-7a80-48e7-aec5-1094b75beea2 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:03.455095 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee2640-7a80-48e7-aec5-1094b75beea2 None admin] PATH_INFO: `/v3/users/1a1e2e2331c34fd9bc6677933e175300` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:03.475168 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee2640-7a80-48e7-aec5-1094b75beea2 None admin] RBAC: Authorizing `identity:get_user(user_id=1a1e2e2331c34fd9bc6677933e175300)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:03.477636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee2640-7a80-48e7-aec5-1094b75beea2 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:03.481328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 139/281] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:04:03 2022] GET /identity/v3/users/1a1e2e2331c34fd9bc6677933e175300 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:03.486667 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee2640-7a80-48e7-aec5-1094b75beea2 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:03.526078 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b43f9-6fd5-4e01-992c-02acc86b236d None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:03.545300 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:03.578403 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc40c8d-6500-499a-adb6-b93b81056099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:03.579409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc40c8d-6500-499a-adb6-b93b81056099 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:03.579892 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc40c8d-6500-499a-adb6-b93b81056099 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:03.580368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc40c8d-6500-499a-adb6-b93b81056099 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:03.581322 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc40c8d-6500-499a-adb6-b93b81056099 None admin] RBAC: Adding query filter params (name=reader) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:03.581905 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc40c8d-6500-499a-adb6-b93b81056099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:03.583601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc40c8d-6500-499a-adb6-b93b81056099 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:03.602495 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 143/282] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:03.608567 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc40c8d-6500-499a-adb6-b93b81056099 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:03.610595 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee2640-7a80-48e7-aec5-1094b75beea2 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:03.647606 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d40ea6-50ab-414b-9cd2-b414465413c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:03.648282 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d40ea6-50ab-414b-9cd2-b414465413c3 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:03.648565 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d40ea6-50ab-414b-9cd2-b414465413c3 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:03.648838 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d40ea6-50ab-414b-9cd2-b414465413c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:03.649686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d40ea6-50ab-414b-9cd2-b414465413c3 None admin] RBAC: Adding query filter params (role.id=6aee6f5679c84fc1be21561ddb7d1b56, user.id=1a1e2e2331c34fd9bc6677933e175300, scope.system=all) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:03.650003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d40ea6-50ab-414b-9cd2-b414465413c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:03.651524 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d40ea6-50ab-414b-9cd2-b414465413c3 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:03.657597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 140/283] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 20:04:03 2022] GET /identity/v3/role_assignments?user.id=1a1e2e2331c34fd9bc6677933e175300&scope.system=all&role.id=6aee6f5679c84fc1be21561ddb7d1b56 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:03.704940 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:03.771167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc40c8d-6500-499a-adb6-b93b81056099 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:03.808968 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd3fdd-253d-45aa-acc1-b75325491149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:03.810014 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3fdd-253d-45aa-acc1-b75325491149 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:03.810334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3fdd-253d-45aa-acc1-b75325491149 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:03.810616 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3fdd-253d-45aa-acc1-b75325491149 None admin] PATH_INFO: `/v3/users/fcf21c2104594c4f85c6557e2a6dafc6` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:03.812018 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd3fdd-253d-45aa-acc1-b75325491149 None admin] RBAC: Authorizing `identity:get_user(user_id=fcf21c2104594c4f85c6557e2a6dafc6)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:03.813559 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd3fdd-253d-45aa-acc1-b75325491149 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:03.815670 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 144/284] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:04:03 2022] GET /identity/v3/users/fcf21c2104594c4f85c6557e2a6dafc6 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:03.821254 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d40ea6-50ab-414b-9cd2-b414465413c3 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:04.005196 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d40ea6-50ab-414b-9cd2-b414465413c3 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:04.011380 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:04.059936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4700d-6af9-4bd5-bea1-84ff52b0d8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:04.060826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4700d-6af9-4bd5-bea1-84ff52b0d8cb None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:04.061107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4700d-6af9-4bd5-bea1-84ff52b0d8cb None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:04.061381 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4700d-6af9-4bd5-bea1-84ff52b0d8cb None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:04.062627 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4700d-6af9-4bd5-bea1-84ff52b0d8cb None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:04.064106 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4700d-6af9-4bd5-bea1-84ff52b0d8cb None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:04.066237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 141/285] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:03 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:04.071862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd3fdd-253d-45aa-acc1-b75325491149 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:04.190916 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd3fdd-253d-45aa-acc1-b75325491149 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:04.200482 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:04.237840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f67374-2060-406e-9985-47a5e0dcd591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:04.238781 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f67374-2060-406e-9985-47a5e0dcd591 None admin] REQUEST_METHOD: `PUT` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:04.239067 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f67374-2060-406e-9985-47a5e0dcd591 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:04.239342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f67374-2060-406e-9985-47a5e0dcd591 None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e/users/fcf21c2104594c4f85c6557e2a6dafc6/roles/609bb86f2a5b44b0b244b9bfe7c073de` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:04.241380 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f67374-2060-406e-9985-47a5e0dcd591 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb83c47ec18443db80a5cf2ed51d142e, user_id=fcf21c2104594c4f85c6557e2a6dafc6, role_id=609bb86f2a5b44b0b244b9bfe7c073de)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:04.242938 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f67374-2060-406e-9985-47a5e0dcd591 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:04.269985 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 145/286] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:04:04 2022] PUT /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e/users/fcf21c2104594c4f85c6557e2a6dafc6/roles/609bb86f2a5b44b0b244b9bfe7c073de => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:04:04.454279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981146e-2519-4235-8800-ab541b414c18 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:04.454826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981146e-2519-4235-8800-ab541b414c18 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:04.454826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981146e-2519-4235-8800-ab541b414c18 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:04.456475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 142/287] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:04.462127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008fe77-40f4-4f30-8843-39241adfe950 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:04.462459 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008fe77-40f4-4f30-8843-39241adfe950 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:04.462739 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008fe77-40f4-4f30-8843-39241adfe950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:04.488832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-0008fe77-40f4-4f30-8843-39241adfe950 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:04.490081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-0008fe77-40f4-4f30-8843-39241adfe950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:04.583588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 146/288] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:04.610209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981146e-2519-4235-8800-ab541b414c18 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:04.737792 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981146e-2519-4235-8800-ab541b414c18 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:04.774016 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538cda9-c49e-4ba9-bb02-a2aeb28d7574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:04.774886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538cda9-c49e-4ba9-bb02-a2aeb28d7574 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:04.775168 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538cda9-c49e-4ba9-bb02-a2aeb28d7574 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:04.775440 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538cda9-c49e-4ba9-bb02-a2aeb28d7574 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:04.788116 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538cda9-c49e-4ba9-bb02-a2aeb28d7574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-5538cda9-c49e-4ba9-bb02-a2aeb28d7574 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:04.796573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:04.797597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:04.798552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:04.798552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:04.798552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:04.798552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:04.798552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:04.798552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:04.798552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:04.798552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:04.798552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:04.798552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:04.798552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:04.798552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:04.798552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 143/289] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:04.806646 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008fe77-40f4-4f30-8843-39241adfe950 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:04.931521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:04.939821 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008fe77-40f4-4f30-8843-39241adfe950 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:04.985883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a90a9-979e-43ab-a49c-d856b5c85185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:04.986674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a90a9-979e-43ab-a49c-d856b5c85185 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:04.986957 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a90a9-979e-43ab-a49c-d856b5c85185 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:04.987232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a90a9-979e-43ab-a49c-d856b5c85185 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:04.988120 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a90a9-979e-43ab-a49c-d856b5c85185 None admin] RBAC: Adding query filter params (name=member) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:04.988371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a90a9-979e-43ab-a49c-d856b5c85185 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:04.989821 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a90a9-979e-43ab-a49c-d856b5c85185 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:04.999032 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 147/290] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:05.004473 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538cda9-c49e-4ba9-bb02-a2aeb28d7574 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:05.068031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:05.163856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538cda9-c49e-4ba9-bb02-a2aeb28d7574 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:05.198176 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93155c-17ad-40c1-af96-dcd4f3dfc113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:05.199387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93155c-17ad-40c1-af96-dcd4f3dfc113 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:05.199387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93155c-17ad-40c1-af96-dcd4f3dfc113 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:05.199686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93155c-17ad-40c1-af96-dcd4f3dfc113 None admin] PATH_INFO: `/v3/users/28658090be4845cb9ab0220fc87a5812` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:05.200890 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93155c-17ad-40c1-af96-dcd4f3dfc113 None admin] RBAC: Authorizing `identity:get_user(user_id=28658090be4845cb9ab0220fc87a5812)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:05.202485 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93155c-17ad-40c1-af96-dcd4f3dfc113 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:05.204528 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 144/291] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:04:05 2022] GET /identity/v3/users/28658090be4845cb9ab0220fc87a5812 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:05.209965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a90a9-979e-43ab-a49c-d856b5c85185 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:05.363842 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a90a9-979e-43ab-a49c-d856b5c85185 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:05.416349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdebcb-9d3a-45d6-9842-6983c2e7dca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:05.417031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdebcb-9d3a-45d6-9842-6983c2e7dca2 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:05.417435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdebcb-9d3a-45d6-9842-6983c2e7dca2 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:05.417745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdebcb-9d3a-45d6-9842-6983c2e7dca2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:05.453098 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:05.502287 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdebcb-9d3a-45d6-9842-6983c2e7dca2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:05.503774 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdebcb-9d3a-45d6-9842-6983c2e7dca2 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:05.643203 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 148/292] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:05.648414 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93155c-17ad-40c1-af96-dcd4f3dfc113 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:05.755826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:05.759810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93155c-17ad-40c1-af96-dcd4f3dfc113 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:05.804007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f5bab-986e-4713-abec-6ee760edcdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:05.804902 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f5bab-986e-4713-abec-6ee760edcdfd None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:05.805180 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f5bab-986e-4713-abec-6ee760edcdfd None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:05.805501 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f5bab-986e-4713-abec-6ee760edcdfd None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:05.806707 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f5bab-986e-4713-abec-6ee760edcdfd None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:05.808412 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f5bab-986e-4713-abec-6ee760edcdfd None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:05.810497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 145/293] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:05 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:05.816268 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdebcb-9d3a-45d6-9842-6983c2e7dca2 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:05.950155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:05.986374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96211f2-3d83-4ad0-bd03-391e3a15cbb3 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:05.986748 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96211f2-3d83-4ad0-bd03-391e3a15cbb3 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:05.986941 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96211f2-3d83-4ad0-bd03-391e3a15cbb3 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:05.988581 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 146/294] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:04:05.994006 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ab8e7-8c5d-4239-84ca-7645153c189c None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:05.994328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ab8e7-8c5d-4239-84ca-7645153c189c None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:05.994613 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ab8e7-8c5d-4239-84ca-7645153c189c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:06.002580 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdebcb-9d3a-45d6-9842-6983c2e7dca2 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:06.017369 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-c69ab8e7-8c5d-4239-84ca-7645153c189c None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:06.020050 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-c69ab8e7-8c5d-4239-84ca-7645153c189c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:06.066276 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd25e2-46c0-41ef-9d24-f4d730496602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:06.066950 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd25e2-46c0-41ef-9d24-f4d730496602 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:06.067232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd25e2-46c0-41ef-9d24-f4d730496602 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:06.067508 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd25e2-46c0-41ef-9d24-f4d730496602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:06.068355 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd25e2-46c0-41ef-9d24-f4d730496602 None admin] RBAC: Adding query filter params (role.id=6aee6f5679c84fc1be21561ddb7d1b56, scope.project.id=bb83c47ec18443db80a5cf2ed51d142e, user.id=28658090be4845cb9ab0220fc87a5812) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:06.068669 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd25e2-46c0-41ef-9d24-f4d730496602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:06.070131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd25e2-46c0-41ef-9d24-f4d730496602 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:06.083603 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 149/295] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:04:05 2022] GET /identity/v3/role_assignments?user.id=28658090be4845cb9ab0220fc87a5812&scope.project.id=bb83c47ec18443db80a5cf2ed51d142e&role.id=6aee6f5679c84fc1be21561ddb7d1b56 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:06.127222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 147/296] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:06.151226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd25e2-46c0-41ef-9d24-f4d730496602 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:06.224592 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0e2ce-9583-4236-b761-d1a7cda1a73a None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:06.224820 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0e2ce-9583-4236-b761-d1a7cda1a73a None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:06.225067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0e2ce-9583-4236-b761-d1a7cda1a73a None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:06.226632 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 148/297] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:04:06.234001 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5454e-0cbd-4cc9-9bae-3d095bc8fb87 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:06.234341 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5454e-0cbd-4cc9-9bae-3d095bc8fb87 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:06.234587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5454e-0cbd-4cc9-9bae-3d095bc8fb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:06.270162 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd25e2-46c0-41ef-9d24-f4d730496602 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:06.270561 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-a5b5454e-0cbd-4cc9-9bae-3d095bc8fb87 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:06.271811 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-a5b5454e-0cbd-4cc9-9bae-3d095bc8fb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:06.306674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:06.312026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dff65d-fab3-4e0d-bbc8-8e135c990338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:06.312859 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dff65d-fab3-4e0d-bbc8-8e135c990338 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:06.313140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dff65d-fab3-4e0d-bbc8-8e135c990338 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:06.313417 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dff65d-fab3-4e0d-bbc8-8e135c990338 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:06.314813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dff65d-fab3-4e0d-bbc8-8e135c990338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:06.316401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dff65d-fab3-4e0d-bbc8-8e135c990338 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:06.318553 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 150/298] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:06.324694 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dff65d-fab3-4e0d-bbc8-8e135c990338 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:06.418432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 149/299] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:06.434274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dff65d-fab3-4e0d-bbc8-8e135c990338 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:06.447348 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5454e-0cbd-4cc9-9bae-3d095bc8fb87 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:06.478200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c6712-0619-4641-aa9c-cbd47ad126ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:06.479034 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c6712-0619-4641-aa9c-cbd47ad126ca None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:06.479322 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c6712-0619-4641-aa9c-cbd47ad126ca None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:06.480027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c6712-0619-4641-aa9c-cbd47ad126ca None admin] PATH_INFO: `/v3/groups` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:06.480804 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c6712-0619-4641-aa9c-cbd47ad126ca None admin] RBAC: Authorizing `identity:create_group()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:06.482601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c6712-0619-4641-aa9c-cbd47ad126ca None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:06.490734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:06.515212 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 151/300] 10.222.0.84 () {60 vars in 1134 bytes} [Thu May 19 20:04:06 2022] POST /identity/v3/groups => generated 229 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:04:06.522314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8e6d3-36e2-4176-baca-0a4a9300c0e3 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:06.522616 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8e6d3-36e2-4176-baca-0a4a9300c0e3 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:06.522903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8e6d3-36e2-4176-baca-0a4a9300c0e3 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:06.524462 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 152/301] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:04:06.531254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab44adc-a6a8-4db0-87eb-64329de79abb None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:06.531507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab44adc-a6a8-4db0-87eb-64329de79abb None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:06.531592 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab44adc-a6a8-4db0-87eb-64329de79abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:06.554498 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-eab44adc-a6a8-4db0-87eb-64329de79abb None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:06.555743 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-eab44adc-a6a8-4db0-87eb-64329de79abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:06.598732 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5454e-0cbd-4cc9-9bae-3d095bc8fb87 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:06.625824 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:06.654294 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 153/302] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:04:06.658420 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc3897-be37-428b-9713-48082b69d43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:06.659745 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc3897-be37-428b-9713-48082b69d43c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:06.660172 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc3897-be37-428b-9713-48082b69d43c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:06.660588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc3897-be37-428b-9713-48082b69d43c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:06.673020 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc3897-be37-428b-9713-48082b69d43c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:06.680510 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab44adc-a6a8-4db0-87eb-64329de79abb None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-40cc3897-be37-428b-9713-48082b69d43c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:06.687499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:06.688638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:06.689747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:06.689747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:06.689747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:06.689747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:06.689747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:06.689747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:06.689747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:06.689747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:06.689747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:06.689747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:06.689747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:04:06.689747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:06.689747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 150/303] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:06.695409 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc3897-be37-428b-9713-48082b69d43c None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:06.802003 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab44adc-a6a8-4db0-87eb-64329de79abb None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:06.839035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:06.840927 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2894dc-e65d-43a2-a2bb-27130fc363c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:06.841979 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2894dc-e65d-43a2-a2bb-27130fc363c5 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:06.842468 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2894dc-e65d-43a2-a2bb-27130fc363c5 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:06.842906 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2894dc-e65d-43a2-a2bb-27130fc363c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:06.853126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc3897-be37-428b-9713-48082b69d43c None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:06.855347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2894dc-e65d-43a2-a2bb-27130fc363c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-7c2894dc-e65d-43a2-a2bb-27130fc363c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:06.865849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:06.867081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:06.868155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:06.868155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:06.868155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:06.868155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:06.868155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:06.868155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:06.868155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:06.868155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:06.868155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:06.868155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:06.868155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:06.868155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:06.871552 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 154/304] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:06.878746 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2894dc-e65d-43a2-a2bb-27130fc363c5 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:06.893338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9636e-18d0-40e3-85ca-33a1be4a45f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:06.894319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9636e-18d0-40e3-85ca-33a1be4a45f6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:06.894808 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9636e-18d0-40e3-85ca-33a1be4a45f6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:06.895287 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9636e-18d0-40e3-85ca-33a1be4a45f6 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:06.896220 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9636e-18d0-40e3-85ca-33a1be4a45f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:06.896710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9636e-18d0-40e3-85ca-33a1be4a45f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:06.898325 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9636e-18d0-40e3-85ca-33a1be4a45f6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:06.907893 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 151/305] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:06.915728 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9636e-18d0-40e3-85ca-33a1be4a45f6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:06.924441 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:06.984317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2894dc-e65d-43a2-a2bb-27130fc363c5 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:07.019259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc00219-f0e3-43bc-a7eb-37541c631921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:07.019948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc00219-f0e3-43bc-a7eb-37541c631921 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:07.020231 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc00219-f0e3-43bc-a7eb-37541c631921 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:07.020507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc00219-f0e3-43bc-a7eb-37541c631921 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:07.021272 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc00219-f0e3-43bc-a7eb-37541c631921 None admin] RBAC: Adding query filter params (name=member) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:07.021584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc00219-f0e3-43bc-a7eb-37541c631921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:07.023009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc00219-f0e3-43bc-a7eb-37541c631921 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:07.023281 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9636e-18d0-40e3-85ca-33a1be4a45f6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:07.037306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 155/306] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:07.038295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc00219-f0e3-43bc-a7eb-37541c631921 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:07.061303 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ddc15-a5e2-46a5-8e3f-18bd26c95e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:07.062145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ddc15-a5e2-46a5-8e3f-18bd26c95e7a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:07.062457 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ddc15-a5e2-46a5-8e3f-18bd26c95e7a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:07.062741 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ddc15-a5e2-46a5-8e3f-18bd26c95e7a None admin] PATH_INFO: `/v3/users/490e3645964e4351a1ada3a117ce13a5` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:07.078640 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ddc15-a5e2-46a5-8e3f-18bd26c95e7a None admin] RBAC: Authorizing `identity:get_user(user_id=490e3645964e4351a1ada3a117ce13a5)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:07.080133 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ddc15-a5e2-46a5-8e3f-18bd26c95e7a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:07.082172 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 152/307] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:04:06 2022] GET /identity/v3/users/490e3645964e4351a1ada3a117ce13a5 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:07.087956 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ddc15-a5e2-46a5-8e3f-18bd26c95e7a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:07.100217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:07.145092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc00219-f0e3-43bc-a7eb-37541c631921 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:07.154897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:07.190847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50571b-63fe-4240-ab5a-bff59d6e3afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:07.192263 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50571b-63fe-4240-ab5a-bff59d6e3afe None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:07.192874 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50571b-63fe-4240-ab5a-bff59d6e3afe None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:07.193350 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50571b-63fe-4240-ab5a-bff59d6e3afe None admin] PATH_INFO: `/v3/users/1a1e2e2331c34fd9bc6677933e175300` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:07.194884 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50571b-63fe-4240-ab5a-bff59d6e3afe None admin] RBAC: Authorizing `identity:get_user(user_id=1a1e2e2331c34fd9bc6677933e175300)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:07.196642 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50571b-63fe-4240-ab5a-bff59d6e3afe None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:07.198619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 156/308] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:04:07 2022] GET /identity/v3/users/1a1e2e2331c34fd9bc6677933e175300 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:07.204178 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50571b-63fe-4240-ab5a-bff59d6e3afe None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:07.222196 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ddc15-a5e2-46a5-8e3f-18bd26c95e7a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:07.266803 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0539c69-30eb-4c80-9924-5626acd48568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:07.267735 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0539c69-30eb-4c80-9924-5626acd48568 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:07.268025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0539c69-30eb-4c80-9924-5626acd48568 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:07.268722 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0539c69-30eb-4c80-9924-5626acd48568 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:07.269864 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0539c69-30eb-4c80-9924-5626acd48568 None admin] RBAC: Adding query filter params (role.id=609bb86f2a5b44b0b244b9bfe7c073de, user.id=490e3645964e4351a1ada3a117ce13a5, scope.system=all) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:07.270184 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0539c69-30eb-4c80-9924-5626acd48568 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:07.272656 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0539c69-30eb-4c80-9924-5626acd48568 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:07.281134 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 153/309] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 20:04:07 2022] GET /identity/v3/role_assignments?user.id=490e3645964e4351a1ada3a117ce13a5&scope.system=all&role.id=609bb86f2a5b44b0b244b9bfe7c073de => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:07.322384 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50571b-63fe-4240-ab5a-bff59d6e3afe None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:07.370278 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a4e47-66e6-4dc5-9839-bad707e3edfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:07.371258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a4e47-66e6-4dc5-9839-bad707e3edfb None admin] REQUEST_METHOD: `PUT` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:07.371540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a4e47-66e6-4dc5-9839-bad707e3edfb None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:07.371812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a4e47-66e6-4dc5-9839-bad707e3edfb None admin] PATH_INFO: `/v3/system/users/1a1e2e2331c34fd9bc6677933e175300/roles/6aee6f5679c84fc1be21561ddb7d1b56` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:07.373627 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a4e47-66e6-4dc5-9839-bad707e3edfb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1a1e2e2331c34fd9bc6677933e175300, role_id=6aee6f5679c84fc1be21561ddb7d1b56)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:07.375172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a4e47-66e6-4dc5-9839-bad707e3edfb None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:07.389795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 157/310] 10.222.0.84 () {58 vars in 1334 bytes} [Thu May 19 20:04:07 2022] PUT /identity/v3/system/users/1a1e2e2331c34fd9bc6677933e175300/roles/6aee6f5679c84fc1be21561ddb7d1b56 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:04:07.584369 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352703a7-1240-4076-9766-f7a81df9655a None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:07.584581 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352703a7-1240-4076-9766-f7a81df9655a None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:07.584830 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352703a7-1240-4076-9766-f7a81df9655a None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:07.586530 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 154/311] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:07.592389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7837852-3198-4e46-a5b4-a815016b91e9 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:07.592689 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7837852-3198-4e46-a5b4-a815016b91e9 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:07.592971 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7837852-3198-4e46-a5b4-a815016b91e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:07.611467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-d7837852-3198-4e46-a5b4-a815016b91e9 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:07.612660 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-d7837852-3198-4e46-a5b4-a815016b91e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:07.643752 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:07.695218 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 158/312] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:07.718918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352703a7-1240-4076-9766-f7a81df9655a None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:07.793548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:07.831293 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352703a7-1240-4076-9766-f7a81df9655a None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:07.867895 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc3b05-217f-45dd-be19-1d3f1ad3f5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:07.868618 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc3b05-217f-45dd-be19-1d3f1ad3f5b4 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:07.868891 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc3b05-217f-45dd-be19-1d3f1ad3f5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:07.869161 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc3b05-217f-45dd-be19-1d3f1ad3f5b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:07.880165 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc3b05-217f-45dd-be19-1d3f1ad3f5b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-38bc3b05-217f-45dd-be19-1d3f1ad3f5b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:07.887922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:07.889955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:07.891602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:07.891602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:07.891602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:07.891602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:07.891602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:07.891602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:07.891602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:07.891602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:07.891602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:07.891602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:07.891602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:04:07.891602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:07.891602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 155/313] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:07.898284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7837852-3198-4e46-a5b4-a815016b91e9 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:08.012342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7837852-3198-4e46-a5b4-a815016b91e9 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:08.047704 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cc863-1f41-40eb-af41-6c88eb1c34eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:08.048384 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cc863-1f41-40eb-af41-6c88eb1c34eb None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:08.048664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cc863-1f41-40eb-af41-6c88eb1c34eb None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:08.048938 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cc863-1f41-40eb-af41-6c88eb1c34eb None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:08.049696 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485cc863-1f41-40eb-af41-6c88eb1c34eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:08.050001 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485cc863-1f41-40eb-af41-6c88eb1c34eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:08.051401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485cc863-1f41-40eb-af41-6c88eb1c34eb None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:08.060375 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 159/314] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:08.067582 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc3b05-217f-45dd-be19-1d3f1ad3f5b4 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:08.179808 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc3b05-217f-45dd-be19-1d3f1ad3f5b4 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:08.210734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:08.217657 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad9496-31f8-42c3-af5d-721f264b4ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:08.218573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad9496-31f8-42c3-af5d-721f264b4ac7 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:08.218853 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad9496-31f8-42c3-af5d-721f264b4ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:08.219125 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad9496-31f8-42c3-af5d-721f264b4ac7 None admin] PATH_INFO: `/v3/users/fcf21c2104594c4f85c6557e2a6dafc6` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:08.220656 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad9496-31f8-42c3-af5d-721f264b4ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=fcf21c2104594c4f85c6557e2a6dafc6)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:08.222114 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad9496-31f8-42c3-af5d-721f264b4ac7 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:08.224117 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 156/315] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:04:08 2022] GET /identity/v3/users/fcf21c2104594c4f85c6557e2a6dafc6 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:08.229444 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cc863-1f41-40eb-af41-6c88eb1c34eb None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:08.259120 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:08.338751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cc863-1f41-40eb-af41-6c88eb1c34eb None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:08.374089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d1a49-269e-4621-a5d2-52a57a3b325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:08.374788 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d1a49-269e-4621-a5d2-52a57a3b325a None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:08.375069 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d1a49-269e-4621-a5d2-52a57a3b325a None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:08.375348 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d1a49-269e-4621-a5d2-52a57a3b325a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:08.445963 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d1a49-269e-4621-a5d2-52a57a3b325a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:08.447405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d1a49-269e-4621-a5d2-52a57a3b325a None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:08.482678 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:08.557257 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 160/316] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:08.562798 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad9496-31f8-42c3-af5d-721f264b4ac7 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:08.671014 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad9496-31f8-42c3-af5d-721f264b4ac7 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:08.712271 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9bfec-5fc2-47d2-ad79-f68700032b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:08.713062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9bfec-5fc2-47d2-ad79-f68700032b80 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:08.713339 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9bfec-5fc2-47d2-ad79-f68700032b80 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:08.713612 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9bfec-5fc2-47d2-ad79-f68700032b80 None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:08.715045 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9bfec-5fc2-47d2-ad79-f68700032b80 None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:08.716521 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9bfec-5fc2-47d2-ad79-f68700032b80 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:08.718557 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 157/317] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:08 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:08.724718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d1a49-269e-4621-a5d2-52a57a3b325a None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:08.836447 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d1a49-269e-4621-a5d2-52a57a3b325a None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:08.864262 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:08.872591 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaef47e-ddfb-48f4-8c43-b3be1bf05348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:08.873267 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaef47e-ddfb-48f4-8c43-b3be1bf05348 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:08.873553 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaef47e-ddfb-48f4-8c43-b3be1bf05348 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:08.873888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaef47e-ddfb-48f4-8c43-b3be1bf05348 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:08.874778 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaef47e-ddfb-48f4-8c43-b3be1bf05348 None admin] RBAC: Adding query filter params (role.id=609bb86f2a5b44b0b244b9bfe7c073de, scope.project.id=bb83c47ec18443db80a5cf2ed51d142e, user.id=fcf21c2104594c4f85c6557e2a6dafc6) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:08.875093 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaef47e-ddfb-48f4-8c43-b3be1bf05348 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:08.876538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaef47e-ddfb-48f4-8c43-b3be1bf05348 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:08.883380 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 161/318] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:04:08 2022] GET /identity/v3/role_assignments?user.id=fcf21c2104594c4f85c6557e2a6dafc6&scope.project.id=bb83c47ec18443db80a5cf2ed51d142e&role.id=609bb86f2a5b44b0b244b9bfe7c073de => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:08.935344 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec897b4-a33a-4e6f-94de-f514d456d685 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:08.935904 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec897b4-a33a-4e6f-94de-f514d456d685 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:08.935904 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec897b4-a33a-4e6f-94de-f514d456d685 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:08.937527 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 158/319] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:08.943388 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463354a-820a-49d6-9e06-62bc77d09064 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:08.943709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463354a-820a-49d6-9e06-62bc77d09064 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:08.943967 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463354a-820a-49d6-9e06-62bc77d09064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:08.962942 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-7463354a-820a-49d6-9e06-62bc77d09064 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:08.964182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-7463354a-820a-49d6-9e06-62bc77d09064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:09.040315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 162/320] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:09.064130 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec897b4-a33a-4e6f-94de-f514d456d685 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:09.138916 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:09.175687 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec897b4-a33a-4e6f-94de-f514d456d685 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:09.210556 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a77993-18dd-4c65-a728-ccb23f416d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:09.211422 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a77993-18dd-4c65-a728-ccb23f416d58 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:09.211749 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a77993-18dd-4c65-a728-ccb23f416d58 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:09.212030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a77993-18dd-4c65-a728-ccb23f416d58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:09.213256 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a77993-18dd-4c65-a728-ccb23f416d58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:09.215239 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a77993-18dd-4c65-a728-ccb23f416d58 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:09.217332 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 159/321] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:09.223429 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463354a-820a-49d6-9e06-62bc77d09064 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:09.326199 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463354a-820a-49d6-9e06-62bc77d09064 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:09.350163 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:09.364073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9cef3-4141-4cf7-b90b-026eecc560d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:09.365034 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9cef3-4141-4cf7-b90b-026eecc560d9 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:09.365496 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9cef3-4141-4cf7-b90b-026eecc560d9 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:09.365934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9cef3-4141-4cf7-b90b-026eecc560d9 None admin] PATH_INFO: `/v3/groups` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:09.366834 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9cef3-4141-4cf7-b90b-026eecc560d9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:09.368619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9cef3-4141-4cf7-b90b-026eecc560d9 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:09.392689 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 163/322] 10.222.0.84 () {60 vars in 1134 bytes} [Thu May 19 20:04:09 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:04:09.654578 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2285e7-457b-4c16-a776-194905f1c38c None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:09.654958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2285e7-457b-4c16-a776-194905f1c38c None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:09.655139 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2285e7-457b-4c16-a776-194905f1c38c None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:09.656589 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 160/323] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:09.662504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed45f21-33ef-41eb-9d7d-0de9224725db None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:09.662735 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed45f21-33ef-41eb-9d7d-0de9224725db None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:09.663043 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed45f21-33ef-41eb-9d7d-0de9224725db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:09.681549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-9ed45f21-33ef-41eb-9d7d-0de9224725db None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:09.683031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-9ed45f21-33ef-41eb-9d7d-0de9224725db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:09.755842 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 164/324] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:09.778443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2285e7-457b-4c16-a776-194905f1c38c None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:09.823673 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b94c92-e032-48cb-945f-8e2849769cf0 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:09.823877 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b94c92-e032-48cb-945f-8e2849769cf0 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:09.824078 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b94c92-e032-48cb-945f-8e2849769cf0 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:09.827148 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 165/325] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:09.831731 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab321c1-332d-42c6-aeaa-cefbb8d9c4b2 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:09.831894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab321c1-332d-42c6-aeaa-cefbb8d9c4b2 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:09.832113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab321c1-332d-42c6-aeaa-cefbb8d9c4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:09.854999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-7ab321c1-332d-42c6-aeaa-cefbb8d9c4b2 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:09.856664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-7ab321c1-332d-42c6-aeaa-cefbb8d9c4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:09.909993 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2285e7-457b-4c16-a776-194905f1c38c None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:09.948517 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860f602-5ae8-44d1-9c42-a9469146e53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:09.949814 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 166/326] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:04:09.950462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860f602-5ae8-44d1-9c42-a9469146e53a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:09.950906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860f602-5ae8-44d1-9c42-a9469146e53a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:09.951339 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860f602-5ae8-44d1-9c42-a9469146e53a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:09.955136 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:09.964997 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860f602-5ae8-44d1-9c42-a9469146e53a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-1860f602-5ae8-44d1-9c42-a9469146e53a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:09.974101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:09.975328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:09.976410 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:09.976410 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:09.976410 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:09.976410 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:09.976410 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:09.976410 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:09.976410 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:09.976410 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:09.976410 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:09.976410 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:09.976410 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:04:09.976410 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:09.977205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab321c1-332d-42c6-aeaa-cefbb8d9c4b2 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:09.978686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 161/327] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:09.991136 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860f602-5ae8-44d1-9c42-a9469146e53a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:10.030057 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:10.094876 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab321c1-332d-42c6-aeaa-cefbb8d9c4b2 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:10.109530 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860f602-5ae8-44d1-9c42-a9469146e53a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:10.132788 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace5dee-7679-4d85-9ec7-69d9a96db04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:10.133601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace5dee-7679-4d85-9ec7-69d9a96db04a None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:10.133877 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace5dee-7679-4d85-9ec7-69d9a96db04a None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:10.134146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace5dee-7679-4d85-9ec7-69d9a96db04a None admin] PATH_INFO: `/v3/roles/member` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:10.145872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace5dee-7679-4d85-9ec7-69d9a96db04a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:10.152841 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207e744-3386-4470-af17-04ad724898e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-0ace5dee-7679-4d85-9ec7-69d9a96db04a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:10.153720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:10.154786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:10.155740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:10.155740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:10.155740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:10.155740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:10.155740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:10.155740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:10.155740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:10.155740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:10.155740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:10.155740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:10.155740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:10.155740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:10.155740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 167/328] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:10.156863 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207e744-3386-4470-af17-04ad724898e7 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:10.157361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207e744-3386-4470-af17-04ad724898e7 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:10.157642 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207e744-3386-4470-af17-04ad724898e7 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:10.159738 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207e744-3386-4470-af17-04ad724898e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:10.159738 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207e744-3386-4470-af17-04ad724898e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:10.160227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207e744-3386-4470-af17-04ad724898e7 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:10.161251 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace5dee-7679-4d85-9ec7-69d9a96db04a None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:10.171012 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 162/329] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:10.176807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207e744-3386-4470-af17-04ad724898e7 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:10.245660 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:10.284091 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace5dee-7679-4d85-9ec7-69d9a96db04a None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:10.286104 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:10.305847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207e744-3386-4470-af17-04ad724898e7 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:10.333595 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd57ad0-654f-410a-9fcf-d333f891cab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:10.336957 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd57ad0-654f-410a-9fcf-d333f891cab0 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:10.337769 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd57ad0-654f-410a-9fcf-d333f891cab0 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:10.338286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd57ad0-654f-410a-9fcf-d333f891cab0 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:10.340127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd57ad0-654f-410a-9fcf-d333f891cab0 None admin] RBAC: Adding query filter params (name=member) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:10.340696 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd57ad0-654f-410a-9fcf-d333f891cab0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:10.342933 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd57ad0-654f-410a-9fcf-d333f891cab0 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:10.356233 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 168/330] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:10.360754 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de6420-9efd-4f13-96ce-4ba2ae2a6a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:10.360754 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de6420-9efd-4f13-96ce-4ba2ae2a6a6b None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:10.360754 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de6420-9efd-4f13-96ce-4ba2ae2a6a6b None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:10.360754 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de6420-9efd-4f13-96ce-4ba2ae2a6a6b None admin] PATH_INFO: `/v3/users/490e3645964e4351a1ada3a117ce13a5` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:10.361943 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd57ad0-654f-410a-9fcf-d333f891cab0 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:10.363047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de6420-9efd-4f13-96ce-4ba2ae2a6a6b None admin] RBAC: Authorizing `identity:get_user(user_id=490e3645964e4351a1ada3a117ce13a5)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:10.364890 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de6420-9efd-4f13-96ce-4ba2ae2a6a6b None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:10.367262 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 163/331] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:04:10 2022] GET /identity/v3/users/490e3645964e4351a1ada3a117ce13a5 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:10.373606 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de6420-9efd-4f13-96ce-4ba2ae2a6a6b None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:10.555676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd57ad0-654f-410a-9fcf-d333f891cab0 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:10.562935 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de6420-9efd-4f13-96ce-4ba2ae2a6a6b None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:10.618247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:10.632142 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f98051-8929-48d1-b668-8e9aa0608059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:10.633876 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f98051-8929-48d1-b668-8e9aa0608059 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:10.635674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f98051-8929-48d1-b668-8e9aa0608059 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:10.636256 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f98051-8929-48d1-b668-8e9aa0608059 None admin] PATH_INFO: `/v3/users/1a1e2e2331c34fd9bc6677933e175300` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:10.637086 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35093605-baad-40b9-866f-28a1c1aedc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:10.638889 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35093605-baad-40b9-866f-28a1c1aedc31 None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:10.639808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f98051-8929-48d1-b668-8e9aa0608059 None admin] RBAC: Authorizing `identity:get_user(user_id=1a1e2e2331c34fd9bc6677933e175300)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:10.641584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f98051-8929-48d1-b668-8e9aa0608059 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:10.642139 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35093605-baad-40b9-866f-28a1c1aedc31 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:10.642658 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35093605-baad-40b9-866f-28a1c1aedc31 None admin] PATH_INFO: `/v3/system/users/490e3645964e4351a1ada3a117ce13a5/roles/609bb86f2a5b44b0b244b9bfe7c073de` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:10.645531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35093605-baad-40b9-866f-28a1c1aedc31 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=490e3645964e4351a1ada3a117ce13a5, role_id=609bb86f2a5b44b0b244b9bfe7c073de)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:10.647239 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35093605-baad-40b9-866f-28a1c1aedc31 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:10.648154 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 169/332] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:04:10 2022] GET /identity/v3/users/1a1e2e2331c34fd9bc6677933e175300 => generated 293 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:10.654216 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f98051-8929-48d1-b668-8e9aa0608059 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:10.664933 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 164/333] 10.222.0.84 () {58 vars in 1334 bytes} [Thu May 19 20:04:10 2022] PUT /identity/v3/system/users/490e3645964e4351a1ada3a117ce13a5/roles/609bb86f2a5b44b0b244b9bfe7c073de => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:04:10.694208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:10.778905 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f98051-8929-48d1-b668-8e9aa0608059 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:10.823706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f6fd1-1758-48f9-951d-8d9cfbe8c892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:10.824800 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f6fd1-1758-48f9-951d-8d9cfbe8c892 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:10.825305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f6fd1-1758-48f9-951d-8d9cfbe8c892 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:10.825791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f6fd1-1758-48f9-951d-8d9cfbe8c892 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:10.826950 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f6fd1-1758-48f9-951d-8d9cfbe8c892 None admin] RBAC: Adding query filter params (role.id=6aee6f5679c84fc1be21561ddb7d1b56, user.id=1a1e2e2331c34fd9bc6677933e175300, scope.system=all) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:10.827481 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f6fd1-1758-48f9-951d-8d9cfbe8c892 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:10.829254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f6fd1-1758-48f9-951d-8d9cfbe8c892 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:10.843587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 170/334] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 20:04:10 2022] GET /identity/v3/role_assignments?user.id=1a1e2e2331c34fd9bc6677933e175300&scope.system=all&role.id=6aee6f5679c84fc1be21561ddb7d1b56 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:12.373423 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232ca62-bf31-4eb2-bfd6-132f12718698 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:12.373423 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232ca62-bf31-4eb2-bfd6-132f12718698 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:12.373423 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232ca62-bf31-4eb2-bfd6-132f12718698 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:12.374448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 165/335] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:12.392633 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2aa292-66cc-44b6-82fe-13d55f35e5f4 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:12.393167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2aa292-66cc-44b6-82fe-13d55f35e5f4 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:12.393530 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2aa292-66cc-44b6-82fe-13d55f35e5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:12.412736 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-4d2aa292-66cc-44b6-82fe-13d55f35e5f4 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:12.413941 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-4d2aa292-66cc-44b6-82fe-13d55f35e5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:12.499629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 171/336] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:12.524468 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25151d06-32ed-4ad9-ab4d-179d0cffd09a None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:12.525121 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25151d06-32ed-4ad9-ab4d-179d0cffd09a None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:12.525630 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25151d06-32ed-4ad9-ab4d-179d0cffd09a None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:12.527861 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 166/337] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:12.534216 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ed2ee-e133-470b-9bbd-9bc63364767d None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:12.534560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ed2ee-e133-470b-9bbd-9bc63364767d None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:12.534841 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ed2ee-e133-470b-9bbd-9bc63364767d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:12.558199 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-120ed2ee-e133-470b-9bbd-9bc63364767d None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:12.559416 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-120ed2ee-e133-470b-9bbd-9bc63364767d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:12.565370 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25151d06-32ed-4ad9-ab4d-179d0cffd09a None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:12.583624 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:12.708043 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 172/338] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:12.731977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25151d06-32ed-4ad9-ab4d-179d0cffd09a None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:12.786530 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2f94c-a028-42be-b476-889cdd9775a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:12.787359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2f94c-a028-42be-b476-889cdd9775a6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:12.787650 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2f94c-a028-42be-b476-889cdd9775a6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:12.787928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2f94c-a028-42be-b476-889cdd9775a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:12.800843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ed2ee-e133-470b-9bbd-9bc63364767d None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:12.806283 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2f94c-a028-42be-b476-889cdd9775a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-40a2f94c-a028-42be-b476-889cdd9775a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:12.812547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:12.813566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:12.814502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:12.814502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:12.814502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:12.814502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:12.814502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:12.814502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:12.814502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:12.814502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:12.814502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:12.814502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:12.814502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:12.814502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:12.814502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 167/339] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:12.825783 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2f94c-a028-42be-b476-889cdd9775a6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:12.885667 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:12.946955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2f94c-a028-42be-b476-889cdd9775a6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:12.987393 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425823d3-4fa0-4207-8e43-9946c411f955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:12.988336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425823d3-4fa0-4207-8e43-9946c411f955 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:12.989482 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425823d3-4fa0-4207-8e43-9946c411f955 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:12.989953 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425823d3-4fa0-4207-8e43-9946c411f955 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:12.991529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425823d3-4fa0-4207-8e43-9946c411f955 None admin] RBAC: Adding query filter params (name=member) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:12.992172 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425823d3-4fa0-4207-8e43-9946c411f955 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:12.994819 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425823d3-4fa0-4207-8e43-9946c411f955 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:13.001022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ed2ee-e133-470b-9bbd-9bc63364767d None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:13.014926 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 168/340] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:13.020638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425823d3-4fa0-4207-8e43-9946c411f955 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:13.083901 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08924971-81d7-4c26-b034-58d1e816f3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:13.084775 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08924971-81d7-4c26-b034-58d1e816f3c8 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:13.085058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08924971-81d7-4c26-b034-58d1e816f3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:13.085333 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08924971-81d7-4c26-b034-58d1e816f3c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:13.111205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08924971-81d7-4c26-b034-58d1e816f3c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-08924971-81d7-4c26-b034-58d1e816f3c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:13.119415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:13.121082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:13.122106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:13.122106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:13.122106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:13.122106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:13.122106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:13.122106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:13.122106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:13.122106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:13.122106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:13.122106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:13.122106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:13.122106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:13.122106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 173/341] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:04:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:13.127337 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08924971-81d7-4c26-b034-58d1e816f3c8 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:13.147998 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425823d3-4fa0-4207-8e43-9946c411f955 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:13.164205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:13.167610 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:13.205842 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab608252-5c47-4a4d-8a46-0d0b230f1c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:13.207482 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab608252-5c47-4a4d-8a46-0d0b230f1c6e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:13.208114 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab608252-5c47-4a4d-8a46-0d0b230f1c6e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:13.208727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab608252-5c47-4a4d-8a46-0d0b230f1c6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:13.294507 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab608252-5c47-4a4d-8a46-0d0b230f1c6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:13.296308 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab608252-5c47-4a4d-8a46-0d0b230f1c6e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:13.384398 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08924971-81d7-4c26-b034-58d1e816f3c8 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:13.422198 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 169/342] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:13.433211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3d52f-3fa6-469f-9f92-726ea82a5d97 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:13.433817 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3d52f-3fa6-469f-9f92-726ea82a5d97 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:13.434304 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3d52f-3fa6-469f-9f92-726ea82a5d97 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:13.436531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 170/343] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:13 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:13.437619 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3d52f-3fa6-469f-9f92-726ea82a5d97 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:13.456120 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:13.471176 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc65ba1-2d88-40a9-b6ad-8a78c73b5c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:13.471913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc65ba1-2d88-40a9-b6ad-8a78c73b5c33 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:13.472198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc65ba1-2d88-40a9-b6ad-8a78c73b5c33 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:13.472478 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc65ba1-2d88-40a9-b6ad-8a78c73b5c33 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:13.473250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc65ba1-2d88-40a9-b6ad-8a78c73b5c33 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:13.473564 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc65ba1-2d88-40a9-b6ad-8a78c73b5c33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:13.475099 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc65ba1-2d88-40a9-b6ad-8a78c73b5c33 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:13.495995 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 174/344] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:13.497343 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db7e36-d77c-499d-8b0c-643a44875246 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:13.497638 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db7e36-d77c-499d-8b0c-643a44875246 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:13.497917 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db7e36-d77c-499d-8b0c-643a44875246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:13.528657 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-88db7e36-d77c-499d-8b0c-643a44875246 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:13.529865 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-88db7e36-d77c-499d-8b0c-643a44875246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:13.562966 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3d52f-3fa6-469f-9f92-726ea82a5d97 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:13.610146 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71664a29-d1b1-4b69-81b2-ce25805b7ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:13.611769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71664a29-d1b1-4b69-81b2-ce25805b7ffc None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:13.612229 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71664a29-d1b1-4b69-81b2-ce25805b7ffc None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:13.612650 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71664a29-d1b1-4b69-81b2-ce25805b7ffc None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:13.614771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71664a29-d1b1-4b69-81b2-ce25805b7ffc None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:13.616773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71664a29-d1b1-4b69-81b2-ce25805b7ffc None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:13.619812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 171/345] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:13 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:13.620828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71664a29-d1b1-4b69-81b2-ce25805b7ffc None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:13.639931 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 175/346] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:13.641441 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db7e36-d77c-499d-8b0c-643a44875246 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:13.644965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:13.765522 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db7e36-d77c-499d-8b0c-643a44875246 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:13.767839 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:13.811605 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54a365-54ec-40b8-9f82-cfbc762c122b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:13.812836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54a365-54ec-40b8-9f82-cfbc762c122b None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:13.813336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54a365-54ec-40b8-9f82-cfbc762c122b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:13.813473 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71664a29-d1b1-4b69-81b2-ce25805b7ffc None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:13.813895 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54a365-54ec-40b8-9f82-cfbc762c122b None admin] PATH_INFO: `/v3/groups/cf4b8c88f95c4431ad46a14140cf1842` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:13.827127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54a365-54ec-40b8-9f82-cfbc762c122b None admin] RBAC: Authorizing `identity:get_group(group_id=cf4b8c88f95c4431ad46a14140cf1842)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:13.828964 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54a365-54ec-40b8-9f82-cfbc762c122b None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:13.832476 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 176/347] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:04:13 2022] GET /identity/v3/groups/cf4b8c88f95c4431ad46a14140cf1842 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:13.833497 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54a365-54ec-40b8-9f82-cfbc762c122b None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:13.894974 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d3d11-d92a-4edc-856f-e535bf95e620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:13.896843 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d3d11-d92a-4edc-856f-e535bf95e620 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:13.898153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d3d11-d92a-4edc-856f-e535bf95e620 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:13.898607 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d3d11-d92a-4edc-856f-e535bf95e620 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:13.953024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:13.996887 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54a365-54ec-40b8-9f82-cfbc762c122b None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:13.999278 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2d3d11-d92a-4edc-856f-e535bf95e620 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:14.000894 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2d3d11-d92a-4edc-856f-e535bf95e620 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:14.037513 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e5bc1-3ce5-4c67-9910-e3ef01831488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:14.038819 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e5bc1-3ce5-4c67-9910-e3ef01831488 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:14.039316 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e5bc1-3ce5-4c67-9910-e3ef01831488 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:14.039817 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e5bc1-3ce5-4c67-9910-e3ef01831488 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:14.052571 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e5bc1-3ce5-4c67-9910-e3ef01831488 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-562e5bc1-3ce5-4c67-9910-e3ef01831488 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:14.062324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:14.063319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:14.064417 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:14.064417 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:14.064417 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:14.064417 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:14.064417 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:14.064417 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:14.064417 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:14.064417 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:14.064417 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:14.064417 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:14.064417 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:14.064417 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:14.067542 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 177/348] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:14.068581 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e5bc1-3ce5-4c67-9910-e3ef01831488 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:14.097473 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:14.125671 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 172/349] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:14.127089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77fff7-3602-49c6-ade9-41309870642a None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:14.127378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77fff7-3602-49c6-ade9-41309870642a None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:14.127661 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77fff7-3602-49c6-ade9-41309870642a None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:14.129196 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 173/350] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:14.129908 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77fff7-3602-49c6-ade9-41309870642a None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:14.189040 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e5bc1-3ce5-4c67-9910-e3ef01831488 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:14.219125 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:14.231000 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6997e6c-7ef5-49e8-b28f-6c57359db9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:14.232089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6997e6c-7ef5-49e8-b28f-6c57359db9e6 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:14.233436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6997e6c-7ef5-49e8-b28f-6c57359db9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:14.234087 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6997e6c-7ef5-49e8-b28f-6c57359db9e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:14.235339 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6997e6c-7ef5-49e8-b28f-6c57359db9e6 None admin] RBAC: Adding query filter params (group.id=cf4b8c88f95c4431ad46a14140cf1842, role.id=6aee6f5679c84fc1be21561ddb7d1b56, scope.project.id=e490252619914b5d80405cb54917237f) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:14.235887 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6997e6c-7ef5-49e8-b28f-6c57359db9e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:14.237630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6997e6c-7ef5-49e8-b28f-6c57359db9e6 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:14.244602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 178/351] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:04:14 2022] GET /identity/v3/role_assignments?group.id=cf4b8c88f95c4431ad46a14140cf1842&scope.project.id=e490252619914b5d80405cb54917237f&role.id=6aee6f5679c84fc1be21561ddb7d1b56 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:14.246573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3fac4-b3ff-43a3-99af-2443cc9f6adc None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:14.247107 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3fac4-b3ff-43a3-99af-2443cc9f6adc None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:14.247608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3fac4-b3ff-43a3-99af-2443cc9f6adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:14.267229 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-55a3fac4-b3ff-43a3-99af-2443cc9f6adc None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:14.268824 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-55a3fac4-b3ff-43a3-99af-2443cc9f6adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:14.281980 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77fff7-3602-49c6-ade9-41309870642a None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:14.326077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea07e8-3fb3-41c3-8978-0ef3ef495804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:14.326901 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea07e8-3fb3-41c3-8978-0ef3ef495804 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:14.327186 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea07e8-3fb3-41c3-8978-0ef3ef495804 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:14.327481 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea07e8-3fb3-41c3-8978-0ef3ef495804 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:14.328235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea07e8-3fb3-41c3-8978-0ef3ef495804 None admin] RBAC: Adding query filter params (name=member) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:14.328539 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea07e8-3fb3-41c3-8978-0ef3ef495804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:14.331275 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea07e8-3fb3-41c3-8978-0ef3ef495804 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:14.341473 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 174/352] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:14.343001 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea07e8-3fb3-41c3-8978-0ef3ef495804 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:14.356348 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 179/353] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:14.357636 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3fac4-b3ff-43a3-99af-2443cc9f6adc None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:14.414812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:14.476106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3fac4-b3ff-43a3-99af-2443cc9f6adc None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:14.491047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea07e8-3fb3-41c3-8978-0ef3ef495804 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:14.501280 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:14.520707 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8414990-02fe-45d6-8941-00260d607c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:14.521786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8414990-02fe-45d6-8941-00260d607c19 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:14.522312 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8414990-02fe-45d6-8941-00260d607c19 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:14.522800 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8414990-02fe-45d6-8941-00260d607c19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:14.562029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289133e-4e9a-4a57-a6e0-69a3682244a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:14.562986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289133e-4e9a-4a57-a6e0-69a3682244a1 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:14.563274 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289133e-4e9a-4a57-a6e0-69a3682244a1 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:14.563558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289133e-4e9a-4a57-a6e0-69a3682244a1 None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:14.564814 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289133e-4e9a-4a57-a6e0-69a3682244a1 None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:14.566525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289133e-4e9a-4a57-a6e0-69a3682244a1 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:14.568103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 175/354] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:14 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:14.569016 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289133e-4e9a-4a57-a6e0-69a3682244a1 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:14.607378 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8414990-02fe-45d6-8941-00260d607c19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:14.609236 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8414990-02fe-45d6-8941-00260d607c19 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:14.755179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 180/355] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:14.756979 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4ed8e-055d-4082-91dd-a8ed6635d124 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:14.757574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4ed8e-055d-4082-91dd-a8ed6635d124 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:14.758012 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4ed8e-055d-4082-91dd-a8ed6635d124 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:14.759423 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289133e-4e9a-4a57-a6e0-69a3682244a1 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:14.760192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 181/356] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:14.761311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4ed8e-055d-4082-91dd-a8ed6635d124 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:14.774423 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:14.793986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:14.822020 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc66bc-2da0-4d9f-bff7-e6016f9c2872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:14.823301 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc66bc-2da0-4d9f-bff7-e6016f9c2872 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:14.824420 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc66bc-2da0-4d9f-bff7-e6016f9c2872 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:14.824911 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc66bc-2da0-4d9f-bff7-e6016f9c2872 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:14.842149 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc66bc-2da0-4d9f-bff7-e6016f9c2872 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-3bbc66bc-2da0-4d9f-bff7-e6016f9c2872 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:14.856499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:14.857726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:14.860769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:14.860769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:14.860769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:14.860769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:14.860769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:14.860769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:14.860769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:14.860769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:14.860769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:14.860769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:14.860769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:04:14.860769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:14.860769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 176/357] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:14.860769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a474e26-f190-44a7-8a08-5edfbe062d66 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:14.862555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a474e26-f190-44a7-8a08-5edfbe062d66 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:14.862555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a474e26-f190-44a7-8a08-5edfbe062d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:14.889973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-4a474e26-f190-44a7-8a08-5edfbe062d66 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:14.891801 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-4a474e26-f190-44a7-8a08-5edfbe062d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:14.925636 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4ed8e-055d-4082-91dd-a8ed6635d124 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:14.971406 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6cd3b-110b-44ae-9afe-7cba368b4d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:14.972290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6cd3b-110b-44ae-9afe-7cba368b4d39 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:14.972576 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6cd3b-110b-44ae-9afe-7cba368b4d39 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:14.972849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6cd3b-110b-44ae-9afe-7cba368b4d39 None admin] PATH_INFO: `/v3/groups/cf4b8c88f95c4431ad46a14140cf1842` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:14.974951 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6cd3b-110b-44ae-9afe-7cba368b4d39 None admin] RBAC: Authorizing `identity:get_group(group_id=cf4b8c88f95c4431ad46a14140cf1842)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:14.976537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6cd3b-110b-44ae-9afe-7cba368b4d39 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:14.977155 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 177/358] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:14.978234 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a474e26-f190-44a7-8a08-5edfbe062d66 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:14.978480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 182/359] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:04:14 2022] GET /identity/v3/groups/cf4b8c88f95c4431ad46a14140cf1842 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:14.979269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6cd3b-110b-44ae-9afe-7cba368b4d39 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:15.084259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:15.099701 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6cd3b-110b-44ae-9afe-7cba368b4d39 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:15.100609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:15.104053 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a474e26-f190-44a7-8a08-5edfbe062d66 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:15.148490 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43841df7-dc6c-4cd2-8081-ff31c7d9be55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:15.149101 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43841df7-dc6c-4cd2-8081-ff31c7d9be55 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:15.149408 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43841df7-dc6c-4cd2-8081-ff31c7d9be55 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:15.149681 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43841df7-dc6c-4cd2-8081-ff31c7d9be55 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:15.149807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb93c9-52e5-474e-a389-4203b7438cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:15.150474 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43841df7-dc6c-4cd2-8081-ff31c7d9be55 None admin] RBAC: Adding query filter params (name=reader) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:15.150615 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb93c9-52e5-474e-a389-4203b7438cc6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:15.150812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43841df7-dc6c-4cd2-8081-ff31c7d9be55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:15.150924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb93c9-52e5-474e-a389-4203b7438cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:15.151156 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb93c9-52e5-474e-a389-4203b7438cc6 None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:15.152611 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43841df7-dc6c-4cd2-8081-ff31c7d9be55 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:15.152733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb93c9-52e5-474e-a389-4203b7438cc6 None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:15.154509 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb93c9-52e5-474e-a389-4203b7438cc6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:15.156732 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 178/360] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:14 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:15.157619 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb93c9-52e5-474e-a389-4203b7438cc6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:15.165138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 183/361] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:15.166782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43841df7-dc6c-4cd2-8081-ff31c7d9be55 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:15.283954 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb93c9-52e5-474e-a389-4203b7438cc6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:15.312481 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43841df7-dc6c-4cd2-8081-ff31c7d9be55 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:15.325914 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c4954-2450-4bc7-91b9-392804f565af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:15.326677 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c4954-2450-4bc7-91b9-392804f565af None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:15.326956 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c4954-2450-4bc7-91b9-392804f565af None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:15.327510 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c4954-2450-4bc7-91b9-392804f565af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:15.328066 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c4954-2450-4bc7-91b9-392804f565af None admin] RBAC: Adding query filter params (group.id=cf4b8c88f95c4431ad46a14140cf1842, role.id=b21c1004651d40f1b44fb0b7becbe3c8, scope.project.id=e490252619914b5d80405cb54917237f) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:15.328371 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c4954-2450-4bc7-91b9-392804f565af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:15.329834 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c4954-2450-4bc7-91b9-392804f565af None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:15.336300 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 179/362] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:04:15 2022] GET /identity/v3/role_assignments?group.id=cf4b8c88f95c4431ad46a14140cf1842&scope.project.id=e490252619914b5d80405cb54917237f&role.id=b21c1004651d40f1b44fb0b7becbe3c8 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:15.337565 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e040fd-308f-4313-9337-c0b967ceb2e4 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:15.337847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e040fd-308f-4313-9337-c0b967ceb2e4 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:15.338123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e040fd-308f-4313-9337-c0b967ceb2e4 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:15.339765 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 180/363] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:15.340520 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e040fd-308f-4313-9337-c0b967ceb2e4 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:15.372762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e19b3b-9ff1-4c73-a7c4-9a00206c7c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:15.373614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e19b3b-9ff1-4c73-a7c4-9a00206c7c3a None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:15.373899 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e19b3b-9ff1-4c73-a7c4-9a00206c7c3a None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:15.374177 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e19b3b-9ff1-4c73-a7c4-9a00206c7c3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:15.377061 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:15.397990 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e19b3b-9ff1-4c73-a7c4-9a00206c7c3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-f1e19b3b-9ff1-4c73-a7c4-9a00206c7c3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:15.410123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:15.411290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:15.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:15.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:15.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:15.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:15.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:15.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:15.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:15.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:15.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:15.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:15.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:15.412202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:15.415737 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 184/364] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:04:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:15.416872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e19b3b-9ff1-4c73-a7c4-9a00206c7c3a None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:15.435452 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:15.475085 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e040fd-308f-4313-9337-c0b967ceb2e4 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:15.526123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7095a30-2514-46d1-a775-7df86c81c1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:15.527938 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7095a30-2514-46d1-a775-7df86c81c1d1 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:15.528388 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7095a30-2514-46d1-a775-7df86c81c1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:15.528865 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7095a30-2514-46d1-a775-7df86c81c1d1 None admin] PATH_INFO: `/v3/groups/cf4b8c88f95c4431ad46a14140cf1842` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:15.530737 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7095a30-2514-46d1-a775-7df86c81c1d1 None admin] RBAC: Authorizing `identity:get_group(group_id=cf4b8c88f95c4431ad46a14140cf1842)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:15.533299 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7095a30-2514-46d1-a775-7df86c81c1d1 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:15.535691 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 181/365] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:04:15 2022] GET /identity/v3/groups/cf4b8c88f95c4431ad46a14140cf1842 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:15.544752 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfb295-f4bf-487f-bb12-523363c53809 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:15.545491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfb295-f4bf-487f-bb12-523363c53809 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:15.545925 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfb295-f4bf-487f-bb12-523363c53809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:15.577794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-c6cfb295-f4bf-487f-bb12-523363c53809 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:15.579083 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-c6cfb295-f4bf-487f-bb12-523363c53809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:15.585541 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e19b3b-9ff1-4c73-a7c4-9a00206c7c3a None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:15.630811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e90ad5-f21f-46a3-b690-4e30f15ae6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:15.632098 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e90ad5-f21f-46a3-b690-4e30f15ae6d9 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:15.632623 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e90ad5-f21f-46a3-b690-4e30f15ae6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:15.633117 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e90ad5-f21f-46a3-b690-4e30f15ae6d9 None admin] PATH_INFO: `/v3/users/490e3645964e4351a1ada3a117ce13a5` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:15.634613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e90ad5-f21f-46a3-b690-4e30f15ae6d9 None admin] RBAC: Authorizing `identity:get_user(user_id=490e3645964e4351a1ada3a117ce13a5)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:15.636855 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e90ad5-f21f-46a3-b690-4e30f15ae6d9 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:15.639304 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 185/366] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:04:15 2022] GET /identity/v3/users/490e3645964e4351a1ada3a117ce13a5 => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:15.640962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e90ad5-f21f-46a3-b690-4e30f15ae6d9 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:15.673614 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 182/367] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:15.674526 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfb295-f4bf-487f-bb12-523363c53809 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:15.716649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:15.754044 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:15.783583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e90ad5-f21f-46a3-b690-4e30f15ae6d9 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:15.812825 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfb295-f4bf-487f-bb12-523363c53809 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:15.829265 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72c85a-57b4-4822-99aa-49710742c226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:15.830828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72c85a-57b4-4822-99aa-49710742c226 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:15.831574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72c85a-57b4-4822-99aa-49710742c226 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:15.832061 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72c85a-57b4-4822-99aa-49710742c226 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:15.833267 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72c85a-57b4-4822-99aa-49710742c226 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:15.833788 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72c85a-57b4-4822-99aa-49710742c226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:15.835807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72c85a-57b4-4822-99aa-49710742c226 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:15.847686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 186/368] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:15.848862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72c85a-57b4-4822-99aa-49710742c226 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:15.860755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bfeb2-50e8-4320-870c-e9ba70ef013d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:15.861460 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bfeb2-50e8-4320-870c-e9ba70ef013d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:15.861745 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bfeb2-50e8-4320-870c-e9ba70ef013d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:15.862018 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bfeb2-50e8-4320-870c-e9ba70ef013d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:15.862952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bfeb2-50e8-4320-870c-e9ba70ef013d None admin] RBAC: Adding query filter params (group.id=cf4b8c88f95c4431ad46a14140cf1842, role.id=6aee6f5679c84fc1be21561ddb7d1b56, scope.project.id=bb83c47ec18443db80a5cf2ed51d142e) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:15.863273 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bfeb2-50e8-4320-870c-e9ba70ef013d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:15.864867 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bfeb2-50e8-4320-870c-e9ba70ef013d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:15.878446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 183/369] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:04:15 2022] GET /identity/v3/role_assignments?group.id=cf4b8c88f95c4431ad46a14140cf1842&scope.project.id=bb83c47ec18443db80a5cf2ed51d142e&role.id=6aee6f5679c84fc1be21561ddb7d1b56 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:15.879466 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bfeb2-50e8-4320-870c-e9ba70ef013d None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:15.977410 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72c85a-57b4-4822-99aa-49710742c226 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:16.021180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1895a1b-2a5b-48c3-a946-1ee5ea578008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:16.022313 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1895a1b-2a5b-48c3-a946-1ee5ea578008 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:16.022811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1895a1b-2a5b-48c3-a946-1ee5ea578008 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:16.023311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1895a1b-2a5b-48c3-a946-1ee5ea578008 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:16.024383 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1895a1b-2a5b-48c3-a946-1ee5ea578008 None admin] RBAC: Adding query filter params (role.id=609bb86f2a5b44b0b244b9bfe7c073de, user.id=490e3645964e4351a1ada3a117ce13a5, scope.system=all) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:16.024906 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1895a1b-2a5b-48c3-a946-1ee5ea578008 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:16.026830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1895a1b-2a5b-48c3-a946-1ee5ea578008 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:16.034845 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:16.045439 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 187/370] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 20:04:15 2022] GET /identity/v3/role_assignments?user.id=490e3645964e4351a1ada3a117ce13a5&scope.system=all&role.id=609bb86f2a5b44b0b244b9bfe7c073de => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:16.046695 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1895a1b-2a5b-48c3-a946-1ee5ea578008 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:16.152991 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bfeb2-50e8-4320-870c-e9ba70ef013d None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:16.154585 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:16.170863 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1895a1b-2a5b-48c3-a946-1ee5ea578008 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:16.194067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde9344-11b0-4ce4-bcf0-75a5296d13c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:16.195460 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde9344-11b0-4ce4-bcf0-75a5296d13c7 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:16.195460 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde9344-11b0-4ce4-bcf0-75a5296d13c7 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:16.195460 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde9344-11b0-4ce4-bcf0-75a5296d13c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:16.211023 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde9344-11b0-4ce4-bcf0-75a5296d13c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:16.212608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623e067-8714-4501-92ca-b8700b681f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:16.213515 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623e067-8714-4501-92ca-b8700b681f56 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:16.214081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623e067-8714-4501-92ca-b8700b681f56 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:16.214605 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623e067-8714-4501-92ca-b8700b681f56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-ccde9344-11b0-4ce4-bcf0-75a5296d13c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:16.221720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:16.222822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:16.223729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:16.223729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:16.223729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:16.223729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:16.223729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:16.223729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:16.223729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:16.223729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:16.223729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:16.223729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:16.223729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:04:16.223729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:16.223729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 184/371] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:04:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:16.231784 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde9344-11b0-4ce4-bcf0-75a5296d13c7 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:16.291449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:16.297568 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623e067-8714-4501-92ca-b8700b681f56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:16.299449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623e067-8714-4501-92ca-b8700b681f56 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:16.408828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 188/372] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:16.415321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623e067-8714-4501-92ca-b8700b681f56 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:16.492469 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde9344-11b0-4ce4-bcf0-75a5296d13c7 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:16.569993 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d57c9-7b6e-4152-a538-c7f70754821d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:16.573862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d57c9-7b6e-4152-a538-c7f70754821d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:16.574126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d57c9-7b6e-4152-a538-c7f70754821d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:16.574677 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d57c9-7b6e-4152-a538-c7f70754821d None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:16.575210 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d57c9-7b6e-4152-a538-c7f70754821d None admin] RBAC: Adding query filter params (name=admin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:16.575515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d57c9-7b6e-4152-a538-c7f70754821d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:16.576922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d57c9-7b6e-4152-a538-c7f70754821d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:16.598569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 185/373] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:16.607001 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d57c9-7b6e-4152-a538-c7f70754821d None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:16.633892 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:16.634678 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:16.640447 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623e067-8714-4501-92ca-b8700b681f56 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:16.687402 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0e94d-d164-4ea3-979c-80c22793b5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:16.688286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0e94d-d164-4ea3-979c-80c22793b5b6 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:16.688570 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0e94d-d164-4ea3-979c-80c22793b5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:16.688958 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0e94d-d164-4ea3-979c-80c22793b5b6 None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:16.690181 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0e94d-d164-4ea3-979c-80c22793b5b6 None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:16.691786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0e94d-d164-4ea3-979c-80c22793b5b6 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:16.693840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 189/374] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:16 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:16.699331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0e94d-d164-4ea3-979c-80c22793b5b6 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:16.793720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d57c9-7b6e-4152-a538-c7f70754821d None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:16.860589 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba623eb-0510-45eb-a5b5-a23af638a882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:16.861257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba623eb-0510-45eb-a5b5-a23af638a882 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:16.861533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba623eb-0510-45eb-a5b5-a23af638a882 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:16.861811 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba623eb-0510-45eb-a5b5-a23af638a882 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:16.869837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0e94d-d164-4ea3-979c-80c22793b5b6 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:16.922399 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2b9d6-22cd-441d-9b85-dba3585d7372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:16.923663 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2b9d6-22cd-441d-9b85-dba3585d7372 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:16.924189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2b9d6-22cd-441d-9b85-dba3585d7372 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:16.924682 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2b9d6-22cd-441d-9b85-dba3585d7372 None admin] PATH_INFO: `/v3/groups/cf4b8c88f95c4431ad46a14140cf1842` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:16.926099 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab2b9d6-22cd-441d-9b85-dba3585d7372 None admin] RBAC: Authorizing `identity:get_group(group_id=cf4b8c88f95c4431ad46a14140cf1842)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:16.927769 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab2b9d6-22cd-441d-9b85-dba3585d7372 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:16.930001 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 190/375] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:04:16 2022] GET /identity/v3/groups/cf4b8c88f95c4431ad46a14140cf1842 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:16.939309 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2b9d6-22cd-441d-9b85-dba3585d7372 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:16.956315 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba623eb-0510-45eb-a5b5-a23af638a882 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:16.958122 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba623eb-0510-45eb-a5b5-a23af638a882 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:16.981160 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:16.983182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:17.090922 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2b9d6-22cd-441d-9b85-dba3585d7372 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:17.146399 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 186/376] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:04:17.153766 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba623eb-0510-45eb-a5b5-a23af638a882 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:17.157555 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843cbed-5430-48e9-8741-bfb16ce24f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:17.158232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843cbed-5430-48e9-8741-bfb16ce24f5c None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:17.158534 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843cbed-5430-48e9-8741-bfb16ce24f5c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:17.158811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843cbed-5430-48e9-8741-bfb16ce24f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:17.159657 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843cbed-5430-48e9-8741-bfb16ce24f5c None admin] RBAC: Adding query filter params (group.id=cf4b8c88f95c4431ad46a14140cf1842, role.id=b21c1004651d40f1b44fb0b7becbe3c8, scope.project.id=bb83c47ec18443db80a5cf2ed51d142e) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:17.159971 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843cbed-5430-48e9-8741-bfb16ce24f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:17.161410 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843cbed-5430-48e9-8741-bfb16ce24f5c None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:17.175199 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 191/377] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:04:16 2022] GET /identity/v3/role_assignments?group.id=cf4b8c88f95c4431ad46a14140cf1842&scope.project.id=bb83c47ec18443db80a5cf2ed51d142e&role.id=b21c1004651d40f1b44fb0b7becbe3c8 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:17.298147 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba623eb-0510-45eb-a5b5-a23af638a882 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:17.326384 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:17.354461 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201feab5-ed86-4fc2-b988-ce20693bd857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:17.355618 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201feab5-ed86-4fc2-b988-ce20693bd857 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:17.355899 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201feab5-ed86-4fc2-b988-ce20693bd857 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:17.356175 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201feab5-ed86-4fc2-b988-ce20693bd857 None admin] PATH_INFO: `/v3/projects/026346139f294a8c8380faacc93b3be0` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:17.363265 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201feab5-ed86-4fc2-b988-ce20693bd857 None admin] RBAC: Authorizing `identity:get_project(project_id=026346139f294a8c8380faacc93b3be0)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:17.364835 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201feab5-ed86-4fc2-b988-ce20693bd857 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:17.367068 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 187/378] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:17 2022] GET /identity/v3/projects/026346139f294a8c8380faacc93b3be0 => generated 344 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:17.372445 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843cbed-5430-48e9-8741-bfb16ce24f5c None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:17.435068 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce97bd8-19d4-40df-bbb1-88527cafff87 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:17.435429 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce97bd8-19d4-40df-bbb1-88527cafff87 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:17.435600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce97bd8-19d4-40df-bbb1-88527cafff87 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:17.437295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 188/379] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:04:17.442987 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26d69d-d34f-47f6-8d10-f38ee19c17a1 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:17.443299 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26d69d-d34f-47f6-8d10-f38ee19c17a1 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:17.443534 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26d69d-d34f-47f6-8d10-f38ee19c17a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:17.468593 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-4c26d69d-d34f-47f6-8d10-f38ee19c17a1 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:17.470110 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-4c26d69d-d34f-47f6-8d10-f38ee19c17a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:17.508337 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843cbed-5430-48e9-8741-bfb16ce24f5c None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:17.530660 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:17.575089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5279a69-9289-472d-b3d8-575ec0afde28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:17.575992 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5279a69-9289-472d-b3d8-575ec0afde28 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:17.576272 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5279a69-9289-472d-b3d8-575ec0afde28 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:17.576546 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5279a69-9289-472d-b3d8-575ec0afde28 None admin] PATH_INFO: `/v3/groups/ebf2b429b7f5404faa458962f9755a03` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:17.585028 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5279a69-9289-472d-b3d8-575ec0afde28 None admin] RBAC: Authorizing `identity:get_group(group_id=ebf2b429b7f5404faa458962f9755a03)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:17.586546 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5279a69-9289-472d-b3d8-575ec0afde28 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:17.588311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 189/380] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:04:17.588962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 192/381] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:04:17 2022] GET /identity/v3/groups/ebf2b429b7f5404faa458962f9755a03 => generated 229 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:17.594596 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26d69d-d34f-47f6-8d10-f38ee19c17a1 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:17.626478 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5279a69-9289-472d-b3d8-575ec0afde28 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:17.782762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5279a69-9289-472d-b3d8-575ec0afde28 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:17.827412 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad36f2a-061b-4965-a281-5e32113663c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:17.828238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad36f2a-061b-4965-a281-5e32113663c3 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:17.828518 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad36f2a-061b-4965-a281-5e32113663c3 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:17.828795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad36f2a-061b-4965-a281-5e32113663c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:17.840954 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad36f2a-061b-4965-a281-5e32113663c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:17.848376 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26d69d-d34f-47f6-8d10-f38ee19c17a1 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-0ad36f2a-061b-4965-a281-5e32113663c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:17.858111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:17.859196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:17.861188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:17.861188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:17.861188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:17.861188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:17.861188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:17.861188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:17.861188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:17.861188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:17.861188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:17.861188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:17.861188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:17.861188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:17.861188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 193/382] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:17.866566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad36f2a-061b-4965-a281-5e32113663c3 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:17.883957 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:17.905163 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:17.943021 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba65cd-58a9-4149-999f-7f2c2635e4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:17.943689 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba65cd-58a9-4149-999f-7f2c2635e4c5 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:17.943966 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba65cd-58a9-4149-999f-7f2c2635e4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:17.944237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba65cd-58a9-4149-999f-7f2c2635e4c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:17.945079 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba65cd-58a9-4149-999f-7f2c2635e4c5 None admin] RBAC: Adding query filter params (group.id=ebf2b429b7f5404faa458962f9755a03, role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=026346139f294a8c8380faacc93b3be0) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:17.945387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba65cd-58a9-4149-999f-7f2c2635e4c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:17.954362 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba65cd-58a9-4149-999f-7f2c2635e4c5 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:17.972358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 190/383] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:04:17 2022] GET /identity/v3/role_assignments?group.id=ebf2b429b7f5404faa458962f9755a03&scope.project.id=026346139f294a8c8380faacc93b3be0&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 260 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:18.013375 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad36f2a-061b-4965-a281-5e32113663c3 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:18.092674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ada4a-f7d0-4a24-a17b-43d4e530bb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:18.093366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ada4a-f7d0-4a24-a17b-43d4e530bb05 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:18.093699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ada4a-f7d0-4a24-a17b-43d4e530bb05 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:18.093986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ada4a-f7d0-4a24-a17b-43d4e530bb05 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:18.095136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ada4a-f7d0-4a24-a17b-43d4e530bb05 None admin] RBAC: Adding query filter params (name=member) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:18.095446 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ada4a-f7d0-4a24-a17b-43d4e530bb05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:18.096840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ada4a-f7d0-4a24-a17b-43d4e530bb05 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:18.111819 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 194/384] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:18.117533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba65cd-58a9-4149-999f-7f2c2635e4c5 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:18.272091 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba65cd-58a9-4149-999f-7f2c2635e4c5 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:18.311118 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7add3-34a6-49df-97e8-459eb8761ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:18.311993 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7add3-34a6-49df-97e8-459eb8761ea5 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:18.312267 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7add3-34a6-49df-97e8-459eb8761ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:18.312538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7add3-34a6-49df-97e8-459eb8761ea5 None admin] PATH_INFO: `/v3/groups/cf4b8c88f95c4431ad46a14140cf1842` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:18.314223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7add3-34a6-49df-97e8-459eb8761ea5 None admin] RBAC: Authorizing `identity:get_group(group_id=cf4b8c88f95c4431ad46a14140cf1842)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:18.315740 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7add3-34a6-49df-97e8-459eb8761ea5 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:18.317698 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 191/385] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:04:18 2022] GET /identity/v3/groups/cf4b8c88f95c4431ad46a14140cf1842 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:18.323011 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ada4a-f7d0-4a24-a17b-43d4e530bb05 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:18.409806 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650a81e-7ce4-4314-85e9-0c9f22249cb6 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:18.410168 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650a81e-7ce4-4314-85e9-0c9f22249cb6 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:18.410397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650a81e-7ce4-4314-85e9-0c9f22249cb6 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:18.411908 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 192/386] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:04:18.417562 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6999fd7-c1d6-459e-885f-530b6d8ef4e2 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:18.418135 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6999fd7-c1d6-459e-885f-530b6d8ef4e2 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:18.418653 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6999fd7-c1d6-459e-885f-530b6d8ef4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:18.431210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:18.443736 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ada4a-f7d0-4a24-a17b-43d4e530bb05 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:18.458282 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-f6999fd7-c1d6-459e-885f-530b6d8ef4e2 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:18.459472 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-f6999fd7-c1d6-459e-885f-530b6d8ef4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:18.487690 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d088b-b560-4ae7-90cb-5d529d587c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:18.489212 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d088b-b560-4ae7-90cb-5d529d587c9c None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:18.489733 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d088b-b560-4ae7-90cb-5d529d587c9c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:18.490213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d088b-b560-4ae7-90cb-5d529d587c9c None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:18.492151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d088b-b560-4ae7-90cb-5d529d587c9c None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:18.493945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d088b-b560-4ae7-90cb-5d529d587c9c None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:18.497180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 195/387] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:18 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:18.504680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d088b-b560-4ae7-90cb-5d529d587c9c None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:18.557281 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:18.615216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 193/388] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:04:18.628416 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d088b-b560-4ae7-90cb-5d529d587c9c None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:18.647883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6999fd7-c1d6-459e-885f-530b6d8ef4e2 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:18.664733 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dea2c0-8c5a-4ecf-bc02-45e4a42d47b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:18.665637 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dea2c0-8c5a-4ecf-bc02-45e4a42d47b6 None admin] REQUEST_METHOD: `PUT` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:18.665919 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dea2c0-8c5a-4ecf-bc02-45e4a42d47b6 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:18.666316 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dea2c0-8c5a-4ecf-bc02-45e4a42d47b6 None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f/groups/cf4b8c88f95c4431ad46a14140cf1842/roles/6aee6f5679c84fc1be21561ddb7d1b56` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:18.668906 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dea2c0-8c5a-4ecf-bc02-45e4a42d47b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e490252619914b5d80405cb54917237f, group_id=cf4b8c88f95c4431ad46a14140cf1842, role_id=6aee6f5679c84fc1be21561ddb7d1b56)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:18.670515 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dea2c0-8c5a-4ecf-bc02-45e4a42d47b6 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:18.687242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 196/389] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 20:04:18 2022] PUT /identity/v3/projects/e490252619914b5d80405cb54917237f/groups/cf4b8c88f95c4431ad46a14140cf1842/roles/6aee6f5679c84fc1be21561ddb7d1b56 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:04:18.773334 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6999fd7-c1d6-459e-885f-530b6d8ef4e2 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:18.811296 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe30b33-93c9-4458-80b8-6daf9e2921e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:18.812054 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe30b33-93c9-4458-80b8-6daf9e2921e5 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:18.812333 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe30b33-93c9-4458-80b8-6daf9e2921e5 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:18.812604 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe30b33-93c9-4458-80b8-6daf9e2921e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:18.824766 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe30b33-93c9-4458-80b8-6daf9e2921e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-bfe30b33-93c9-4458-80b8-6daf9e2921e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:18.833204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:18.834295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:18.835211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:18.835211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:18.835211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:18.835211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:18.835211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:18.835211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:18.835211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:18.835211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:18.835211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:18.835211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:18.835211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:18.835211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:18.835211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 194/390] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:04:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:18.841966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dea2c0-8c5a-4ecf-bc02-45e4a42d47b6 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:18.862668 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:18.970853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dea2c0-8c5a-4ecf-bc02-45e4a42d47b6 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:19.005960 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d9ce8-82af-4894-8a39-418496dcc785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:19.006677 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d9ce8-82af-4894-8a39-418496dcc785 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:19.006962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d9ce8-82af-4894-8a39-418496dcc785 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:19.007240 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d9ce8-82af-4894-8a39-418496dcc785 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:19.008005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d9ce8-82af-4894-8a39-418496dcc785 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:19.008311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d9ce8-82af-4894-8a39-418496dcc785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:19.009693 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d9ce8-82af-4894-8a39-418496dcc785 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:19.019812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 197/391] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:19.025786 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe30b33-93c9-4458-80b8-6daf9e2921e5 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:19.077515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:19.112905 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007f0a2-247e-4848-a94a-b55f6a3c39d4 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:19.113677 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007f0a2-247e-4848-a94a-b55f6a3c39d4 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:19.114305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007f0a2-247e-4848-a94a-b55f6a3c39d4 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:19.115905 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 198/392] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:04:19.121649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340f345-3cb7-47a4-be31-d6e9344b25ec None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:19.122321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340f345-3cb7-47a4-be31-d6e9344b25ec None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:19.122826 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340f345-3cb7-47a4-be31-d6e9344b25ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:19.141222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-f340f345-3cb7-47a4-be31-d6e9344b25ec None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:19.142883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-f340f345-3cb7-47a4-be31-d6e9344b25ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:19.168923 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe30b33-93c9-4458-80b8-6daf9e2921e5 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:19.226943 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 199/393] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:04:19.247560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96674a2c-8fb8-43c1-b76c-756517968e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:19.248440 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96674a2c-8fb8-43c1-b76c-756517968e85 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:19.248720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96674a2c-8fb8-43c1-b76c-756517968e85 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:19.248993 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96674a2c-8fb8-43c1-b76c-756517968e85 None admin] PATH_INFO: `/v3/groups/cf4b8c88f95c4431ad46a14140cf1842` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:19.255055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96674a2c-8fb8-43c1-b76c-756517968e85 None admin] RBAC: Authorizing `identity:get_group(group_id=cf4b8c88f95c4431ad46a14140cf1842)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:19.256089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340f345-3cb7-47a4-be31-d6e9344b25ec None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:19.256537 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96674a2c-8fb8-43c1-b76c-756517968e85 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:19.258501 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 195/394] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:04:19 2022] GET /identity/v3/groups/cf4b8c88f95c4431ad46a14140cf1842 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:19.263720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96674a2c-8fb8-43c1-b76c-756517968e85 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:19.282294 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:19.377458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96674a2c-8fb8-43c1-b76c-756517968e85 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:19.380396 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:19.413624 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d92f8d-5c60-4adb-9eba-3959e71ef0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:19.414712 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d92f8d-5c60-4adb-9eba-3959e71ef0a7 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:19.415111 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d92f8d-5c60-4adb-9eba-3959e71ef0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:19.415442 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d92f8d-5c60-4adb-9eba-3959e71ef0a7 None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:19.418982 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d92f8d-5c60-4adb-9eba-3959e71ef0a7 None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:19.418982 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d92f8d-5c60-4adb-9eba-3959e71ef0a7 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:19.420525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 196/395] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:19 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:19.426086 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d92f8d-5c60-4adb-9eba-3959e71ef0a7 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:19.428578 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340f345-3cb7-47a4-be31-d6e9344b25ec None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:19.513275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac2606-f5f0-478a-9ec3-7b78fd4ac27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:19.514422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac2606-f5f0-478a-9ec3-7b78fd4ac27e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:19.514877 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac2606-f5f0-478a-9ec3-7b78fd4ac27e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:19.515308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac2606-f5f0-478a-9ec3-7b78fd4ac27e None admin] PATH_INFO: `/v3/roles/member` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:19.537971 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d92f8d-5c60-4adb-9eba-3959e71ef0a7 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:19.546925 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac2606-f5f0-478a-9ec3-7b78fd4ac27e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-7fac2606-f5f0-478a-9ec3-7b78fd4ac27e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:19.555138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:19.556113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:19.557050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:19.557050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:19.557050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:19.557050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:19.557050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:19.557050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:19.557050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:19.557050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:19.557050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:19.557050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:19.557050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:19.557050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:19.560267 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 200/396] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:19.566376 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac2606-f5f0-478a-9ec3-7b78fd4ac27e None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:19.598154 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cced565-8cf0-4b02-bf74-2ef479894b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:19.599091 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cced565-8cf0-4b02-bf74-2ef479894b2e None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:19.599365 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cced565-8cf0-4b02-bf74-2ef479894b2e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:19.599635 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cced565-8cf0-4b02-bf74-2ef479894b2e None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f/groups/cf4b8c88f95c4431ad46a14140cf1842/roles/b21c1004651d40f1b44fb0b7becbe3c8` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:19.603714 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cced565-8cf0-4b02-bf74-2ef479894b2e None admin] RBAC: Authorizing `identity:create_grant(project_id=e490252619914b5d80405cb54917237f, group_id=cf4b8c88f95c4431ad46a14140cf1842, role_id=b21c1004651d40f1b44fb0b7becbe3c8)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:19.605319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cced565-8cf0-4b02-bf74-2ef479894b2e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:19.631530 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 197/397] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 20:04:19 2022] PUT /identity/v3/projects/e490252619914b5d80405cb54917237f/groups/cf4b8c88f95c4431ad46a14140cf1842/roles/b21c1004651d40f1b44fb0b7becbe3c8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:04:19.651413 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:19.713293 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac2606-f5f0-478a-9ec3-7b78fd4ac27e None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:19.758297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602934f-5114-4dda-9ddc-7953b699fb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:19.759049 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602934f-5114-4dda-9ddc-7953b699fb35 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:19.759344 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602934f-5114-4dda-9ddc-7953b699fb35 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:19.759614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602934f-5114-4dda-9ddc-7953b699fb35 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:19.760389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602934f-5114-4dda-9ddc-7953b699fb35 None admin] RBAC: Adding query filter params (name=member) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:19.760826 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602934f-5114-4dda-9ddc-7953b699fb35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:19.762287 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602934f-5114-4dda-9ddc-7953b699fb35 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:19.772843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 201/398] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:19.778325 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cced565-8cf0-4b02-bf74-2ef479894b2e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:19.837265 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:19.929895 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cced565-8cf0-4b02-bf74-2ef479894b2e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:19.965574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d33255-4464-432a-8aaf-a1cea78be01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:19.966853 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d33255-4464-432a-8aaf-a1cea78be01e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:19.967365 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d33255-4464-432a-8aaf-a1cea78be01e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:19.967847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d33255-4464-432a-8aaf-a1cea78be01e None admin] PATH_INFO: `/v3/groups/cf4b8c88f95c4431ad46a14140cf1842` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:19.969608 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d33255-4464-432a-8aaf-a1cea78be01e None admin] RBAC: Authorizing `identity:get_group(group_id=cf4b8c88f95c4431ad46a14140cf1842)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:19.971418 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d33255-4464-432a-8aaf-a1cea78be01e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:19.973295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 198/399] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:04:19 2022] GET /identity/v3/groups/cf4b8c88f95c4431ad46a14140cf1842 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:19.978990 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602934f-5114-4dda-9ddc-7953b699fb35 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:20.101129 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602934f-5114-4dda-9ddc-7953b699fb35 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:20.130039 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:20.141683 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2ef58-3cac-491c-b04e-8369f78740a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:20.142688 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2ef58-3cac-491c-b04e-8369f78740a2 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:20.142969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2ef58-3cac-491c-b04e-8369f78740a2 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:20.143285 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2ef58-3cac-491c-b04e-8369f78740a2 None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:20.144573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2ef58-3cac-491c-b04e-8369f78740a2 None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:20.146133 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2ef58-3cac-491c-b04e-8369f78740a2 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:20.148235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 202/400] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:19 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:20.154097 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d33255-4464-432a-8aaf-a1cea78be01e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:20.279790 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:20.283353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d33255-4464-432a-8aaf-a1cea78be01e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:20.322515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104782b-e826-4fa1-b5cc-45d8898f1a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:20.323890 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104782b-e826-4fa1-b5cc-45d8898f1a80 None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:20.324403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104782b-e826-4fa1-b5cc-45d8898f1a80 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:20.324918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104782b-e826-4fa1-b5cc-45d8898f1a80 None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e/groups/cf4b8c88f95c4431ad46a14140cf1842/roles/6aee6f5679c84fc1be21561ddb7d1b56` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:20.327225 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104782b-e826-4fa1-b5cc-45d8898f1a80 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb83c47ec18443db80a5cf2ed51d142e, group_id=cf4b8c88f95c4431ad46a14140cf1842, role_id=6aee6f5679c84fc1be21561ddb7d1b56)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:20.328957 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104782b-e826-4fa1-b5cc-45d8898f1a80 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:20.356400 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 199/401] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 20:04:20 2022] PUT /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e/groups/cf4b8c88f95c4431ad46a14140cf1842/roles/6aee6f5679c84fc1be21561ddb7d1b56 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:04:20.379447 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426b801-cde7-4d31-995d-e8a0bc894d48 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:20.379805 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426b801-cde7-4d31-995d-e8a0bc894d48 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:20.380078 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426b801-cde7-4d31-995d-e8a0bc894d48 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:20.381702 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 203/402] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:20.387011 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90c83d-4c47-4783-9de7-637b19bf7aab None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:20.387662 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90c83d-4c47-4783-9de7-637b19bf7aab None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:20.388245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90c83d-4c47-4783-9de7-637b19bf7aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:20.409503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-fe90c83d-4c47-4783-9de7-637b19bf7aab None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:20.411711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-fe90c83d-4c47-4783-9de7-637b19bf7aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:20.491961 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 200/403] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:20.516807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426b801-cde7-4d31-995d-e8a0bc894d48 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:20.657030 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426b801-cde7-4d31-995d-e8a0bc894d48 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:20.704152 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad91e51-e7b7-498d-b2d5-fc70c30d7184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:20.704978 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad91e51-e7b7-498d-b2d5-fc70c30d7184 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:20.705256 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad91e51-e7b7-498d-b2d5-fc70c30d7184 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:20.705530 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad91e51-e7b7-498d-b2d5-fc70c30d7184 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:20.721660 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad91e51-e7b7-498d-b2d5-fc70c30d7184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-dad91e51-e7b7-498d-b2d5-fc70c30d7184 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:20.730131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:20.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:20.733183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:20.733183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:20.733183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:20.733183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:20.733183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:20.733183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:20.733183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:20.733183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:20.733183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:20.733183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:20.733183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:20.733183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:20.733183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 204/404] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:04:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:20.743726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90c83d-4c47-4783-9de7-637b19bf7aab None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:20.841967 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:20.855764 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90c83d-4c47-4783-9de7-637b19bf7aab None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:20.894464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f05818-9362-44f6-9ff8-843506490a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:20.895143 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f05818-9362-44f6-9ff8-843506490a46 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:20.895429 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f05818-9362-44f6-9ff8-843506490a46 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:20.895699 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f05818-9362-44f6-9ff8-843506490a46 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:20.896446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f05818-9362-44f6-9ff8-843506490a46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:20.896745 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f05818-9362-44f6-9ff8-843506490a46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:20.898121 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f05818-9362-44f6-9ff8-843506490a46 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:20.907332 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 201/405] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:20.913044 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad91e51-e7b7-498d-b2d5-fc70c30d7184 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:20.985460 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:21.048910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad91e51-e7b7-498d-b2d5-fc70c30d7184 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:21.087713 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a43d3-c4b5-4b2b-b37e-91513947dd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:21.089019 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a43d3-c4b5-4b2b-b37e-91513947dd1f None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:21.089529 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a43d3-c4b5-4b2b-b37e-91513947dd1f None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:21.090147 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a43d3-c4b5-4b2b-b37e-91513947dd1f None admin] PATH_INFO: `/v3/groups/cf4b8c88f95c4431ad46a14140cf1842` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:21.091629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a43d3-c4b5-4b2b-b37e-91513947dd1f None admin] RBAC: Authorizing `identity:get_group(group_id=cf4b8c88f95c4431ad46a14140cf1842)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:21.093362 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a43d3-c4b5-4b2b-b37e-91513947dd1f None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:21.095932 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 205/406] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:04:20 2022] GET /identity/v3/groups/cf4b8c88f95c4431ad46a14140cf1842 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:21.102284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f05818-9362-44f6-9ff8-843506490a46 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:21.241557 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f05818-9362-44f6-9ff8-843506490a46 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:21.292030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf30294-4126-4ce4-a06f-eb739e085e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:21.293020 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf30294-4126-4ce4-a06f-eb739e085e4a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:21.293296 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf30294-4126-4ce4-a06f-eb739e085e4a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:21.293565 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf30294-4126-4ce4-a06f-eb739e085e4a None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:21.293678 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6eece-d795-4f4a-ba30-cca602c50ee3 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:21.293897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6eece-d795-4f4a-ba30-cca602c50ee3 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:21.294177 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6eece-d795-4f4a-ba30-cca602c50ee3 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:21.294825 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf30294-4126-4ce4-a06f-eb739e085e4a None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:21.295807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 206/407] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:21.296354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf30294-4126-4ce4-a06f-eb739e085e4a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:21.298298 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 202/408] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:21 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:21.301043 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeca389-1b80-4b70-9c44-c959308e35cf None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:21.301329 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeca389-1b80-4b70-9c44-c959308e35cf None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:21.301606 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeca389-1b80-4b70-9c44-c959308e35cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:21.303878 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf30294-4126-4ce4-a06f-eb739e085e4a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:21.319292 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-8eeca389-1b80-4b70-9c44-c959308e35cf None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:21.320475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-8eeca389-1b80-4b70-9c44-c959308e35cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:21.342520 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:21.390536 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 207/409] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:04:21.413166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeca389-1b80-4b70-9c44-c959308e35cf None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:21.465801 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:21.484423 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf30294-4126-4ce4-a06f-eb739e085e4a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:21.513699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeca389-1b80-4b70-9c44-c959308e35cf None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:21.555228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce882e77-a68d-440d-beb0-49e2266ff0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:21.556047 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce882e77-a68d-440d-beb0-49e2266ff0cf None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:21.556326 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce882e77-a68d-440d-beb0-49e2266ff0cf None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:21.556596 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce882e77-a68d-440d-beb0-49e2266ff0cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:21.571047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26e671-c464-42ec-9895-53f7edd764f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:21.571269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce882e77-a68d-440d-beb0-49e2266ff0cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:21.572032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26e671-c464-42ec-9895-53f7edd764f4 None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:21.572257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26e671-c464-42ec-9895-53f7edd764f4 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:21.572538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26e671-c464-42ec-9895-53f7edd764f4 None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e/groups/cf4b8c88f95c4431ad46a14140cf1842/roles/b21c1004651d40f1b44fb0b7becbe3c8` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-ce882e77-a68d-440d-beb0-49e2266ff0cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:21.579234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:21.580232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:21.581560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:21.581560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:21.581560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:21.581560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:21.581560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:21.581560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:21.581560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:21.581560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:21.581560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:21.581560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:21.581560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:04:21.581560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:21.582696 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26e671-c464-42ec-9895-53f7edd764f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb83c47ec18443db80a5cf2ed51d142e, group_id=cf4b8c88f95c4431ad46a14140cf1842, role_id=b21c1004651d40f1b44fb0b7becbe3c8)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:21.582789 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 208/410] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:04:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:21.582865 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26e671-c464-42ec-9895-53f7edd764f4 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:21.586442 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce882e77-a68d-440d-beb0-49e2266ff0cf None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:21.614427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 203/411] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 20:04:21 2022] PUT /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e/groups/cf4b8c88f95c4431ad46a14140cf1842/roles/b21c1004651d40f1b44fb0b7becbe3c8 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:04:21.703790 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce882e77-a68d-440d-beb0-49e2266ff0cf None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:21.718670 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:21.743599 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa6e86-e846-411f-9c9d-842af6646444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:21.744317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa6e86-e846-411f-9c9d-842af6646444 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:21.744596 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa6e86-e846-411f-9c9d-842af6646444 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:21.744871 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa6e86-e846-411f-9c9d-842af6646444 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:21.745775 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa6e86-e846-411f-9c9d-842af6646444 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:21.746084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa6e86-e846-411f-9c9d-842af6646444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:21.747524 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa6e86-e846-411f-9c9d-842af6646444 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:21.757315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 209/412] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:21.763292 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26e671-c464-42ec-9895-53f7edd764f4 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:21.894960 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:21.913443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755d756-4405-454c-bf99-6375ca8223fe None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:21.913731 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755d756-4405-454c-bf99-6375ca8223fe None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:21.914010 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755d756-4405-454c-bf99-6375ca8223fe None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:21.915540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 210/413] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:21.920807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26e671-c464-42ec-9895-53f7edd764f4 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:21.922309 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a3b7e-4c77-4172-b65e-e4120f3ca32a None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:21.922598 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a3b7e-4c77-4172-b65e-e4120f3ca32a None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:21.922876 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a3b7e-4c77-4172-b65e-e4120f3ca32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:21.959713 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-120a3b7e-4c77-4172-b65e-e4120f3ca32a None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:21.961432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-120a3b7e-4c77-4172-b65e-e4120f3ca32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:21.982746 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45bc50-8898-4535-9957-614747a01438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:21.985686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45bc50-8898-4535-9957-614747a01438 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:21.986189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45bc50-8898-4535-9957-614747a01438 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:21.987016 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45bc50-8898-4535-9957-614747a01438 None admin] PATH_INFO: `/v3/groups/ebf2b429b7f5404faa458962f9755a03` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:21.989572 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45bc50-8898-4535-9957-614747a01438 None admin] RBAC: Authorizing `identity:get_group(group_id=ebf2b429b7f5404faa458962f9755a03)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:21.991525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45bc50-8898-4535-9957-614747a01438 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:21.994121 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 204/414] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:04:21 2022] GET /identity/v3/groups/ebf2b429b7f5404faa458962f9755a03 => generated 229 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:21.999875 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45bc50-8898-4535-9957-614747a01438 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:22.113450 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 211/415] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:22.139296 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a3b7e-4c77-4172-b65e-e4120f3ca32a None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:22.204056 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45bc50-8898-4535-9957-614747a01438 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:22.271230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613e344-d414-40e9-94a3-7ac55b5f2510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:22.273655 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613e344-d414-40e9-94a3-7ac55b5f2510 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:22.274378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613e344-d414-40e9-94a3-7ac55b5f2510 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:22.275490 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613e344-d414-40e9-94a3-7ac55b5f2510 None admin] PATH_INFO: `/v3/projects/026346139f294a8c8380faacc93b3be0` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:22.278679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613e344-d414-40e9-94a3-7ac55b5f2510 None admin] RBAC: Authorizing `identity:get_project(project_id=026346139f294a8c8380faacc93b3be0)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:22.283939 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613e344-d414-40e9-94a3-7ac55b5f2510 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:22.286002 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 205/416] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:21 2022] GET /identity/v3/projects/026346139f294a8c8380faacc93b3be0 => generated 344 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:22.291684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613e344-d414-40e9-94a3-7ac55b5f2510 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:22.327081 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:22.344692 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a3b7e-4c77-4172-b65e-e4120f3ca32a None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:22.350999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:22.424711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613e344-d414-40e9-94a3-7ac55b5f2510 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:22.447894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d58b3-09e1-460c-b8df-7dbf7e971446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:22.448777 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d58b3-09e1-460c-b8df-7dbf7e971446 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:22.449179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d58b3-09e1-460c-b8df-7dbf7e971446 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:22.449458 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d58b3-09e1-460c-b8df-7dbf7e971446 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:22.459720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21d2cf-405b-4848-a538-18fb4ec62453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:22.460773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21d2cf-405b-4848-a538-18fb4ec62453 None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:22.461059 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21d2cf-405b-4848-a538-18fb4ec62453 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:22.461337 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21d2cf-405b-4848-a538-18fb4ec62453 None admin] PATH_INFO: `/v3/projects/026346139f294a8c8380faacc93b3be0/groups/ebf2b429b7f5404faa458962f9755a03/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:22.463331 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21d2cf-405b-4848-a538-18fb4ec62453 None admin] RBAC: Authorizing `identity:create_grant(project_id=026346139f294a8c8380faacc93b3be0, group_id=ebf2b429b7f5404faa458962f9755a03, role_id=b34e4ad4fe7b4a60bd2fc669e2b31a66)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:22.464819 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21d2cf-405b-4848-a538-18fb4ec62453 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:22.480361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 206/417] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 20:04:22 2022] PUT /identity/v3/projects/026346139f294a8c8380faacc93b3be0/groups/ebf2b429b7f5404faa458962f9755a03/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:04:22.495708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d58b3-09e1-460c-b8df-7dbf7e971446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-ff5d58b3-09e1-460c-b8df-7dbf7e971446 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:22.503554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:22.504843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:22.506213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:22.506213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:22.506213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:22.506213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:22.506213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:22.506213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:22.506213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:22.506213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:22.506213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:22.506213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:22.506213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:22.506213 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:22.510897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 212/418] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:22.517531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21d2cf-405b-4848-a538-18fb4ec62453 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:22.657789 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:22.683231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21d2cf-405b-4848-a538-18fb4ec62453 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:22.719486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a699c-d5c8-42e1-ad1e-d130989f3acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:22.720188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a699c-d5c8-42e1-ad1e-d130989f3acb None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:22.720463 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a699c-d5c8-42e1-ad1e-d130989f3acb None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:22.720730 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a699c-d5c8-42e1-ad1e-d130989f3acb None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:22.721493 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a699c-d5c8-42e1-ad1e-d130989f3acb None admin] RBAC: Adding query filter params (name=member) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:22.721796 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a699c-d5c8-42e1-ad1e-d130989f3acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:22.723240 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a699c-d5c8-42e1-ad1e-d130989f3acb None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:22.733428 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 207/419] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:22.739222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d58b3-09e1-460c-b8df-7dbf7e971446 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:22.800359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98431cf8-7845-4614-a5ee-2c04a17e9e74 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:22.801054 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98431cf8-7845-4614-a5ee-2c04a17e9e74 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:22.801342 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98431cf8-7845-4614-a5ee-2c04a17e9e74 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:22.803338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 208/420] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:22.816576 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa79585-2769-4597-9c03-1077d15d0b48 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:22.816893 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa79585-2769-4597-9c03-1077d15d0b48 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:22.817543 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa79585-2769-4597-9c03-1077d15d0b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:22.840908 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-bfa79585-2769-4597-9c03-1077d15d0b48 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:22.842032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-bfa79585-2769-4597-9c03-1077d15d0b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:22.883230 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d58b3-09e1-460c-b8df-7dbf7e971446 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:22.916151 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 209/421] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:04:22.939943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa79585-2769-4597-9c03-1077d15d0b48 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:22.959631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324ee65-dec2-4543-836f-dc63d02b53b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:22.960302 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ee65-dec2-4543-836f-dc63d02b53b6 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:22.960582 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ee65-dec2-4543-836f-dc63d02b53b6 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:22.960854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ee65-dec2-4543-836f-dc63d02b53b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:23.023700 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:23.073960 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:23.074863 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324ee65-dec2-4543-836f-dc63d02b53b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:23.076843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324ee65-dec2-4543-836f-dc63d02b53b6 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:23.147957 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa79585-2769-4597-9c03-1077d15d0b48 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:23.195618 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7cb12-309d-4d51-ad49-08dded4756c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:23.196744 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7cb12-309d-4d51-ad49-08dded4756c7 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:23.197189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7cb12-309d-4d51-ad49-08dded4756c7 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:23.197701 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7cb12-309d-4d51-ad49-08dded4756c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:23.210153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7cb12-309d-4d51-ad49-08dded4756c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-1ad7cb12-309d-4d51-ad49-08dded4756c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:23.218679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:23.220209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:23.221290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:23.221290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:23.221290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:23.221290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:23.221290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:23.221290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:23.221290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:23.221290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:23.221290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:23.221290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:23.221290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:23.221290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:23.223679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 210/422] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:04:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:23.229723 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7cb12-309d-4d51-ad49-08dded4756c7 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:23.307456 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 213/423] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:23.313423 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324ee65-dec2-4543-836f-dc63d02b53b6 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:23.339590 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7cb12-309d-4d51-ad49-08dded4756c7 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:23.368144 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:23.381017 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d725e6-7978-4f79-b552-ba943fe1a37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:23.382603 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d725e6-7978-4f79-b552-ba943fe1a37e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:23.383417 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d725e6-7978-4f79-b552-ba943fe1a37e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:23.383925 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d725e6-7978-4f79-b552-ba943fe1a37e None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:23.385640 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d725e6-7978-4f79-b552-ba943fe1a37e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:23.386788 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d725e6-7978-4f79-b552-ba943fe1a37e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:23.389388 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d725e6-7978-4f79-b552-ba943fe1a37e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:23.391660 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:23.403395 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 211/424] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:23.405025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2a89f-7183-4fa3-ab28-50dc9cf35a67 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:23.405536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2a89f-7183-4fa3-ab28-50dc9cf35a67 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:23.406045 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2a89f-7183-4fa3-ab28-50dc9cf35a67 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:23.407777 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 212/425] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:23.410994 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2a89f-7183-4fa3-ab28-50dc9cf35a67 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:23.447180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324ee65-dec2-4543-836f-dc63d02b53b6 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:23.503218 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca492922-f95a-4c9a-8163-cdea6b6902c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:23.504881 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca492922-f95a-4c9a-8163-cdea6b6902c6 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:23.505398 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca492922-f95a-4c9a-8163-cdea6b6902c6 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:23.505965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca492922-f95a-4c9a-8163-cdea6b6902c6 None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:23.507944 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca492922-f95a-4c9a-8163-cdea6b6902c6 None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:23.510184 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca492922-f95a-4c9a-8163-cdea6b6902c6 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:23.518320 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 214/426] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:23 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:23.518320 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cf76c-1851-4b13-9e61-aafabd1b5c98 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:23.518320 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cf76c-1851-4b13-9e61-aafabd1b5c98 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:23.518320 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cf76c-1851-4b13-9e61-aafabd1b5c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:23.528919 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2a89f-7183-4fa3-ab28-50dc9cf35a67 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:23.537252 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-032cf76c-1851-4b13-9e61-aafabd1b5c98 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:23.538665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-032cf76c-1851-4b13-9e61-aafabd1b5c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:23.578452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693587d-7dc8-4543-a9d9-859a8cf78a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:23.579317 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693587d-7dc8-4543-a9d9-859a8cf78a9e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:23.579698 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693587d-7dc8-4543-a9d9-859a8cf78a9e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:23.580107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693587d-7dc8-4543-a9d9-859a8cf78a9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:23.633504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 215/427] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:23.634799 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cf76c-1851-4b13-9e61-aafabd1b5c98 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:23.642750 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:23.658419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693587d-7dc8-4543-a9d9-859a8cf78a9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:23.658419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693587d-7dc8-4543-a9d9-859a8cf78a9e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:23.715442 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:23.758063 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cf76c-1851-4b13-9e61-aafabd1b5c98 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:23.761311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 213/428] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:23.766973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693587d-7dc8-4543-a9d9-859a8cf78a9e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:23.796222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914929bd-43aa-4232-b658-1e4d54f471c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:23.797376 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914929bd-43aa-4232-b658-1e4d54f471c4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:23.797812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914929bd-43aa-4232-b658-1e4d54f471c4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:23.798236 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914929bd-43aa-4232-b658-1e4d54f471c4 None admin] PATH_INFO: `/v3/groups/cf4b8c88f95c4431ad46a14140cf1842` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:23.799900 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914929bd-43aa-4232-b658-1e4d54f471c4 None admin] RBAC: Authorizing `identity:get_group(group_id=cf4b8c88f95c4431ad46a14140cf1842)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:23.801525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914929bd-43aa-4232-b658-1e4d54f471c4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:23.808678 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 216/429] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:04:23 2022] GET /identity/v3/groups/cf4b8c88f95c4431ad46a14140cf1842 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:23.809954 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914929bd-43aa-4232-b658-1e4d54f471c4 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:23.863938 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:23.874944 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693587d-7dc8-4543-a9d9-859a8cf78a9e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:23.912471 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57df829-9087-4f75-bdd0-5a11db3bcd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:23.913345 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57df829-9087-4f75-bdd0-5a11db3bcd70 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:23.913722 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57df829-9087-4f75-bdd0-5a11db3bcd70 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:23.914095 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57df829-9087-4f75-bdd0-5a11db3bcd70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:23.927191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57df829-9087-4f75-bdd0-5a11db3bcd70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:23.928162 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914929bd-43aa-4232-b658-1e4d54f471c4 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-e57df829-9087-4f75-bdd0-5a11db3bcd70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:23.936005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:23.937329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:23.938503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:23.938503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:23.938503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:23.938503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:23.938503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:23.938503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:23.938503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:23.938503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:23.938503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:23.938503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:23.938503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:04:23.938503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:23.946077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 214/430] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:23.946077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57df829-9087-4f75-bdd0-5a11db3bcd70 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:23.972299 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:23.980127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec475288-c92c-4a6e-93a6-81c08072065e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:23.981535 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec475288-c92c-4a6e-93a6-81c08072065e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:23.981818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec475288-c92c-4a6e-93a6-81c08072065e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:23.982090 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec475288-c92c-4a6e-93a6-81c08072065e None admin] PATH_INFO: `/v3/projects/e490252619914b5d80405cb54917237f` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:23.983622 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec475288-c92c-4a6e-93a6-81c08072065e None admin] RBAC: Authorizing `identity:get_project(project_id=e490252619914b5d80405cb54917237f)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:23.985244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec475288-c92c-4a6e-93a6-81c08072065e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:23.990342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 217/431] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:23 2022] GET /identity/v3/projects/e490252619914b5d80405cb54917237f => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:23.991446 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec475288-c92c-4a6e-93a6-81c08072065e None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:24.066024 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57df829-9087-4f75-bdd0-5a11db3bcd70 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:24.103070 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d72f2-e29b-4fa3-905a-98793b35432d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:24.103748 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d72f2-e29b-4fa3-905a-98793b35432d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:24.104025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d72f2-e29b-4fa3-905a-98793b35432d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:24.104349 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d72f2-e29b-4fa3-905a-98793b35432d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:24.105208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d72f2-e29b-4fa3-905a-98793b35432d None admin] RBAC: Adding query filter params (group.id=cf4b8c88f95c4431ad46a14140cf1842, role.id=6aee6f5679c84fc1be21561ddb7d1b56, scope.project.id=e490252619914b5d80405cb54917237f) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:24.105522 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d72f2-e29b-4fa3-905a-98793b35432d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:24.107061 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d72f2-e29b-4fa3-905a-98793b35432d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:24.107915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec475288-c92c-4a6e-93a6-81c08072065e None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:24.118350 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 215/432] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:04:23 2022] GET /identity/v3/role_assignments?group.id=cf4b8c88f95c4431ad46a14140cf1842&scope.project.id=e490252619914b5d80405cb54917237f&role.id=6aee6f5679c84fc1be21561ddb7d1b56 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:24.118350 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d72f2-e29b-4fa3-905a-98793b35432d None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:24.145692 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:24.160744 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7719cc-b01e-448e-b126-aa85e301717e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:24.161925 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7719cc-b01e-448e-b126-aa85e301717e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:24.162637 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7719cc-b01e-448e-b126-aa85e301717e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:24.163151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7719cc-b01e-448e-b126-aa85e301717e None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:24.164155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7719cc-b01e-448e-b126-aa85e301717e None admin] RBAC: Adding query filter params (name=member) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:24.164710 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7719cc-b01e-448e-b126-aa85e301717e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:24.166928 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7719cc-b01e-448e-b126-aa85e301717e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:24.182679 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 218/433] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:04:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:24.184958 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7719cc-b01e-448e-b126-aa85e301717e None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:24.225186 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d72f2-e29b-4fa3-905a-98793b35432d None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:24.254830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:24.261616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb9c6e-662d-4048-8ea7-3b84fffe44bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:24.262675 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb9c6e-662d-4048-8ea7-3b84fffe44bf None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:24.262953 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb9c6e-662d-4048-8ea7-3b84fffe44bf None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:24.263227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb9c6e-662d-4048-8ea7-3b84fffe44bf None admin] PATH_INFO: `/v3/groups/cf4b8c88f95c4431ad46a14140cf1842` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:24.265449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb9c6e-662d-4048-8ea7-3b84fffe44bf None admin] RBAC: Authorizing `identity:get_group(group_id=cf4b8c88f95c4431ad46a14140cf1842)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:24.266952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb9c6e-662d-4048-8ea7-3b84fffe44bf None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:24.269023 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 216/434] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:04:24 2022] GET /identity/v3/groups/cf4b8c88f95c4431ad46a14140cf1842 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:24.277023 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb9c6e-662d-4048-8ea7-3b84fffe44bf None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:24.303683 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7719cc-b01e-448e-b126-aa85e301717e None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:24.343916 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed88b8f-31fe-483b-bbfa-a8ae8e9bac09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:24.345768 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed88b8f-31fe-483b-bbfa-a8ae8e9bac09 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:24.346291 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed88b8f-31fe-483b-bbfa-a8ae8e9bac09 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:24.346979 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed88b8f-31fe-483b-bbfa-a8ae8e9bac09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:24.412448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:24.430905 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed88b8f-31fe-483b-bbfa-a8ae8e9bac09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:24.432507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed88b8f-31fe-483b-bbfa-a8ae8e9bac09 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:24.445791 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb9c6e-662d-4048-8ea7-3b84fffe44bf None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:24.493802 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30f905-9f3b-401e-8040-2be01fa3ff20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:24.493802 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30f905-9f3b-401e-8040-2be01fa3ff20 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:24.493802 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30f905-9f3b-401e-8040-2be01fa3ff20 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:24.493802 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30f905-9f3b-401e-8040-2be01fa3ff20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:24.494388 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30f905-9f3b-401e-8040-2be01fa3ff20 None admin] RBAC: Adding query filter params (group.id=cf4b8c88f95c4431ad46a14140cf1842, role.id=b21c1004651d40f1b44fb0b7becbe3c8, scope.project.id=e490252619914b5d80405cb54917237f) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:24.494388 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30f905-9f3b-401e-8040-2be01fa3ff20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:24.496498 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30f905-9f3b-401e-8040-2be01fa3ff20 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:24.505102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 217/435] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:04:24 2022] GET /identity/v3/role_assignments?group.id=cf4b8c88f95c4431ad46a14140cf1842&scope.project.id=e490252619914b5d80405cb54917237f&role.id=b21c1004651d40f1b44fb0b7becbe3c8 => generated 611 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:24.507140 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec79822-dad2-46a2-8d94-f6ace40bf785 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:24.507537 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec79822-dad2-46a2-8d94-f6ace40bf785 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:24.508298 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec79822-dad2-46a2-8d94-f6ace40bf785 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:24.509939 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 218/436] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:24.516675 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00060b-8c06-452e-a8bd-ed2e0bc74542 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:24.517255 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00060b-8c06-452e-a8bd-ed2e0bc74542 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:24.517536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00060b-8c06-452e-a8bd-ed2e0bc74542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:24.538690 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:24.544498 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-5e00060b-8c06-452e-a8bd-ed2e0bc74542 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:24.546387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-5e00060b-8c06-452e-a8bd-ed2e0bc74542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:24.584648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 219/437] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:04:24.586424 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed88b8f-31fe-483b-bbfa-a8ae8e9bac09 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:24.651365 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 219/438] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:04:24.691489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00060b-8c06-452e-a8bd-ed2e0bc74542 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:24.739245 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed88b8f-31fe-483b-bbfa-a8ae8e9bac09 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:24.777435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc72801-981b-4c0f-a787-6880b0b1cced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:24.778387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc72801-981b-4c0f-a787-6880b0b1cced None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:24.778670 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc72801-981b-4c0f-a787-6880b0b1cced None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:24.778938 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc72801-981b-4c0f-a787-6880b0b1cced None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:24.780778 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc72801-981b-4c0f-a787-6880b0b1cced None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:24.782373 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc72801-981b-4c0f-a787-6880b0b1cced None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:24.784131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 220/439] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:24 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:24.789897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc72801-981b-4c0f-a787-6880b0b1cced None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:24.791156 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:24.821104 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:24.828022 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00060b-8c06-452e-a8bd-ed2e0bc74542 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:24.870529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e408d92-4e6f-4075-b04d-dde5aa148740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:24.871550 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e408d92-4e6f-4075-b04d-dde5aa148740 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:24.871792 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e408d92-4e6f-4075-b04d-dde5aa148740 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:24.872066 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e408d92-4e6f-4075-b04d-dde5aa148740 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:24.884458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e408d92-4e6f-4075-b04d-dde5aa148740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-6e408d92-4e6f-4075-b04d-dde5aa148740 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:24.893574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:24.894625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:24.896455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:24.896455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:24.896455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:24.896455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:24.896455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:24.896455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:24.896455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:24.896455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:24.896455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:24.896455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:24.896455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:24.896455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:24.896455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 220/440] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:04:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:24.901885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e408d92-4e6f-4075-b04d-dde5aa148740 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:24.920691 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc72801-981b-4c0f-a787-6880b0b1cced None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:24.973354 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578a2e2-2ec3-4099-9724-f980e4c647d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:24.974227 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578a2e2-2ec3-4099-9724-f980e4c647d6 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:24.974544 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578a2e2-2ec3-4099-9724-f980e4c647d6 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:24.974826 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578a2e2-2ec3-4099-9724-f980e4c647d6 None admin] PATH_INFO: `/v3/groups/cf4b8c88f95c4431ad46a14140cf1842` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:24.976259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578a2e2-2ec3-4099-9724-f980e4c647d6 None admin] RBAC: Authorizing `identity:get_group(group_id=cf4b8c88f95c4431ad46a14140cf1842)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:24.977834 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578a2e2-2ec3-4099-9724-f980e4c647d6 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:24.981359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 221/441] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:04:24 2022] GET /identity/v3/groups/cf4b8c88f95c4431ad46a14140cf1842 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:24.988175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578a2e2-2ec3-4099-9724-f980e4c647d6 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:25.047980 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e408d92-4e6f-4075-b04d-dde5aa148740 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:25.086283 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7ff5b-7687-491d-abd7-e7890b35cde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:25.087092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7ff5b-7687-491d-abd7-e7890b35cde7 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:25.087465 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7ff5b-7687-491d-abd7-e7890b35cde7 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:25.087749 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7ff5b-7687-491d-abd7-e7890b35cde7 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:25.088716 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7ff5b-7687-491d-abd7-e7890b35cde7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:25.089140 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7ff5b-7687-491d-abd7-e7890b35cde7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:25.090854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7ff5b-7687-491d-abd7-e7890b35cde7 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:25.095527 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:25.107783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:25.108727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 221/442] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:25.111090 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7ff5b-7687-491d-abd7-e7890b35cde7 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:25.115823 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578a2e2-2ec3-4099-9724-f980e4c647d6 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:25.155156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f993d24-fffc-4f29-a0de-8a5dd9b2fb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:25.155812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f993d24-fffc-4f29-a0de-8a5dd9b2fb73 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:25.156088 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f993d24-fffc-4f29-a0de-8a5dd9b2fb73 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:25.156369 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f993d24-fffc-4f29-a0de-8a5dd9b2fb73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:25.157186 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f993d24-fffc-4f29-a0de-8a5dd9b2fb73 None admin] RBAC: Adding query filter params (group.id=cf4b8c88f95c4431ad46a14140cf1842, role.id=6aee6f5679c84fc1be21561ddb7d1b56, scope.project.id=bb83c47ec18443db80a5cf2ed51d142e) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:25.157493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f993d24-fffc-4f29-a0de-8a5dd9b2fb73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:25.158946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f993d24-fffc-4f29-a0de-8a5dd9b2fb73 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:25.164760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 222/443] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:04:24 2022] GET /identity/v3/role_assignments?group.id=cf4b8c88f95c4431ad46a14140cf1842&scope.project.id=bb83c47ec18443db80a5cf2ed51d142e&role.id=6aee6f5679c84fc1be21561ddb7d1b56 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:25.166449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28a2b3-51b4-464a-a6c5-d4bc72e09aee None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:25.166756 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28a2b3-51b4-464a-a6c5-d4bc72e09aee None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:25.167086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28a2b3-51b4-464a-a6c5-d4bc72e09aee None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:25.168315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 223/444] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:25.174196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf5143-9fe2-422e-8ff6-a8c1adeda6e9 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:25.174530 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf5143-9fe2-422e-8ff6-a8c1adeda6e9 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:25.174803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf5143-9fe2-422e-8ff6-a8c1adeda6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:25.193709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-8dbf5143-9fe2-422e-8ff6-a8c1adeda6e9 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:25.195000 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-8dbf5143-9fe2-422e-8ff6-a8c1adeda6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:25.240443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7ff5b-7687-491d-abd7-e7890b35cde7 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:25.274618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 224/445] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:25.284398 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ca8b9-0b3a-4a5f-bf90-82a199ddb0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:25.285457 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ca8b9-0b3a-4a5f-bf90-82a199ddb0fd None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:25.285946 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ca8b9-0b3a-4a5f-bf90-82a199ddb0fd None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:25.286464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ca8b9-0b3a-4a5f-bf90-82a199ddb0fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:25.298180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf5143-9fe2-422e-8ff6-a8c1adeda6e9 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:25.362905 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:25.367166 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ca8b9-0b3a-4a5f-bf90-82a199ddb0fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:25.369233 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ca8b9-0b3a-4a5f-bf90-82a199ddb0fd None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:25.406843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:25.420454 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf5143-9fe2-422e-8ff6-a8c1adeda6e9 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:25.460052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fcab4c-5218-478d-9997-825a0267f25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:25.460864 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fcab4c-5218-478d-9997-825a0267f25c None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:25.461143 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fcab4c-5218-478d-9997-825a0267f25c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:25.461415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fcab4c-5218-478d-9997-825a0267f25c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:25.475831 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fcab4c-5218-478d-9997-825a0267f25c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-54fcab4c-5218-478d-9997-825a0267f25c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:25.484618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:25.485639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:25.486648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:25.486648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:25.486648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:25.486648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:25.486648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:25.486648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:25.486648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:25.486648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:25.486648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:25.486648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:25.486648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:04:25.486648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:25.487378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 222/446] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:25.489266 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 225/447] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:04:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:25.491364 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ca8b9-0b3a-4a5f-bf90-82a199ddb0fd None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:25.494904 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fcab4c-5218-478d-9997-825a0267f25c None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:25.610189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:25.613099 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ca8b9-0b3a-4a5f-bf90-82a199ddb0fd None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:25.623707 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fcab4c-5218-478d-9997-825a0267f25c None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:25.648134 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c587e-e82f-4ca5-9991-a72d57f5bb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:25.649004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c587e-e82f-4ca5-9991-a72d57f5bb39 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:25.649264 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c587e-e82f-4ca5-9991-a72d57f5bb39 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:25.649533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c587e-e82f-4ca5-9991-a72d57f5bb39 None admin] PATH_INFO: `/v3/projects/bb83c47ec18443db80a5cf2ed51d142e` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:25.650755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c587e-e82f-4ca5-9991-a72d57f5bb39 None admin] RBAC: Authorizing `identity:get_project(project_id=bb83c47ec18443db80a5cf2ed51d142e)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:25.652417 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c587e-e82f-4ca5-9991-a72d57f5bb39 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:25.654441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 223/448] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:25 2022] GET /identity/v3/projects/bb83c47ec18443db80a5cf2ed51d142e => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:25.661022 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c587e-e82f-4ca5-9991-a72d57f5bb39 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:25.662466 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722626b-bd3e-443a-a0f3-f843b4587f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:25.663146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722626b-bd3e-443a-a0f3-f843b4587f30 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:25.663435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722626b-bd3e-443a-a0f3-f843b4587f30 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:25.663734 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722626b-bd3e-443a-a0f3-f843b4587f30 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:25.664479 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722626b-bd3e-443a-a0f3-f843b4587f30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:25.664791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722626b-bd3e-443a-a0f3-f843b4587f30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:25.666155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722626b-bd3e-443a-a0f3-f843b4587f30 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:25.675352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 226/449] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:25.680780 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722626b-bd3e-443a-a0f3-f843b4587f30 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:25.695462 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:25.762370 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c587e-e82f-4ca5-9991-a72d57f5bb39 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:25.781552 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722626b-bd3e-443a-a0f3-f843b4587f30 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:25.796294 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c43764-91ec-4f6e-8ce3-ecf72e004b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:25.797158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c43764-91ec-4f6e-8ce3-ecf72e004b45 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:25.797452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c43764-91ec-4f6e-8ce3-ecf72e004b45 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:25.797715 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c43764-91ec-4f6e-8ce3-ecf72e004b45 None admin] PATH_INFO: `/v3/groups/cf4b8c88f95c4431ad46a14140cf1842` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:25.798907 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c43764-91ec-4f6e-8ce3-ecf72e004b45 None admin] RBAC: Authorizing `identity:get_group(group_id=cf4b8c88f95c4431ad46a14140cf1842)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:25.800372 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c43764-91ec-4f6e-8ce3-ecf72e004b45 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:25.802004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 224/450] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:04:25 2022] GET /identity/v3/groups/cf4b8c88f95c4431ad46a14140cf1842 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:25.807967 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c43764-91ec-4f6e-8ce3-ecf72e004b45 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:25.815493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d0c24-a259-4973-833c-41569b07c22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:25.816150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d0c24-a259-4973-833c-41569b07c22b None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:25.816435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d0c24-a259-4973-833c-41569b07c22b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:25.816713 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d0c24-a259-4973-833c-41569b07c22b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:25.841495 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:25.890446 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d0c24-a259-4973-833c-41569b07c22b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:25.891908 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d0c24-a259-4973-833c-41569b07c22b None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:25.906006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:25.910706 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c43764-91ec-4f6e-8ce3-ecf72e004b45 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:25.944251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937186c7-348e-4965-9636-32c677a6edda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:25.944903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937186c7-348e-4965-9636-32c677a6edda None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:25.945200 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937186c7-348e-4965-9636-32c677a6edda None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:25.945476 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937186c7-348e-4965-9636-32c677a6edda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:25.946339 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937186c7-348e-4965-9636-32c677a6edda None admin] RBAC: Adding query filter params (group.id=cf4b8c88f95c4431ad46a14140cf1842, role.id=b21c1004651d40f1b44fb0b7becbe3c8, scope.project.id=bb83c47ec18443db80a5cf2ed51d142e) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:25.946651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937186c7-348e-4965-9636-32c677a6edda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:25.948276 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937186c7-348e-4965-9636-32c677a6edda None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:25.954417 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 225/451] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:04:25 2022] GET /identity/v3/role_assignments?group.id=cf4b8c88f95c4431ad46a14140cf1842&scope.project.id=bb83c47ec18443db80a5cf2ed51d142e&role.id=b21c1004651d40f1b44fb0b7becbe3c8 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:25.996087 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 227/452] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:04:26.002060 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937186c7-348e-4965-9636-32c677a6edda None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:26.111712 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:26.115638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937186c7-348e-4965-9636-32c677a6edda None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:26.151471 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efdf79-ef35-4f83-a851-c90203f7ed34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:26.153763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efdf79-ef35-4f83-a851-c90203f7ed34 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:26.153763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efdf79-ef35-4f83-a851-c90203f7ed34 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:26.153763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efdf79-ef35-4f83-a851-c90203f7ed34 None admin] PATH_INFO: `/v3/projects/026346139f294a8c8380faacc93b3be0` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:26.153763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45efdf79-ef35-4f83-a851-c90203f7ed34 None admin] RBAC: Authorizing `identity:get_project(project_id=026346139f294a8c8380faacc93b3be0)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:26.155290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45efdf79-ef35-4f83-a851-c90203f7ed34 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:26.156982 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 226/453] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:04:26 2022] GET /identity/v3/projects/026346139f294a8c8380faacc93b3be0 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:26.162730 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d0c24-a259-4973-833c-41569b07c22b None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:26.266247 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d0c24-a259-4973-833c-41569b07c22b None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:26.279506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:26.301398 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d31d17-470e-4b53-9fbd-6f8245487aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:26.302285 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d31d17-470e-4b53-9fbd-6f8245487aa9 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:26.302707 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d31d17-470e-4b53-9fbd-6f8245487aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:26.302980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d31d17-470e-4b53-9fbd-6f8245487aa9 None admin] PATH_INFO: `/v3/groups/ebf2b429b7f5404faa458962f9755a03` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:26.304139 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d31d17-470e-4b53-9fbd-6f8245487aa9 None admin] RBAC: Authorizing `identity:get_group(group_id=ebf2b429b7f5404faa458962f9755a03)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:26.305685 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d31d17-470e-4b53-9fbd-6f8245487aa9 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:26.307371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 228/454] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:04:26 2022] GET /identity/v3/groups/ebf2b429b7f5404faa458962f9755a03 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:26.313268 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efdf79-ef35-4f83-a851-c90203f7ed34 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:26.420093 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efdf79-ef35-4f83-a851-c90203f7ed34 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:26.471041 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2d4a7-e1de-46a5-a680-b4bb7a094f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:26.471704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2d4a7-e1de-46a5-a680-b4bb7a094f7a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:26.471988 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2d4a7-e1de-46a5-a680-b4bb7a094f7a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:26.472299 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2d4a7-e1de-46a5-a680-b4bb7a094f7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:26.473144 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2d4a7-e1de-46a5-a680-b4bb7a094f7a None admin] RBAC: Adding query filter params (group.id=ebf2b429b7f5404faa458962f9755a03, role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=026346139f294a8c8380faacc93b3be0) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:26.473460 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2d4a7-e1de-46a5-a680-b4bb7a094f7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:26.474980 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2d4a7-e1de-46a5-a680-b4bb7a094f7a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:26.481942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 227/455] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:04:26 2022] GET /identity/v3/role_assignments?group.id=ebf2b429b7f5404faa458962f9755a03&scope.project.id=026346139f294a8c8380faacc93b3be0&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:29.343852 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3068c-d74f-4ac8-ae5c-34fc1c14f4f7 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:29.344733 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3068c-d74f-4ac8-ae5c-34fc1c14f4f7 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:29.344733 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3068c-d74f-4ac8-ae5c-34fc1c14f4f7 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:29.346462 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 229/456] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:29.356798 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d685eb-8aac-4546-be17-79237a135615 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:29.357394 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d685eb-8aac-4546-be17-79237a135615 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:29.357394 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d685eb-8aac-4546-be17-79237a135615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:29.379304 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-27d685eb-8aac-4546-be17-79237a135615 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:29.380502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-27d685eb-8aac-4546-be17-79237a135615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:29.409688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:29.470157 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 228/457] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:29.515822 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3068c-d74f-4ac8-ae5c-34fc1c14f4f7 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:29.652608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3068c-d74f-4ac8-ae5c-34fc1c14f4f7 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:29.696655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d637f6e-2c31-4dba-8d45-ab705c40ae7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:29.697464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d637f6e-2c31-4dba-8d45-ab705c40ae7e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:29.697743 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d637f6e-2c31-4dba-8d45-ab705c40ae7e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:29.698392 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d637f6e-2c31-4dba-8d45-ab705c40ae7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:29.712938 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d637f6e-2c31-4dba-8d45-ab705c40ae7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:29.714507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d637f6e-2c31-4dba-8d45-ab705c40ae7e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:29.740698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-6d637f6e-2c31-4dba-8d45-ab705c40ae7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:29.740698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:29.740698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:29.740698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:29.740698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:29.740698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:29.740698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:29.740698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:29.740698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:29.740698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:29.740698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:29.740698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:29.740698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:29.740698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:29.740698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:29.743031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:29.746091 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:29.746091 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:29.746091 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:29.746091 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:29.746091 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 230/458] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:29.752737 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d685eb-8aac-4546-be17-79237a135615 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:29.826943 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564fae5-18c5-4169-af80-c91ffd8c5e08 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:29.827270 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564fae5-18c5-4169-af80-c91ffd8c5e08 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:29.827485 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564fae5-18c5-4169-af80-c91ffd8c5e08 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:29.829168 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 231/459] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:04:29.835887 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af884027-016d-4bfd-9c05-366fc790a063 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:29.835887 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af884027-016d-4bfd-9c05-366fc790a063 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:29.835887 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af884027-016d-4bfd-9c05-366fc790a063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:29.863576 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-af884027-016d-4bfd-9c05-366fc790a063 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:29.864761 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-af884027-016d-4bfd-9c05-366fc790a063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:29.894930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:29.917364 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d685eb-8aac-4546-be17-79237a135615 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:29.948202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 232/460] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:04:29.950053 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9e99c-1b29-4783-bc32-760b041f514f None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:29.950376 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9e99c-1b29-4783-bc32-760b041f514f None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:29.950662 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9e99c-1b29-4783-bc32-760b041f514f None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:29.952591 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 233/461] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:29.958321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef8f91-fef7-4393-9a32-39339c6b9406 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:29.958321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef8f91-fef7-4393-9a32-39339c6b9406 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:29.958530 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef8f91-fef7-4393-9a32-39339c6b9406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:29.975568 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe1f1f-bfa9-4156-9ff3-64138a4c72ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:29.976253 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe1f1f-bfa9-4156-9ff3-64138a4c72ce None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:29.976530 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe1f1f-bfa9-4156-9ff3-64138a4c72ce None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:29.976803 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe1f1f-bfa9-4156-9ff3-64138a4c72ce None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:29.977205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-13ef8f91-fef7-4393-9a32-39339c6b9406 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:29.977532 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe1f1f-bfa9-4156-9ff3-64138a4c72ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:29.977839 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe1f1f-bfa9-4156-9ff3-64138a4c72ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:29.978420 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-13ef8f91-fef7-4393-9a32-39339c6b9406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:29.979806 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe1f1f-bfa9-4156-9ff3-64138a4c72ce None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:29.998609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 229/462] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:29.999471 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe1f1f-bfa9-4156-9ff3-64138a4c72ce None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:30.022484 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:30.065107 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 234/463] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:04:30.066059 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef8f91-fef7-4393-9a32-39339c6b9406 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:30.116048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe1f1f-bfa9-4156-9ff3-64138a4c72ce None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:30.153384 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33bd93b-e1de-447a-94a8-d9a457723161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:30.154542 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bd93b-e1de-447a-94a8-d9a457723161 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:30.154889 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bd93b-e1de-447a-94a8-d9a457723161 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:30.155074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bd93b-e1de-447a-94a8-d9a457723161 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:30.165259 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33bd93b-e1de-447a-94a8-d9a457723161 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:30.166859 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33bd93b-e1de-447a-94a8-d9a457723161 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:30.167086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:30.177591 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef8f91-fef7-4393-9a32-39339c6b9406 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:30.187162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-a33bd93b-e1de-447a-94a8-d9a457723161 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:30.187162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:30.187162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:30.187162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:30.187162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:30.187162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:30.187162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:30.187162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:30.187162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:30.187162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:30.187162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:30.187162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:30.187162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:30.187162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:30.187162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:30.188169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:30.189126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:30.189126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:30.189126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:30.189126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:30.189126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 230/464] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:30.190188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20228c7e-a09d-4856-a775-4ad8583f4cea None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:30.190188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20228c7e-a09d-4856-a775-4ad8583f4cea None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:30.190188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20228c7e-a09d-4856-a775-4ad8583f4cea None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:30.191545 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 231/465] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:30.196175 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20228c7e-a09d-4856-a775-4ad8583f4cea None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:30.225516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b2f90-48cf-40eb-9a71-11611ea2e7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:30.226383 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b2f90-48cf-40eb-9a71-11611ea2e7a7 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:30.226671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b2f90-48cf-40eb-9a71-11611ea2e7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:30.226952 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b2f90-48cf-40eb-9a71-11611ea2e7a7 None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:30.227842 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b2f90-48cf-40eb-9a71-11611ea2e7a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:30.230424 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b2f90-48cf-40eb-9a71-11611ea2e7a7 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:30.254790 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 235/466] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:04:30 2022] POST /identity/v3/users => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:04:30.255773 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b2f90-48cf-40eb-9a71-11611ea2e7a7 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:30.283536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:30.326482 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20228c7e-a09d-4856-a775-4ad8583f4cea None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:30.368145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69687308-7a59-4a01-9cdf-f82aa761ab9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:30.369050 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69687308-7a59-4a01-9cdf-f82aa761ab9d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:30.369379 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69687308-7a59-4a01-9cdf-f82aa761ab9d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:30.369656 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69687308-7a59-4a01-9cdf-f82aa761ab9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:30.382932 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69687308-7a59-4a01-9cdf-f82aa761ab9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:30.384604 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69687308-7a59-4a01-9cdf-f82aa761ab9d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:30.400970 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b2f90-48cf-40eb-9a71-11611ea2e7a7 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:30.404248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-69687308-7a59-4a01-9cdf-f82aa761ab9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:30.404248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:30.404248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:30.404248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:30.404248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:30.404248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:30.404248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:30.404248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:30.404248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:30.404248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:30.404248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:30.404248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:30.404248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:30.404248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:30.404248 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:30.405221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:30.406193 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:30.406193 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:30.406193 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:30.406193 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:30.406193 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 232/467] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:30.408378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9f4ed-0dec-48dc-bfe5-7315437972c9 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:30.408378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9f4ed-0dec-48dc-bfe5-7315437972c9 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:30.408666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9f4ed-0dec-48dc-bfe5-7315437972c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:30.429946 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-e8b9f4ed-0dec-48dc-bfe5-7315437972c9 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:30.432784 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-e8b9f4ed-0dec-48dc-bfe5-7315437972c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:30.454254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10ab32-c55e-4eaf-9d3f-c50cec0c50ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:30.454985 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10ab32-c55e-4eaf-9d3f-c50cec0c50ca None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:30.455265 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10ab32-c55e-4eaf-9d3f-c50cec0c50ca None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:30.455623 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10ab32-c55e-4eaf-9d3f-c50cec0c50ca None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:30.456335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10ab32-c55e-4eaf-9d3f-c50cec0c50ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:30.456961 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10ab32-c55e-4eaf-9d3f-c50cec0c50ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:30.458122 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10ab32-c55e-4eaf-9d3f-c50cec0c50ca None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:30.476725 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 236/468] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:30.478002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10ab32-c55e-4eaf-9d3f-c50cec0c50ca None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:30.491645 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:30.531849 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 233/469] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:30.532497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9f4ed-0dec-48dc-bfe5-7315437972c9 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:30.595959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:30.623051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10ab32-c55e-4eaf-9d3f-c50cec0c50ca None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:30.647264 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9f4ed-0dec-48dc-bfe5-7315437972c9 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:30.665445 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72838a6d-b69c-4650-89f8-56524a7e5113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:30.666452 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72838a6d-b69c-4650-89f8-56524a7e5113 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:30.666934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72838a6d-b69c-4650-89f8-56524a7e5113 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:30.667382 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72838a6d-b69c-4650-89f8-56524a7e5113 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:30.668266 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72838a6d-b69c-4650-89f8-56524a7e5113 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:30.668723 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72838a6d-b69c-4650-89f8-56524a7e5113 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:30.670288 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72838a6d-b69c-4650-89f8-56524a7e5113 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:30.683016 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 237/470] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:30.684275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72838a6d-b69c-4650-89f8-56524a7e5113 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:30.685197 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49f6c6-1cf0-40fa-a833-afeb7abfb948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:30.685875 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49f6c6-1cf0-40fa-a833-afeb7abfb948 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:30.686152 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49f6c6-1cf0-40fa-a833-afeb7abfb948 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:30.686464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49f6c6-1cf0-40fa-a833-afeb7abfb948 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:30.687172 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49f6c6-1cf0-40fa-a833-afeb7abfb948 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:30.688616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49f6c6-1cf0-40fa-a833-afeb7abfb948 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:30.715857 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 234/471] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:04:30 2022] POST /identity/v3/users => generated 248 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:04:30.717262 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86af4ca-7062-41ab-ac5f-146f67e1c923 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:30.717537 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86af4ca-7062-41ab-ac5f-146f67e1c923 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:30.717873 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86af4ca-7062-41ab-ac5f-146f67e1c923 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:30.719893 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 235/472] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:30.720800 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86af4ca-7062-41ab-ac5f-146f67e1c923 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:30.770457 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:30.812539 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72838a6d-b69c-4650-89f8-56524a7e5113 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:30.852565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca55a7c-fb5e-4527-8614-871ce4de4523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:30.853433 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca55a7c-fb5e-4527-8614-871ce4de4523 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:30.853722 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca55a7c-fb5e-4527-8614-871ce4de4523 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:30.854000 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca55a7c-fb5e-4527-8614-871ce4de4523 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:30.871571 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca55a7c-fb5e-4527-8614-871ce4de4523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:30.873571 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca55a7c-fb5e-4527-8614-871ce4de4523 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:30.879762 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86af4ca-7062-41ab-ac5f-146f67e1c923 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:30.894064 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:30.895330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-8ca55a7c-fb5e-4527-8614-871ce4de4523 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:30.895330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:30.895330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:30.895330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:30.895330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:30.895330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:30.895330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:30.895330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:30.895330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:30.895330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:30.895330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:30.895330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:30.895330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:30.895330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:30.895330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:30.896222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:30.897146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:30.897146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:30.897146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:30.897146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:30.897146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 238/473] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:30.897940 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9447c-89a3-4c22-a2c5-0da83a07f558 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:30.898235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9447c-89a3-4c22-a2c5-0da83a07f558 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:30.903851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9447c-89a3-4c22-a2c5-0da83a07f558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:30.922536 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-95c9447c-89a3-4c22-a2c5-0da83a07f558 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:30.923716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-95c9447c-89a3-4c22-a2c5-0da83a07f558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:30.933570 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67320356-e084-43fc-b0c2-0093e7f547a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:30.934593 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67320356-e084-43fc-b0c2-0093e7f547a4 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:30.935147 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67320356-e084-43fc-b0c2-0093e7f547a4 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:30.935989 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67320356-e084-43fc-b0c2-0093e7f547a4 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:30.936705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67320356-e084-43fc-b0c2-0093e7f547a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:30.939986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67320356-e084-43fc-b0c2-0093e7f547a4 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:30.965956 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 236/474] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:04:30 2022] POST /identity/v3/users => generated 249 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:04:30.966892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67320356-e084-43fc-b0c2-0093e7f547a4 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:31.006117 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 239/475] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:31.033590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9447c-89a3-4c22-a2c5-0da83a07f558 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:31.098649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:31.104445 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67320356-e084-43fc-b0c2-0093e7f547a4 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:31.178821 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9447c-89a3-4c22-a2c5-0da83a07f558 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:31.189827 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d1da5-9abd-456e-bdf6-0758ff4a5d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:31.190470 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d1da5-9abd-456e-bdf6-0758ff4a5d18 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:31.190757 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d1da5-9abd-456e-bdf6-0758ff4a5d18 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:31.191034 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d1da5-9abd-456e-bdf6-0758ff4a5d18 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:31.191777 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d1da5-9abd-456e-bdf6-0758ff4a5d18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:31.192091 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d1da5-9abd-456e-bdf6-0758ff4a5d18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:31.193548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d1da5-9abd-456e-bdf6-0758ff4a5d18 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:31.210687 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 237/476] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:31.216661 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d1da5-9abd-456e-bdf6-0758ff4a5d18 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:31.247332 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:31.255046 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df4ac0-257c-45c2-bc23-4e6a7a7e7135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:31.256080 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df4ac0-257c-45c2-bc23-4e6a7a7e7135 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:31.256361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df4ac0-257c-45c2-bc23-4e6a7a7e7135 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:31.256671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df4ac0-257c-45c2-bc23-4e6a7a7e7135 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:31.257446 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df4ac0-257c-45c2-bc23-4e6a7a7e7135 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:31.259064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df4ac0-257c-45c2-bc23-4e6a7a7e7135 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:31.269239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.sql.core [None req-64df4ac0-257c-45c2-bc23-4e6a7a7e7135 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:04:31.269239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 20:04:31.269239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [parameters: {'id': '3efb2f25045342e595c78801d0068b27', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 20:04:31.269239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=214315) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:04:31.288158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-64df4ac0-257c-45c2-bc23-4e6a7a7e7135 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 20:04:31.288158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:31.288158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:04:31.288158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:04:31.288158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:04:31.288158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:04:31.288158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:04:31.288158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:04:31.288158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:04:31.288158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application conn.query(q) May 19 20:04:31.288158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:04:31.288158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:04:31.288158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:04:31.288158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application result.read() May 19 20:04:31.288158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return ref.to_dict() May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:04:31.289254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application next(self.gen) May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application yield resource May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application next(self.gen) May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise self.value May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application session.commit() May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:04:31.290342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self.session.flush() May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application compat.raise_( May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:04:31.291298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:04:31.292347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application conn.query(q) May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application result.read() May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:04:31.293254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [parameters: {'id': '3efb2f25045342e595c78801d0068b27', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:31.294174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 20:04:31.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:31.301639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 240/477] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 20:04:31 2022] POST /identity/v3/roles => generated 144 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:31.301639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df4ac0-257c-45c2-bc23-4e6a7a7e7135 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:31.349151 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d1da5-9abd-456e-bdf6-0758ff4a5d18 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:31.399660 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61cd24-83c1-4154-b76b-671b97b8d2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:31.400420 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61cd24-83c1-4154-b76b-671b97b8d2c6 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:31.400626 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61cd24-83c1-4154-b76b-671b97b8d2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:31.401885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61cd24-83c1-4154-b76b-671b97b8d2c6 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:31.402790 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61cd24-83c1-4154-b76b-671b97b8d2c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:31.404327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61cd24-83c1-4154-b76b-671b97b8d2c6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:31.434055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 238/478] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:04:31 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:04:31.443309 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df4ac0-257c-45c2-bc23-4e6a7a7e7135 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:31.460736 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:31.486735 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725102c-1145-4f2f-b5f9-ba8bc2fbed12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:31.487587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725102c-1145-4f2f-b5f9-ba8bc2fbed12 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:31.487870 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725102c-1145-4f2f-b5f9-ba8bc2fbed12 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:31.488141 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725102c-1145-4f2f-b5f9-ba8bc2fbed12 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:31.500203 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725102c-1145-4f2f-b5f9-ba8bc2fbed12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-3725102c-1145-4f2f-b5f9-ba8bc2fbed12 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:31.509070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:31.511566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:31.513588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:31.513588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:31.513588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:31.513588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:31.513588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:31.513588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:31.513588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:31.513588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:31.513588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:31.513588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:31.513588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:04:31.513588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:31.513588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 241/479] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:04:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:31.518296 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61cd24-83c1-4154-b76b-671b97b8d2c6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:31.614252 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:31.634633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61cd24-83c1-4154-b76b-671b97b8d2c6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:31.672611 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946a658-0fe4-4be5-a814-f6371a7adecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:31.673301 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a658-0fe4-4be5-a814-f6371a7adecc None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:31.673576 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a658-0fe4-4be5-a814-f6371a7adecc None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:31.673898 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a658-0fe4-4be5-a814-f6371a7adecc None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:31.674714 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946a658-0fe4-4be5-a814-f6371a7adecc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:31.675022 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946a658-0fe4-4be5-a814-f6371a7adecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:31.676426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946a658-0fe4-4be5-a814-f6371a7adecc None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:31.685895 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 239/480] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:32.964952 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91d082-4a89-4de8-a136-d111d50c87a0 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:32.965665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91d082-4a89-4de8-a136-d111d50c87a0 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:32.965665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91d082-4a89-4de8-a136-d111d50c87a0 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:32.967064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 242/481] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:32.972902 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4681144-7d32-4fc4-87fa-cadc6376b6d4 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:32.973708 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4681144-7d32-4fc4-87fa-cadc6376b6d4 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:32.974229 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4681144-7d32-4fc4-87fa-cadc6376b6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:33.003325 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-c4681144-7d32-4fc4-87fa-cadc6376b6d4 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:33.004919 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-c4681144-7d32-4fc4-87fa-cadc6376b6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:33.095330 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 240/482] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:33.121524 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91d082-4a89-4de8-a136-d111d50c87a0 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:33.291179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91d082-4a89-4de8-a136-d111d50c87a0 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:33.338668 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f379d6-29c2-42cb-b6a3-5241554dcb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:33.339628 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f379d6-29c2-42cb-b6a3-5241554dcb54 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:33.339913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f379d6-29c2-42cb-b6a3-5241554dcb54 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:33.340190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f379d6-29c2-42cb-b6a3-5241554dcb54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:33.353594 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f379d6-29c2-42cb-b6a3-5241554dcb54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-d4f379d6-29c2-42cb-b6a3-5241554dcb54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:33.361587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:33.362615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:33.363507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:33.363507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:33.363507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:33.363507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:33.363507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:33.363507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:33.363507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:33.363507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:33.363507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:33.363507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:33.363507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:33.363507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:33.363507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 243/483] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:33.368616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4681144-7d32-4fc4-87fa-cadc6376b6d4 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:33.492708 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:33.518938 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4681144-7d32-4fc4-87fa-cadc6376b6d4 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:33.554722 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce4f25-6821-4bd5-8a35-6522187a49b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:33.555416 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce4f25-6821-4bd5-8a35-6522187a49b8 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:33.555701 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce4f25-6821-4bd5-8a35-6522187a49b8 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:33.555976 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce4f25-6821-4bd5-8a35-6522187a49b8 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:33.556724 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce4f25-6821-4bd5-8a35-6522187a49b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:33.557032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce4f25-6821-4bd5-8a35-6522187a49b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:33.559176 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce4f25-6821-4bd5-8a35-6522187a49b8 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:33.568110 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 241/484] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:33.575366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f379d6-29c2-42cb-b6a3-5241554dcb54 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:33.581014 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a8c8b-18b4-42e7-8544-c9feaf03f437 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:33.581519 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a8c8b-18b4-42e7-8544-c9feaf03f437 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:33.581987 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a8c8b-18b4-42e7-8544-c9feaf03f437 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:33.583819 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 242/485] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:04:33.590023 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753d94c-77d2-4a67-9f4b-7ccd3817563e None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:33.590611 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753d94c-77d2-4a67-9f4b-7ccd3817563e None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:33.591079 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753d94c-77d2-4a67-9f4b-7ccd3817563e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:33.609903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-c753d94c-77d2-4a67-9f4b-7ccd3817563e None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:33.611685 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-c753d94c-77d2-4a67-9f4b-7ccd3817563e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:33.646189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:33.700302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 243/486] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:04:33.724794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753d94c-77d2-4a67-9f4b-7ccd3817563e None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:33.824218 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f379d6-29c2-42cb-b6a3-5241554dcb54 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:33.878685 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:33.899195 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693da1f-475a-465f-877c-d4373f9375da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:33.900037 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693da1f-475a-465f-877c-d4373f9375da None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:33.900309 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693da1f-475a-465f-877c-d4373f9375da None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:33.900583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693da1f-475a-465f-877c-d4373f9375da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:33.929970 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693da1f-475a-465f-877c-d4373f9375da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:33.931480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693da1f-475a-465f-877c-d4373f9375da None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:33.934033 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753d94c-77d2-4a67-9f4b-7ccd3817563e None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:33.967584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-2693da1f-475a-465f-877c-d4373f9375da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:33.967584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:33.967584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:33.967584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:33.967584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:33.967584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:33.967584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:33.967584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:33.967584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:33.967584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:33.967584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:33.967584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:33.967584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:33.967584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:33.967584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:33.968955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:33.970731 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:33.970731 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:33.970731 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:33.970731 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:33.970731 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 244/487] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:33.975606 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693da1f-475a-465f-877c-d4373f9375da None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:33.981440 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605aabc9-63fd-463d-a6fc-bd9ec27c679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:33.982302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605aabc9-63fd-463d-a6fc-bd9ec27c679b None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:33.982585 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605aabc9-63fd-463d-a6fc-bd9ec27c679b None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:33.982836 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605aabc9-63fd-463d-a6fc-bd9ec27c679b None admin] PATH_INFO: `/v3/roles/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:34.003982 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605aabc9-63fd-463d-a6fc-bd9ec27c679b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-605aabc9-63fd-463d-a6fc-bd9ec27c679b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:34.017467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:34.018540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:34.019443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:34.019443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:34.019443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:34.019443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:34.019443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:34.019443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:34.019443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:34.019443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:34.019443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:34.019443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:34.019443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:34.019443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:34.022112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 244/488] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:34.028217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605aabc9-63fd-463d-a6fc-bd9ec27c679b None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:34.098663 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:34.116244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693da1f-475a-465f-877c-d4373f9375da None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:34.153661 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605aabc9-63fd-463d-a6fc-bd9ec27c679b None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:34.154889 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5e488-67fd-4ff4-86c2-1311b63e591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:34.155812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5e488-67fd-4ff4-86c2-1311b63e591e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:34.156323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5e488-67fd-4ff4-86c2-1311b63e591e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:34.156835 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5e488-67fd-4ff4-86c2-1311b63e591e None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:34.157778 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5e488-67fd-4ff4-86c2-1311b63e591e None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:34.158325 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5e488-67fd-4ff4-86c2-1311b63e591e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:34.159915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5e488-67fd-4ff4-86c2-1311b63e591e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:34.175442 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 245/489] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:34.177234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42421e1-aea0-447a-8765-f3462b0e3a87 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:34.177762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42421e1-aea0-447a-8765-f3462b0e3a87 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:34.178247 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42421e1-aea0-447a-8765-f3462b0e3a87 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:34.180426 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 246/490] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:34.181475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42421e1-aea0-447a-8765-f3462b0e3a87 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:34.190661 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:34.195955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9f42c-583f-4cd3-97da-4181d8e852e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:34.196514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9f42c-583f-4cd3-97da-4181d8e852e8 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:34.198293 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9f42c-583f-4cd3-97da-4181d8e852e8 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:34.198293 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9f42c-583f-4cd3-97da-4181d8e852e8 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:34.198293 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9f42c-583f-4cd3-97da-4181d8e852e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:34.198293 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9f42c-583f-4cd3-97da-4181d8e852e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:34.199813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9f42c-583f-4cd3-97da-4181d8e852e8 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:34.214983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 245/491] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:34.214983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aad073-cd54-4b05-b64f-76e34a6e9c0d None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:34.214983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aad073-cd54-4b05-b64f-76e34a6e9c0d None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:34.214983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aad073-cd54-4b05-b64f-76e34a6e9c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:34.231221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-30aad073-cd54-4b05-b64f-76e34a6e9c0d None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:34.231924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-30aad073-cd54-4b05-b64f-76e34a6e9c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:34.307817 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 246/492] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:34.308714 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aad073-cd54-4b05-b64f-76e34a6e9c0d None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:34.310309 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42421e1-aea0-447a-8765-f3462b0e3a87 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:34.350682 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bc7da-f598-46bd-8163-a9b3e83fb464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:34.351839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bc7da-f598-46bd-8163-a9b3e83fb464 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:34.352348 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bc7da-f598-46bd-8163-a9b3e83fb464 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:34.352834 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bc7da-f598-46bd-8163-a9b3e83fb464 None admin] PATH_INFO: `/v3/users/nova` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:34.361888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0bc7da-f598-46bd-8163-a9b3e83fb464 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:34.363723 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0bc7da-f598-46bd-8163-a9b3e83fb464 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-2b0bc7da-f598-46bd-8163-a9b3e83fb464 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:34.381493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:34.382649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:34.383891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:34.383891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:34.383891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:34.383891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:34.383891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:34.383891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:34.383891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:34.383891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:34.383891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:04:34.383891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:34.387023 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 247/493] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:04:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:34.388134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bc7da-f598-46bd-8163-a9b3e83fb464 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:34.403724 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:34.436064 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aad073-cd54-4b05-b64f-76e34a6e9c0d None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:34.459948 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:34.482643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7595e57-364d-43a9-88ed-a617696ed813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:34.483454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7595e57-364d-43a9-88ed-a617696ed813 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:34.483730 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7595e57-364d-43a9-88ed-a617696ed813 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:34.484002 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7595e57-364d-43a9-88ed-a617696ed813 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:34.494543 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7595e57-364d-43a9-88ed-a617696ed813 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:34.496221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7595e57-364d-43a9-88ed-a617696ed813 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:34.514425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-d7595e57-364d-43a9-88ed-a617696ed813 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:34.514425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:34.514425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:34.514425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:34.514425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:34.514425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:34.514425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:34.514425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:34.514425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:34.514425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:34.514425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:34.514425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:34.514425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:34.514425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:34.514425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:34.515529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:34.516561 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:34.516561 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:34.516561 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:34.516561 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:34.516561 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 247/494] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:34.516561 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7595e57-364d-43a9-88ed-a617696ed813 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:34.526597 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bc7da-f598-46bd-8163-a9b3e83fb464 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:34.561411 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956139a8-2ad9-4cc7-8ebb-e808a3b49aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:34.562218 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956139a8-2ad9-4cc7-8ebb-e808a3b49aae None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:34.562530 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956139a8-2ad9-4cc7-8ebb-e808a3b49aae None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:34.562804 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956139a8-2ad9-4cc7-8ebb-e808a3b49aae None admin] PATH_INFO: `/v3/roles/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:34.576565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956139a8-2ad9-4cc7-8ebb-e808a3b49aae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-956139a8-2ad9-4cc7-8ebb-e808a3b49aae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:34.584363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:34.585365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:34.586328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:34.586328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:34.586328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:34.586328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:34.586328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:34.586328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:34.586328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:34.586328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:34.586328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:34.586328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:34.586328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:34.586328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:34.586328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 248/495] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:34.587080 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccb3b1-866b-4ad4-a4e4-36710829d8fb None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:34.587536 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccb3b1-866b-4ad4-a4e4-36710829d8fb None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:34.587635 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccb3b1-866b-4ad4-a4e4-36710829d8fb None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:34.589548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 249/496] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:34.594975 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529befad-f751-4065-8629-d427005d0039 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:34.595272 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529befad-f751-4065-8629-d427005d0039 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:34.595553 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529befad-f751-4065-8629-d427005d0039 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:34.596928 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 250/497] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:34.597495 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529befad-f751-4065-8629-d427005d0039 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:34.623638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7595e57-364d-43a9-88ed-a617696ed813 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:34.656990 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8327905-7c10-4aac-8d20-73696a9b68b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:34.657674 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8327905-7c10-4aac-8d20-73696a9b68b6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:34.657950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8327905-7c10-4aac-8d20-73696a9b68b6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:34.658231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8327905-7c10-4aac-8d20-73696a9b68b6 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:34.659074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8327905-7c10-4aac-8d20-73696a9b68b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:34.659380 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8327905-7c10-4aac-8d20-73696a9b68b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:34.660829 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8327905-7c10-4aac-8d20-73696a9b68b6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:34.664504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:34.702325 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 248/498] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:04:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:34.702325 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8327905-7c10-4aac-8d20-73696a9b68b6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:34.702638 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529befad-f751-4065-8629-d427005d0039 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:34.730793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:34.736289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba83ea5-568a-4659-8ccc-a08db629785b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:34.736969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba83ea5-568a-4659-8ccc-a08db629785b None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:34.737252 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba83ea5-568a-4659-8ccc-a08db629785b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:34.737531 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba83ea5-568a-4659-8ccc-a08db629785b None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:34.738282 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba83ea5-568a-4659-8ccc-a08db629785b None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:34.738611 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba83ea5-568a-4659-8ccc-a08db629785b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:34.740025 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba83ea5-568a-4659-8ccc-a08db629785b None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:34.752211 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 251/499] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:34.753447 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ba665-432d-4c3c-b908-a3f3b849df18 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:34.753738 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ba665-432d-4c3c-b908-a3f3b849df18 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:34.754016 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ba665-432d-4c3c-b908-a3f3b849df18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:34.770964 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-b37ba665-432d-4c3c-b908-a3f3b849df18 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:34.772171 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-b37ba665-432d-4c3c-b908-a3f3b849df18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:34.806640 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8327905-7c10-4aac-8d20-73696a9b68b6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:34.839878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 252/500] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:34.841150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced8348-fbea-49e7-bfa7-53b1e7c79f15 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:34.841436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced8348-fbea-49e7-bfa7-53b1e7c79f15 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:34.841708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced8348-fbea-49e7-bfa7-53b1e7c79f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:34.843339 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f634c-dd67-4ba2-beaa-53cf5c39d37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:34.844011 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f634c-dd67-4ba2-beaa-53cf5c39d37b None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:34.844605 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f634c-dd67-4ba2-beaa-53cf5c39d37b None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:34.844875 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f634c-dd67-4ba2-beaa-53cf5c39d37b None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:34.845621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f634c-dd67-4ba2-beaa-53cf5c39d37b None admin] RBAC: Adding query filter params (name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:34.845924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f634c-dd67-4ba2-beaa-53cf5c39d37b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:34.847435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f634c-dd67-4ba2-beaa-53cf5c39d37b None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:34.860288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 249/501] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:34.860288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f634c-dd67-4ba2-beaa-53cf5c39d37b None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:34.861631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-bced8348-fbea-49e7-bfa7-53b1e7c79f15 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:34.862847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-bced8348-fbea-49e7-bfa7-53b1e7c79f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:34.914112 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:34.932011 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 253/502] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:34.932881 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced8348-fbea-49e7-bfa7-53b1e7c79f15 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:34.955847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:34.966346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f634c-dd67-4ba2-beaa-53cf5c39d37b None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:35.000612 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a3750-9fec-46b9-8d65-12d0d283c788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:35.001242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a3750-9fec-46b9-8d65-12d0d283c788 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:35.001522 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a3750-9fec-46b9-8d65-12d0d283c788 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:35.001788 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a3750-9fec-46b9-8d65-12d0d283c788 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:35.053136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced8348-fbea-49e7-bfa7-53b1e7c79f15 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:35.069301 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a3750-9fec-46b9-8d65-12d0d283c788 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:35.070819 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a3750-9fec-46b9-8d65-12d0d283c788 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:35.087879 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25954f16-2275-4233-a84e-402a5ac369d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:35.089007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25954f16-2275-4233-a84e-402a5ac369d9 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:35.089144 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25954f16-2275-4233-a84e-402a5ac369d9 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:35.089415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25954f16-2275-4233-a84e-402a5ac369d9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:35.096922 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25954f16-2275-4233-a84e-402a5ac369d9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:35.098379 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25954f16-2275-4233-a84e-402a5ac369d9 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-25954f16-2275-4233-a84e-402a5ac369d9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:35.112601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:35.113762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:35.115058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:35.115058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:35.115058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:35.115058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:35.115058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:35.115058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:35.115058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:35.115058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:35.115058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:04:35.115058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:35.115058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 254/503] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:35.115058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25954f16-2275-4233-a84e-402a5ac369d9 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:35.165428 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:35.172743 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:35.176885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 250/504] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:35.177926 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a3750-9fec-46b9-8d65-12d0d283c788 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:35.228991 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25954f16-2275-4233-a84e-402a5ac369d9 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:35.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd3d03-366e-4698-8bd8-ebc81cd40438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:35.264789 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd3d03-366e-4698-8bd8-ebc81cd40438 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:35.265073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd3d03-366e-4698-8bd8-ebc81cd40438 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:35.265350 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd3d03-366e-4698-8bd8-ebc81cd40438 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:35.275407 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd3d03-366e-4698-8bd8-ebc81cd40438 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:35.276998 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd3d03-366e-4698-8bd8-ebc81cd40438 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:35.279262 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a3750-9fec-46b9-8d65-12d0d283c788 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:35.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-00cd3d03-366e-4698-8bd8-ebc81cd40438 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:35.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:35.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:35.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:35.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:35.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:35.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:35.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:35.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:35.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:35.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:35.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:35.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:35.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:35.294527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:35.295493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:35.296430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:35.296430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:35.296430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:35.296430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:35.296430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 255/505] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:35.296727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd3d03-366e-4698-8bd8-ebc81cd40438 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:35.318866 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b118aa-895a-45fb-b8ac-3a96085c74fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:35.319667 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b118aa-895a-45fb-b8ac-3a96085c74fc None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:35.319945 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b118aa-895a-45fb-b8ac-3a96085c74fc None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:35.320218 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b118aa-895a-45fb-b8ac-3a96085c74fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:35.330686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b118aa-895a-45fb-b8ac-3a96085c74fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-32b118aa-895a-45fb-b8ac-3a96085c74fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:35.338363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:35.339942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:35.341052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:35.341052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:35.341052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:35.341052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:35.341052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:35.341052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:35.341052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:35.341052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:35.341052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:35.341052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:35.341052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:35.341052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:35.341052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 251/506] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:35.341052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b118aa-895a-45fb-b8ac-3a96085c74fc None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:35.390447 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:35.402599 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:35.406890 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd3d03-366e-4698-8bd8-ebc81cd40438 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:35.440988 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b118aa-895a-45fb-b8ac-3a96085c74fc None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:35.443484 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6808545-9854-47f7-8351-468536b25387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:35.444283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6808545-9854-47f7-8351-468536b25387 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:35.444563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6808545-9854-47f7-8351-468536b25387 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:35.444854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6808545-9854-47f7-8351-468536b25387 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:35.455368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6808545-9854-47f7-8351-468536b25387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:35.456826 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6808545-9854-47f7-8351-468536b25387 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:35.475395 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e121d0-fc0c-4801-b986-31c919ac1a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:35.475589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-a6808545-9854-47f7-8351-468536b25387 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:35.475589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:35.475589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:35.475589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:35.475589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:35.475589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:35.475589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:35.475589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:35.475589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:35.475589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:35.475589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:35.475589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:35.475589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:35.475589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:35.475589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:35.476664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:35.477616 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:35.477616 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:35.477616 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:35.477616 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:35.477847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e121d0-fc0c-4801-b986-31c919ac1a16 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:35.477847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e121d0-fc0c-4801-b986-31c919ac1a16 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:35.477847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e121d0-fc0c-4801-b986-31c919ac1a16 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:35.478062 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 256/507] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:35.478156 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e121d0-fc0c-4801-b986-31c919ac1a16 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:35.478230 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6808545-9854-47f7-8351-468536b25387 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:35.478359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e121d0-fc0c-4801-b986-31c919ac1a16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:35.479870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e121d0-fc0c-4801-b986-31c919ac1a16 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:35.498016 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 252/508] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 20:04:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:35.498889 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e121d0-fc0c-4801-b986-31c919ac1a16 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:35.589503 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6808545-9854-47f7-8351-468536b25387 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:35.602923 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e121d0-fc0c-4801-b986-31c919ac1a16 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:35.612879 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:35.624060 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917e4a6-78e9-4b4d-bc16-4664509ac540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:35.625271 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917e4a6-78e9-4b4d-bc16-4664509ac540 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:35.625697 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917e4a6-78e9-4b4d-bc16-4664509ac540 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:35.625974 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917e4a6-78e9-4b4d-bc16-4664509ac540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:35.636083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917e4a6-78e9-4b4d-bc16-4664509ac540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:35.637540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917e4a6-78e9-4b4d-bc16-4664509ac540 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:35.638181 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585c9fb-4e8f-46a2-9703-72426b50717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:35.638882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585c9fb-4e8f-46a2-9703-72426b50717a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:35.639185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585c9fb-4e8f-46a2-9703-72426b50717a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:35.639461 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585c9fb-4e8f-46a2-9703-72426b50717a None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:35.640189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585c9fb-4e8f-46a2-9703-72426b50717a None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:35.640495 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585c9fb-4e8f-46a2-9703-72426b50717a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:35.641880 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585c9fb-4e8f-46a2-9703-72426b50717a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:35.653589 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 253/509] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:35.654446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585c9fb-4e8f-46a2-9703-72426b50717a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:35.667643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-1917e4a6-78e9-4b4d-bc16-4664509ac540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:35.667643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:35.667643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:35.667643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:35.667643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:35.667643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:35.667643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:35.667643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:35.667643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:35.667643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:35.667643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:35.667643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:35.667643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:35.667643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:35.667643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:35.668772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:35.670180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:35.670180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:35.670180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:35.670180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:35.670180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 257/510] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:35.671154 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917e4a6-78e9-4b4d-bc16-4664509ac540 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:35.674603 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:35.771258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585c9fb-4e8f-46a2-9703-72426b50717a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:35.778136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917e4a6-78e9-4b4d-bc16-4664509ac540 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:35.805138 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837ba95-a944-47ed-b021-d9540c9bd04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:35.805902 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837ba95-a944-47ed-b021-d9540c9bd04f None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:35.806181 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837ba95-a944-47ed-b021-d9540c9bd04f None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:35.806510 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837ba95-a944-47ed-b021-d9540c9bd04f None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:35.807450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837ba95-a944-47ed-b021-d9540c9bd04f None admin] RBAC: Adding query filter params (name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:35.807807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837ba95-a944-47ed-b021-d9540c9bd04f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:35.809200 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837ba95-a944-47ed-b021-d9540c9bd04f None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:35.812172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9cb8d-bd7e-498f-80aa-57ec106457ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:35.812842 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9cb8d-bd7e-498f-80aa-57ec106457ce None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:35.813128 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9cb8d-bd7e-498f-80aa-57ec106457ce None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:35.813409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9cb8d-bd7e-498f-80aa-57ec106457ce None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:35.814145 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9cb8d-bd7e-498f-80aa-57ec106457ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:35.814485 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9cb8d-bd7e-498f-80aa-57ec106457ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:35.815855 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9cb8d-bd7e-498f-80aa-57ec106457ce None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:35.817319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 254/511] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:35.818111 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837ba95-a944-47ed-b021-d9540c9bd04f None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:35.827776 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 258/512] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:35.828594 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9cb8d-bd7e-498f-80aa-57ec106457ce None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:35.855189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:35.893482 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:35.926393 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837ba95-a944-47ed-b021-d9540c9bd04f None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:35.933000 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9cb8d-bd7e-498f-80aa-57ec106457ce None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:35.959674 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34b474-01bc-44fa-aef2-f89cd794e4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:35.960328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34b474-01bc-44fa-aef2-f89cd794e4de None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:35.960611 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34b474-01bc-44fa-aef2-f89cd794e4de None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:35.960888 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34b474-01bc-44fa-aef2-f89cd794e4de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:35.970738 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec430090-3bc0-4668-83b4-8f982dc773ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:35.971875 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec430090-3bc0-4668-83b4-8f982dc773ee None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:35.972179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec430090-3bc0-4668-83b4-8f982dc773ee None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:35.972461 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec430090-3bc0-4668-83b4-8f982dc773ee None admin] PATH_INFO: `/v3/users/neutron` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:35.980623 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec430090-3bc0-4668-83b4-8f982dc773ee None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:35.982086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec430090-3bc0-4668-83b4-8f982dc773ee None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-ec430090-3bc0-4668-83b4-8f982dc773ee None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:35.996179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:35.997149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:35.998127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:35.998127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:35.998127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:35.998127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:35.998127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:35.998127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:35.998127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:35.998127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:35.998127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:04:35.998127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:35.998127 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 259/513] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:35.999225 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec430090-3bc0-4668-83b4-8f982dc773ee None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:36.029074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34b474-01bc-44fa-aef2-f89cd794e4de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:36.030516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34b474-01bc-44fa-aef2-f89cd794e4de None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:36.058962 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:36.097730 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec430090-3bc0-4668-83b4-8f982dc773ee None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:36.119887 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:36.130227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 255/514] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:36.131174 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34b474-01bc-44fa-aef2-f89cd794e4de None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:36.137579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e3321-117d-47da-aecc-309d957bf25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:36.138247 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e3321-117d-47da-aecc-309d957bf25e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:36.138554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e3321-117d-47da-aecc-309d957bf25e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:36.138821 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e3321-117d-47da-aecc-309d957bf25e None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:36.139548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e3321-117d-47da-aecc-309d957bf25e None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:36.139868 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e3321-117d-47da-aecc-309d957bf25e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:36.141279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e3321-117d-47da-aecc-309d957bf25e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:36.153742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 260/515] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:36.154526 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e3321-117d-47da-aecc-309d957bf25e None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:36.235190 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34b474-01bc-44fa-aef2-f89cd794e4de None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:36.254821 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e3321-117d-47da-aecc-309d957bf25e None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:36.262871 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:36.269256 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf210568-fd5d-4fd0-a830-c93e3022f6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:36.270064 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf210568-fd5d-4fd0-a830-c93e3022f6c7 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:36.270376 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf210568-fd5d-4fd0-a830-c93e3022f6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:36.270656 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf210568-fd5d-4fd0-a830-c93e3022f6c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:36.281629 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf210568-fd5d-4fd0-a830-c93e3022f6c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:36.282930 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf210568-fd5d-4fd0-a830-c93e3022f6c7 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:36.288704 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf877cbe-ec46-49eb-b811-4c800743a07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:36.289371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf877cbe-ec46-49eb-b811-4c800743a07e None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:36.289652 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf877cbe-ec46-49eb-b811-4c800743a07e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:36.289925 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf877cbe-ec46-49eb-b811-4c800743a07e None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:36.290666 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf877cbe-ec46-49eb-b811-4c800743a07e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:36.292155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf877cbe-ec46-49eb-b811-4c800743a07e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:36.301027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-bf210568-fd5d-4fd0-a830-c93e3022f6c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:36.301027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:36.301027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:36.301027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:36.301027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:36.301027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:36.301027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:36.301027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:36.301027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:36.301027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:36.301027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:36.301027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:36.301027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:36.301027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:36.301027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:36.301983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:36.302990 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:36.302990 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:36.302990 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:36.302990 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:36.302990 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 256/516] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:36.303300 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf210568-fd5d-4fd0-a830-c93e3022f6c7 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:36.322377 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 261/517] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:04:36 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:04:36.322377 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf877cbe-ec46-49eb-b811-4c800743a07e None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:36.369939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:36.420988 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf210568-fd5d-4fd0-a830-c93e3022f6c7 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:36.443294 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf877cbe-ec46-49eb-b811-4c800743a07e None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:36.462371 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15861233-6bf9-4e53-aff8-9b8c465f1b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:36.463056 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15861233-6bf9-4e53-aff8-9b8c465f1b6a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:36.463369 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15861233-6bf9-4e53-aff8-9b8c465f1b6a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:36.463910 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15861233-6bf9-4e53-aff8-9b8c465f1b6a None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:36.464700 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15861233-6bf9-4e53-aff8-9b8c465f1b6a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:36.465007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15861233-6bf9-4e53-aff8-9b8c465f1b6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:36.466532 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15861233-6bf9-4e53-aff8-9b8c465f1b6a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:36.481526 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74935fc1-4dc2-4d01-bf3e-f8c74796e1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:36.482395 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74935fc1-4dc2-4d01-bf3e-f8c74796e1b7 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:36.482687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74935fc1-4dc2-4d01-bf3e-f8c74796e1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:36.482962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74935fc1-4dc2-4d01-bf3e-f8c74796e1b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:36.487241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 257/518] 10.222.0.84 () {56 vars in 1139 bytes} [Thu May 19 20:04:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:36.488511 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15861233-6bf9-4e53-aff8-9b8c465f1b6a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:36.495108 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74935fc1-4dc2-4d01-bf3e-f8c74796e1b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:36.497197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74935fc1-4dc2-4d01-bf3e-f8c74796e1b7 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:36.515837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-74935fc1-4dc2-4d01-bf3e-f8c74796e1b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:36.515837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:36.515837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:36.515837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:36.515837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:36.515837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:36.515837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:36.515837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:36.515837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:36.515837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:36.515837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:36.515837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:36.515837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:36.515837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:36.515837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:36.516830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:36.517765 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:36.517765 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:36.517765 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:36.517765 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:36.519615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 262/519] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:36.520484 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74935fc1-4dc2-4d01-bf3e-f8c74796e1b7 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:36.547903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:36.616678 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15861233-6bf9-4e53-aff8-9b8c465f1b6a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:36.631624 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:36.642987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74935fc1-4dc2-4d01-bf3e-f8c74796e1b7 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:36.657588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229aee8-ab19-4178-be11-036756965c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:36.658447 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229aee8-ab19-4178-be11-036756965c8a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:36.658744 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229aee8-ab19-4178-be11-036756965c8a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:36.658997 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229aee8-ab19-4178-be11-036756965c8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:36.666830 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229aee8-ab19-4178-be11-036756965c8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:36.669034 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229aee8-ab19-4178-be11-036756965c8a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:36.677815 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd72bf7-e0fd-494e-a9e7-2d961d50c3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:36.678530 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd72bf7-e0fd-494e-a9e7-2d961d50c3f0 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:36.678817 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd72bf7-e0fd-494e-a9e7-2d961d50c3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:36.679090 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd72bf7-e0fd-494e-a9e7-2d961d50c3f0 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:36.679815 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd72bf7-e0fd-494e-a9e7-2d961d50c3f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:36.680126 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd72bf7-e0fd-494e-a9e7-2d961d50c3f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:36.681485 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd72bf7-e0fd-494e-a9e7-2d961d50c3f0 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:36.685032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-4229aee8-ab19-4178-be11-036756965c8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:36.685032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:36.685032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:36.685032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:36.685032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:36.685032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:36.685032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:36.685032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:36.685032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:36.685032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:36.685032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:36.685032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:36.685032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:36.685032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:36.685032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:36.685973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:36.687002 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:36.687002 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:36.687002 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:36.687002 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:36.687002 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:36.687002 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:36.687002 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:36.687002 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:36.687002 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 258/520] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:36.687560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229aee8-ab19-4178-be11-036756965c8a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:36.702602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 263/521] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:36.704076 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd72bf7-e0fd-494e-a9e7-2d961d50c3f0 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:36.796928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:36.800583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229aee8-ab19-4178-be11-036756965c8a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:36.810731 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd72bf7-e0fd-494e-a9e7-2d961d50c3f0 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:36.839257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b09c4c-45e3-4907-bdf5-c65d06101ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:36.839921 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b09c4c-45e3-4907-bdf5-c65d06101ccb None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:36.840467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b09c4c-45e3-4907-bdf5-c65d06101ccb None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:36.840467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b09c4c-45e3-4907-bdf5-c65d06101ccb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:36.849089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b703b-408e-4df9-a33d-39a52a9195bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:36.850180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b703b-408e-4df9-a33d-39a52a9195bc None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:36.850514 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b703b-408e-4df9-a33d-39a52a9195bc None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:36.850791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b703b-408e-4df9-a33d-39a52a9195bc None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:36.851557 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b703b-408e-4df9-a33d-39a52a9195bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:36.852224 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b703b-408e-4df9-a33d-39a52a9195bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:36.854150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b703b-408e-4df9-a33d-39a52a9195bc None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:36.866462 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 264/522] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:36.867170 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b703b-408e-4df9-a33d-39a52a9195bc None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:36.880647 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:36.918785 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b09c4c-45e3-4907-bdf5-c65d06101ccb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:36.920278 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b09c4c-45e3-4907-bdf5-c65d06101ccb None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:36.974005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b703b-408e-4df9-a33d-39a52a9195bc None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:37.008671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe49820-0748-40e0-aa62-5aedf70239e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:37.009347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe49820-0748-40e0-aa62-5aedf70239e7 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:37.009631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe49820-0748-40e0-aa62-5aedf70239e7 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:37.009907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe49820-0748-40e0-aa62-5aedf70239e7 None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:37.010783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe49820-0748-40e0-aa62-5aedf70239e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:37.011221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe49820-0748-40e0-aa62-5aedf70239e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:37.012680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe49820-0748-40e0-aa62-5aedf70239e7 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:37.024972 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:37.030363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 259/523] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:37.031822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b09c4c-45e3-4907-bdf5-c65d06101ccb None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:37.036578 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 265/524] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:37.037426 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe49820-0748-40e0-aa62-5aedf70239e7 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:37.114584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:37.144619 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b09c4c-45e3-4907-bdf5-c65d06101ccb None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:37.148618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe49820-0748-40e0-aa62-5aedf70239e7 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:37.179985 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f8d94-041d-48a6-adb4-a0c1d27da45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:37.180801 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f8d94-041d-48a6-adb4-a0c1d27da45d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:37.181080 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f8d94-041d-48a6-adb4-a0c1d27da45d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:37.181360 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f8d94-041d-48a6-adb4-a0c1d27da45d None admin] PATH_INFO: `/v3/users/glance` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:37.188342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b02fc-ecde-4657-9423-794e13e4f580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:37.189371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b02fc-ecde-4657-9423-794e13e4f580 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:37.189549 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f8d94-041d-48a6-adb4-a0c1d27da45d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:37.189982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b02fc-ecde-4657-9423-794e13e4f580 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:37.190457 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b02fc-ecde-4657-9423-794e13e4f580 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:37.191105 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f8d94-041d-48a6-adb4-a0c1d27da45d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:37.198067 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b02fc-ecde-4657-9423-794e13e4f580 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:37.199802 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b02fc-ecde-4657-9423-794e13e4f580 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-a12f8d94-041d-48a6-adb4-a0c1d27da45d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:37.207280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:37.208462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:37.209506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:37.209506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:37.209506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:37.209506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:37.209506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:37.209506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:37.209506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:37.209506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:37.209506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:04:37.209506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:37.209506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 260/525] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:37.210136 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f8d94-041d-48a6-adb4-a0c1d27da45d None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:37.218965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-9b7b02fc-ecde-4657-9423-794e13e4f580 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:37.218965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:37.218965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:37.218965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:37.218965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:37.218965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:37.218965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:37.218965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:37.218965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:37.218965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:37.218965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:37.218965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:37.218965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:37.218965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:37.218965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:37.220435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:37.221548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:37.221548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:37.221548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:37.221548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:37.221548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:37.221548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:37.221548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:37.221548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:37.223462 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 266/526] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:37.224505 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b02fc-ecde-4657-9423-794e13e4f580 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:37.279788 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:37.328697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f8d94-041d-48a6-adb4-a0c1d27da45d None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:37.335153 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b02fc-ecde-4657-9423-794e13e4f580 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:37.365380 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701e70f-ab22-4056-90d7-38fac9556cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:37.366212 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e70f-ab22-4056-90d7-38fac9556cf0 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:37.366616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e70f-ab22-4056-90d7-38fac9556cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:37.366892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e70f-ab22-4056-90d7-38fac9556cf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:37.369297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:37.376553 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bab752-ed11-4e8c-a84c-c6c42ae96de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:37.377224 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bab752-ed11-4e8c-a84c-c6c42ae96de4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:37.377509 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bab752-ed11-4e8c-a84c-c6c42ae96de4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:37.377774 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bab752-ed11-4e8c-a84c-c6c42ae96de4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:37.377882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701e70f-ab22-4056-90d7-38fac9556cf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:37.378681 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bab752-ed11-4e8c-a84c-c6c42ae96de4 None admin] RBAC: Adding query filter params (role.id=bdaf8732b34c43a38465efde7e18a060, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=378b046698c04b7b88194a9ffe5b045a) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:37.378996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bab752-ed11-4e8c-a84c-c6c42ae96de4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:37.379378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701e70f-ab22-4056-90d7-38fac9556cf0 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:37.380412 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bab752-ed11-4e8c-a84c-c6c42ae96de4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:37.394424 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 267/527] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:04:37 2022] GET /identity/v3/role_assignments?user.id=378b046698c04b7b88194a9ffe5b045a&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=bdaf8732b34c43a38465efde7e18a060 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:37.394424 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bab752-ed11-4e8c-a84c-c6c42ae96de4 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:37.408896 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-5701e70f-ab22-4056-90d7-38fac9556cf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:37.408896 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:37.408896 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:37.408896 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:37.408896 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:37.408896 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:37.408896 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:37.408896 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:37.408896 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:37.408896 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:37.408896 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:37.408896 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:37.408896 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:37.408896 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:37.408896 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:37.409977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:37.410992 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:37.410992 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:37.410992 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:37.410992 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:37.410992 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 261/528] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:37.415416 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701e70f-ab22-4056-90d7-38fac9556cf0 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:37.536542 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bab752-ed11-4e8c-a84c-c6c42ae96de4 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:37.550242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701e70f-ab22-4056-90d7-38fac9556cf0 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:37.574925 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:37.586245 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0282a-e1a7-4d18-8d11-4dc8490d16de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:37.586899 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0282a-e1a7-4d18-8d11-4dc8490d16de None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:37.587278 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0282a-e1a7-4d18-8d11-4dc8490d16de None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:37.587651 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0282a-e1a7-4d18-8d11-4dc8490d16de None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:37.588661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0282a-e1a7-4d18-8d11-4dc8490d16de None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:37.589096 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0282a-e1a7-4d18-8d11-4dc8490d16de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:37.590773 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0282a-e1a7-4d18-8d11-4dc8490d16de None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:37.604575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81954c-ee21-40e0-a9cc-d6979bb71b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:37.606845 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81954c-ee21-40e0-a9cc-d6979bb71b13 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:37.607280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81954c-ee21-40e0-a9cc-d6979bb71b13 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:37.607696 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81954c-ee21-40e0-a9cc-d6979bb71b13 None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:37.608674 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81954c-ee21-40e0-a9cc-d6979bb71b13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:37.608988 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81954c-ee21-40e0-a9cc-d6979bb71b13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:37.610490 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81954c-ee21-40e0-a9cc-d6979bb71b13 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:37.629880 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 262/529] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:37.630709 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81954c-ee21-40e0-a9cc-d6979bb71b13 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:37.643319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 268/530] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 20:04:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:37.644968 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0282a-e1a7-4d18-8d11-4dc8490d16de None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:37.712965 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:37.763991 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81954c-ee21-40e0-a9cc-d6979bb71b13 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:37.778622 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0282a-e1a7-4d18-8d11-4dc8490d16de None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:37.802584 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d0ec0-708d-4510-b7df-d1837fff3925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:37.803266 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d0ec0-708d-4510-b7df-d1837fff3925 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:37.803549 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d0ec0-708d-4510-b7df-d1837fff3925 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:37.803825 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d0ec0-708d-4510-b7df-d1837fff3925 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:37.804560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d0ec0-708d-4510-b7df-d1837fff3925 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:37.804867 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d0ec0-708d-4510-b7df-d1837fff3925 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:37.806527 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d0ec0-708d-4510-b7df-d1837fff3925 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:37.822881 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce356a-4ba6-445c-8372-8a45c4e9c5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:37.823785 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce356a-4ba6-445c-8372-8a45c4e9c5fb None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:37.824140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce356a-4ba6-445c-8372-8a45c4e9c5fb None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:37.824481 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce356a-4ba6-445c-8372-8a45c4e9c5fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:37.825281 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 263/531] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:37.825644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce356a-4ba6-445c-8372-8a45c4e9c5fb None admin] RBAC: Adding query filter params (role.id=bdaf8732b34c43a38465efde7e18a060, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=c97675d166b84a88aed79d26499a0ed7) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:37.825970 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce356a-4ba6-445c-8372-8a45c4e9c5fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:37.826195 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d0ec0-708d-4510-b7df-d1837fff3925 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:37.827462 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce356a-4ba6-445c-8372-8a45c4e9c5fb None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:37.837075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 269/532] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:04:37 2022] GET /identity/v3/role_assignments?user.id=c97675d166b84a88aed79d26499a0ed7&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=bdaf8732b34c43a38465efde7e18a060 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:37.837982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce356a-4ba6-445c-8372-8a45c4e9c5fb None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:37.895261 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:38.007398 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d0ec0-708d-4510-b7df-d1837fff3925 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:38.057137 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78b9b7-8d03-4677-ab90-8ab52635db9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:38.058477 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78b9b7-8d03-4677-ab90-8ab52635db9c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:38.058761 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78b9b7-8d03-4677-ab90-8ab52635db9c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:38.059040 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78b9b7-8d03-4677-ab90-8ab52635db9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:38.086995 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce356a-4ba6-445c-8372-8a45c4e9c5fb None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:38.123812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:38.152878 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78b9b7-8d03-4677-ab90-8ab52635db9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:38.154317 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78b9b7-8d03-4677-ab90-8ab52635db9c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:38.171821 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61a942-9ca7-42e5-9377-7b5224b3eb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:38.172644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61a942-9ca7-42e5-9377-7b5224b3eb46 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:38.172925 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61a942-9ca7-42e5-9377-7b5224b3eb46 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:38.173197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61a942-9ca7-42e5-9377-7b5224b3eb46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:38.188994 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:38.195508 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61a942-9ca7-42e5-9377-7b5224b3eb46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:38.197012 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61a942-9ca7-42e5-9377-7b5224b3eb46 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:38.218848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-9d61a942-9ca7-42e5-9377-7b5224b3eb46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:38.218848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:38.218848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:38.218848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:38.218848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:38.218848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:38.218848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:38.218848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:38.218848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:38.218848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:38.218848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:38.218848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:38.218848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:38.218848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:38.218848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:38.220198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:38.221763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:38.221763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:38.221763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:38.221763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:38.221763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:38.221763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:38.221763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:38.221763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:38.221763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 270/533] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:38.226300 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61a942-9ca7-42e5-9377-7b5224b3eb46 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:38.275871 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 264/534] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:38.281073 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78b9b7-8d03-4677-ab90-8ab52635db9c None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:38.394508 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78b9b7-8d03-4677-ab90-8ab52635db9c None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:38.429316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:38.435682 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57a958-8455-48c5-9836-c6fd9af6e5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:38.436618 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57a958-8455-48c5-9836-c6fd9af6e5fb None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:38.437119 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57a958-8455-48c5-9836-c6fd9af6e5fb None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:38.437694 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57a958-8455-48c5-9836-c6fd9af6e5fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:38.450813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57a958-8455-48c5-9836-c6fd9af6e5fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:38.452481 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57a958-8455-48c5-9836-c6fd9af6e5fb None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:38.474977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-1f57a958-8455-48c5-9836-c6fd9af6e5fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:38.474977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:38.474977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:38.474977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:38.474977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:38.474977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:38.474977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:38.474977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:38.474977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:38.474977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:38.474977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:38.474977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:38.474977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:38.474977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:38.474977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:38.476089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:38.477121 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:38.477121 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:38.477121 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:38.477121 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:38.477955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 265/535] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:38.487873 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57a958-8455-48c5-9836-c6fd9af6e5fb None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:38.540815 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61a942-9ca7-42e5-9377-7b5224b3eb46 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:38.596023 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea919795-f316-491c-8217-7980d783c380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:38.597100 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea919795-f316-491c-8217-7980d783c380 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:38.597603 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea919795-f316-491c-8217-7980d783c380 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:38.598095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea919795-f316-491c-8217-7980d783c380 None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:38.599121 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea919795-f316-491c-8217-7980d783c380 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:38.599643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea919795-f316-491c-8217-7980d783c380 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:38.601600 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea919795-f316-491c-8217-7980d783c380 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:38.615310 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 271/536] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:38.623234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea919795-f316-491c-8217-7980d783c380 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:38.688986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:38.711283 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57a958-8455-48c5-9836-c6fd9af6e5fb None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:38.758562 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69382f54-4d3d-435f-b913-e6b79a220df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:38.760069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69382f54-4d3d-435f-b913-e6b79a220df4 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:38.760871 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69382f54-4d3d-435f-b913-e6b79a220df4 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:38.761492 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69382f54-4d3d-435f-b913-e6b79a220df4 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:38.762461 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69382f54-4d3d-435f-b913-e6b79a220df4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:38.762987 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69382f54-4d3d-435f-b913-e6b79a220df4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:38.763180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea919795-f316-491c-8217-7980d783c380 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:38.764987 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69382f54-4d3d-435f-b913-e6b79a220df4 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:38.784608 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 266/537] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:38.786833 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d0cef-3ddd-4f47-862b-7d534b45dbc8 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:38.787567 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d0cef-3ddd-4f47-862b-7d534b45dbc8 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:38.788035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d0cef-3ddd-4f47-862b-7d534b45dbc8 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:38.790544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 267/538] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:38.791601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d0cef-3ddd-4f47-862b-7d534b45dbc8 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:38.801270 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cc76c-6cdd-4f9d-854d-35a0345c2909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:38.802019 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cc76c-6cdd-4f9d-854d-35a0345c2909 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:38.802416 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cc76c-6cdd-4f9d-854d-35a0345c2909 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:38.803100 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cc76c-6cdd-4f9d-854d-35a0345c2909 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:38.803750 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cc76c-6cdd-4f9d-854d-35a0345c2909 None admin] RBAC: Adding query filter params (role.id=bdaf8732b34c43a38465efde7e18a060, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=e2c5489b02604fcbae8dd5eff816d39d) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:38.804134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cc76c-6cdd-4f9d-854d-35a0345c2909 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:38.805891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cc76c-6cdd-4f9d-854d-35a0345c2909 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:38.812348 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 272/539] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:04:38 2022] GET /identity/v3/role_assignments?user.id=e2c5489b02604fcbae8dd5eff816d39d&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=bdaf8732b34c43a38465efde7e18a060 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:38.819932 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9273c-caed-44e4-b19a-795231d5e227 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:38.820157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9273c-caed-44e4-b19a-795231d5e227 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:38.820508 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9273c-caed-44e4-b19a-795231d5e227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:38.849394 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:38.859515 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-33a9273c-caed-44e4-b19a-795231d5e227 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:38.860849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-33a9273c-caed-44e4-b19a-795231d5e227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:38.915370 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d0cef-3ddd-4f47-862b-7d534b45dbc8 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:38.961213 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b423e33-fc6c-48d9-b7b8-b0467faab55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:38.962622 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b423e33-fc6c-48d9-b7b8-b0467faab55e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:38.964115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b423e33-fc6c-48d9-b7b8-b0467faab55e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:38.964599 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b423e33-fc6c-48d9-b7b8-b0467faab55e None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:38.982233 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b423e33-fc6c-48d9-b7b8-b0467faab55e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:38.984097 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b423e33-fc6c-48d9-b7b8-b0467faab55e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:39.008284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-2b423e33-fc6c-48d9-b7b8-b0467faab55e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:39.008284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:39.008284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:39.008284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:39.008284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:39.008284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:39.008284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:39.008284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:39.008284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:39.008284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:39.008284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:39.008284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:39.008284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:39.008284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:39.008284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:39.009336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:39.010555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:39.010555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:39.010555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:39.010555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:39.010555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:39.010555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:39.010555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:39.010555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:39.013392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 268/540] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:39.023335 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b423e33-fc6c-48d9-b7b8-b0467faab55e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:39.032082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 273/541] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:39.056751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9273c-caed-44e4-b19a-795231d5e227 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:39.098175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:39.187709 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b423e33-fc6c-48d9-b7b8-b0467faab55e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:39.190402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:39.228655 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eddacb8-5e60-41ec-b8c1-32b200502bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:39.229295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddacb8-5e60-41ec-b8c1-32b200502bac None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:39.229574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddacb8-5e60-41ec-b8c1-32b200502bac None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:39.229849 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddacb8-5e60-41ec-b8c1-32b200502bac None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:39.230683 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddacb8-5e60-41ec-b8c1-32b200502bac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:39.234140 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddacb8-5e60-41ec-b8c1-32b200502bac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:39.237553 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddacb8-5e60-41ec-b8c1-32b200502bac None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:39.253404 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9273c-caed-44e4-b19a-795231d5e227 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:39.266078 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 269/542] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:39.270876 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eddacb8-5e60-41ec-b8c1-32b200502bac None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:39.316218 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ea030-91f3-46c0-9a18-2d5651caa6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:39.317039 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ea030-91f3-46c0-9a18-2d5651caa6c3 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:39.317320 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ea030-91f3-46c0-9a18-2d5651caa6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:39.317594 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ea030-91f3-46c0-9a18-2d5651caa6c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:39.335940 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ea030-91f3-46c0-9a18-2d5651caa6c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-531ea030-91f3-46c0-9a18-2d5651caa6c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:39.343909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:39.345241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:39.346208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:39.346208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:39.346208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:39.346208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:39.346208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:39.346208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:39.346208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:39.346208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:39.346208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:39.346208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:39.346208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:39.346208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:39.351425 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 274/543] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:39.355257 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ea030-91f3-46c0-9a18-2d5651caa6c3 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:39.452801 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eddacb8-5e60-41ec-b8c1-32b200502bac None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:39.496986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29fac2-24c1-46ff-8a07-65c5de77b9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:39.497575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29fac2-24c1-46ff-8a07-65c5de77b9ff None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:39.497855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29fac2-24c1-46ff-8a07-65c5de77b9ff None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:39.498134 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29fac2-24c1-46ff-8a07-65c5de77b9ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:39.499007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29fac2-24c1-46ff-8a07-65c5de77b9ff None admin] RBAC: Adding query filter params (role.id=bdaf8732b34c43a38465efde7e18a060, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=be8df5550506438eb334c649893eb4dd) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:39.499321 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29fac2-24c1-46ff-8a07-65c5de77b9ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:39.500778 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29fac2-24c1-46ff-8a07-65c5de77b9ff None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:39.514805 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 270/544] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:04:39 2022] GET /identity/v3/role_assignments?user.id=be8df5550506438eb334c649893eb4dd&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=bdaf8732b34c43a38465efde7e18a060 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:39.532643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:39.550897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ea030-91f3-46c0-9a18-2d5651caa6c3 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:39.603274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf216f-4e49-4f5e-9480-bf43dc3e83c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:39.605402 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf216f-4e49-4f5e-9480-bf43dc3e83c7 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:39.606032 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf216f-4e49-4f5e-9480-bf43dc3e83c7 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:39.606568 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf216f-4e49-4f5e-9480-bf43dc3e83c7 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:39.607595 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf216f-4e49-4f5e-9480-bf43dc3e83c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:39.608133 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf216f-4e49-4f5e-9480-bf43dc3e83c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:39.609782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf216f-4e49-4f5e-9480-bf43dc3e83c7 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:39.625931 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 275/545] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:39.635592 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29fac2-24c1-46ff-8a07-65c5de77b9ff None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:39.693484 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:39.806252 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29fac2-24c1-46ff-8a07-65c5de77b9ff None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:39.855511 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6386c-07f2-4ee9-a62d-dba2e9fdd235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:39.856760 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6386c-07f2-4ee9-a62d-dba2e9fdd235 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:39.857261 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6386c-07f2-4ee9-a62d-dba2e9fdd235 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:39.857752 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6386c-07f2-4ee9-a62d-dba2e9fdd235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:39.873169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6386c-07f2-4ee9-a62d-dba2e9fdd235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:39.875021 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6386c-07f2-4ee9-a62d-dba2e9fdd235 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:39.897361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-92d6386c-07f2-4ee9-a62d-dba2e9fdd235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:39.897361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:39.897361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:39.897361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:39.897361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:39.897361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:39.897361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:39.897361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:39.897361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:39.897361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:39.897361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:39.897361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:39.897361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:39.897361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:39.897361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:39.898403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:39.899464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:39.899464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:39.899464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:39.899464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:39.899464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 271/546] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:39.906290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf216f-4e49-4f5e-9480-bf43dc3e83c7 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:39.945458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7d4f7-17ef-4b98-a39d-1e4b6d6a289f None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:39.945699 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7d4f7-17ef-4b98-a39d-1e4b6d6a289f None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:39.945945 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7d4f7-17ef-4b98-a39d-1e4b6d6a289f None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:39.947457 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 272/547] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:39.953704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985cefc-280d-4860-8ac7-16fc4ec54594 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:39.953988 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985cefc-280d-4860-8ac7-16fc4ec54594 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:39.954283 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985cefc-280d-4860-8ac7-16fc4ec54594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:39.987370 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-2985cefc-280d-4860-8ac7-16fc4ec54594 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:39.988596 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-2985cefc-280d-4860-8ac7-16fc4ec54594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:40.045903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf216f-4e49-4f5e-9480-bf43dc3e83c7 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:40.116044 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831ebc1-0e91-4eab-854c-b47d4c5cf6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:40.116683 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831ebc1-0e91-4eab-854c-b47d4c5cf6f0 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:40.116971 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831ebc1-0e91-4eab-854c-b47d4c5cf6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:40.117248 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831ebc1-0e91-4eab-854c-b47d4c5cf6f0 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:40.118004 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831ebc1-0e91-4eab-854c-b47d4c5cf6f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:40.118336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831ebc1-0e91-4eab-854c-b47d4c5cf6f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:40.119866 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831ebc1-0e91-4eab-854c-b47d4c5cf6f0 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:40.140823 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 273/548] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:04:40.152677 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 276/549] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:40.158154 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985cefc-280d-4860-8ac7-16fc4ec54594 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:40.170651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:40.210545 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831ebc1-0e91-4eab-854c-b47d4c5cf6f0 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:40.230791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:40.316862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985cefc-280d-4860-8ac7-16fc4ec54594 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:40.345238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831ebc1-0e91-4eab-854c-b47d4c5cf6f0 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:40.359749 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1216498-b4ee-4178-9fac-00163d77f329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:40.361292 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1216498-b4ee-4178-9fac-00163d77f329 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:40.362390 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1216498-b4ee-4178-9fac-00163d77f329 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:40.362918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1216498-b4ee-4178-9fac-00163d77f329 None admin] PATH_INFO: `/v3/users/swift` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:40.374319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1216498-b4ee-4178-9fac-00163d77f329 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:40.377039 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1216498-b4ee-4178-9fac-00163d77f329 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:40.383719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609acda2-a599-4c47-9d72-ed009b4fef43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:40.384607 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609acda2-a599-4c47-9d72-ed009b4fef43 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:40.384953 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609acda2-a599-4c47-9d72-ed009b4fef43 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:40.385326 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609acda2-a599-4c47-9d72-ed009b4fef43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-a1216498-b4ee-4178-9fac-00163d77f329 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:40.396217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:40.397246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:40.398441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:40.398441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:40.398441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:40.398441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:40.398441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:40.398441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:40.398441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:40.398441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:40.398441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:04:40.398441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:40.399054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609acda2-a599-4c47-9d72-ed009b4fef43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:40.405172 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 274/550] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:04:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-609acda2-a599-4c47-9d72-ed009b4fef43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:40.405321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:40.406549 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:40.407676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:40.407676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:40.407676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:40.407676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:40.407676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:40.407676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:40.407676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:40.407676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:40.407676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:40.407676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:40.407676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:40.407676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:40.408712 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 277/551] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:40.409865 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8ef67-5b1e-4cb0-807c-6bf2cf5d489e None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:40.410156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8ef67-5b1e-4cb0-807c-6bf2cf5d489e None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:40.410471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8ef67-5b1e-4cb0-807c-6bf2cf5d489e None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:40.410871 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1216498-b4ee-4178-9fac-00163d77f329 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:40.416011 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 278/552] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:40.417084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8ef67-5b1e-4cb0-807c-6bf2cf5d489e None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:40.490844 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:40.504400 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:40.524146 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1216498-b4ee-4178-9fac-00163d77f329 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:40.542494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8ef67-5b1e-4cb0-807c-6bf2cf5d489e None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:40.570847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280354b-0040-4eaa-98c3-ecfecf84a503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:40.571788 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280354b-0040-4eaa-98c3-ecfecf84a503 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:40.572260 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280354b-0040-4eaa-98c3-ecfecf84a503 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:40.572910 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280354b-0040-4eaa-98c3-ecfecf84a503 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:40.574036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280354b-0040-4eaa-98c3-ecfecf84a503 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:40.576137 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280354b-0040-4eaa-98c3-ecfecf84a503 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:40.578190 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280354b-0040-4eaa-98c3-ecfecf84a503 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:40.581175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6d56d-20e7-44d6-89a4-d292360136d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:40.582543 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6d56d-20e7-44d6-89a4-d292360136d8 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:40.582543 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6d56d-20e7-44d6-89a4-d292360136d8 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:40.582543 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6d56d-20e7-44d6-89a4-d292360136d8 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:40.583242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a6d56d-20e7-44d6-89a4-d292360136d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:40.583552 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a6d56d-20e7-44d6-89a4-d292360136d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:40.584924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a6d56d-20e7-44d6-89a4-d292360136d8 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:40.594683 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 279/553] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:40.596035 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b107e05-d5a7-44ac-8b81-ff18c7bc5c1f None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:40.596330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b107e05-d5a7-44ac-8b81-ff18c7bc5c1f None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:40.596610 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b107e05-d5a7-44ac-8b81-ff18c7bc5c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:40.601325 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 275/554] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 20:04:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:40.602524 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280354b-0040-4eaa-98c3-ecfecf84a503 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:40.619241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-1b107e05-d5a7-44ac-8b81-ff18c7bc5c1f None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:40.620604 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-1b107e05-d5a7-44ac-8b81-ff18c7bc5c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:40.692876 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 280/555] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:40.693700 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b107e05-d5a7-44ac-8b81-ff18c7bc5c1f None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:40.706649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280354b-0040-4eaa-98c3-ecfecf84a503 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:40.734637 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:40.745305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843f5d7-1730-4387-82dd-24d8885f60e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:40.746680 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843f5d7-1730-4387-82dd-24d8885f60e4 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:40.747173 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843f5d7-1730-4387-82dd-24d8885f60e4 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:40.747629 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843f5d7-1730-4387-82dd-24d8885f60e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:40.758431 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:40.761217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843f5d7-1730-4387-82dd-24d8885f60e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:40.762755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843f5d7-1730-4387-82dd-24d8885f60e4 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:40.786302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-8843f5d7-1730-4387-82dd-24d8885f60e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:40.786302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:40.786302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:40.786302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:40.786302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:40.786302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:40.786302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:40.786302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:40.786302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:40.786302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:40.786302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:40.786302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:40.786302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:40.786302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:40.786302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:40.787316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:40.788285 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:40.788285 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:40.788285 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:40.788285 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:40.788285 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 276/556] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:40.789060 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843f5d7-1730-4387-82dd-24d8885f60e4 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:40.805871 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b107e05-d5a7-44ac-8b81-ff18c7bc5c1f None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:40.846065 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac72813-ddd1-4637-af4e-fc4cda9c322d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:40.847137 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac72813-ddd1-4637-af4e-fc4cda9c322d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:40.847651 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac72813-ddd1-4637-af4e-fc4cda9c322d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:40.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac72813-ddd1-4637-af4e-fc4cda9c322d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:40.921260 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac72813-ddd1-4637-af4e-fc4cda9c322d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:40.923232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac72813-ddd1-4637-af4e-fc4cda9c322d None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:40.923378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843f5d7-1730-4387-82dd-24d8885f60e4 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:40.969573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa64616-5226-46c5-9816-75396f8f8eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:40.970328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa64616-5226-46c5-9816-75396f8f8eb8 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:40.970610 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa64616-5226-46c5-9816-75396f8f8eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:40.970882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa64616-5226-46c5-9816-75396f8f8eb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:40.985906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa64616-5226-46c5-9816-75396f8f8eb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-3fa64616-5226-46c5-9816-75396f8f8eb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:40.995807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:40.996906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:40.997892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:40.997892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:40.997892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:40.997892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:40.997892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:40.997892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:40.997892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:40.997892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:40.997892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:40.997892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:40.997892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:40.997892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:40.997892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 277/557] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:40.997892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa64616-5226-46c5-9816-75396f8f8eb8 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:41.005857 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:41.047905 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 281/558] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:41.049248 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac72813-ddd1-4637-af4e-fc4cda9c322d None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:41.050082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:41.123189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa64616-5226-46c5-9816-75396f8f8eb8 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:41.156699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac72813-ddd1-4637-af4e-fc4cda9c322d None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:41.171506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0facc-ec1e-449c-b09f-f2ee277c2c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:41.172775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0facc-ec1e-449c-b09f-f2ee277c2c3f None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:41.173489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0facc-ec1e-449c-b09f-f2ee277c2c3f None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:41.173943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0facc-ec1e-449c-b09f-f2ee277c2c3f None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:41.175003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0facc-ec1e-449c-b09f-f2ee277c2c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:41.175474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0facc-ec1e-449c-b09f-f2ee277c2c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:41.177284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0facc-ec1e-449c-b09f-f2ee277c2c3f None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:41.193594 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 278/559] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:41.193813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f9ff1-6e51-4268-ab1c-96107244b1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:41.194667 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f9ff1-6e51-4268-ab1c-96107244b1e6 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:41.195049 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f9ff1-6e51-4268-ab1c-96107244b1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:41.195245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0facc-ec1e-449c-b09f-f2ee277c2c3f None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:41.195508 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f9ff1-6e51-4268-ab1c-96107244b1e6 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:41.196364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f9ff1-6e51-4268-ab1c-96107244b1e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:41.196772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f9ff1-6e51-4268-ab1c-96107244b1e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:41.198888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f9ff1-6e51-4268-ab1c-96107244b1e6 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:41.208817 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 282/560] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:41.209750 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f9ff1-6e51-4268-ab1c-96107244b1e6 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:41.262740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:41.317813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f9ff1-6e51-4268-ab1c-96107244b1e6 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:41.322091 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:41.326979 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0facc-ec1e-449c-b09f-f2ee277c2c3f None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:41.368937 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba9c89-db88-488a-8673-b36b62f22c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:41.369547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c253bc-27ad-4336-a783-d13d4c215cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:41.370429 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c253bc-27ad-4336-a783-d13d4c215cc4 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:41.370723 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba9c89-db88-488a-8673-b36b62f22c09 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:41.370902 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c253bc-27ad-4336-a783-d13d4c215cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:41.371274 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c253bc-27ad-4336-a783-d13d4c215cc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:41.371460 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba9c89-db88-488a-8673-b36b62f22c09 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:41.371887 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba9c89-db88-488a-8673-b36b62f22c09 None admin] PATH_INFO: `/v3/users/nova` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:41.382363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba9c89-db88-488a-8673-b36b62f22c09 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:41.383050 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c253bc-27ad-4336-a783-d13d4c215cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:41.384227 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba9c89-db88-488a-8673-b36b62f22c09 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:41.384712 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c253bc-27ad-4336-a783-d13d4c215cc4 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-36ba9c89-db88-488a-8673-b36b62f22c09 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:41.401275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:41.402569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:41.403811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:41.403811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:41.403811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:41.403811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:41.403811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:41.403811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:41.403811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:41.403811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:41.403811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:04:41.403811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:41.406324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 283/561] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:04:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:41.406616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-18c253bc-27ad-4336-a783-d13d4c215cc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:41.406616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:41.406616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:41.406616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:41.406616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:41.406616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:41.406616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:41.406616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:41.406616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:41.406616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:41.406616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:41.406616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:41.406616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:41.406616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:41.406616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:41.407684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:41.408611 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:41.408611 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:41.408611 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:41.408611 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:41.409844 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba9c89-db88-488a-8673-b36b62f22c09 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:41.413125 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 279/562] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:41.414136 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c253bc-27ad-4336-a783-d13d4c215cc4 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:41.513442 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:41.529770 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba9c89-db88-488a-8673-b36b62f22c09 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:41.546727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c253bc-27ad-4336-a783-d13d4c215cc4 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:41.566750 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280b34a-e741-4893-bca4-f1700f1d9535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:41.567606 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280b34a-e741-4893-bca4-f1700f1d9535 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:41.568267 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280b34a-e741-4893-bca4-f1700f1d9535 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:41.568550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280b34a-e741-4893-bca4-f1700f1d9535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:41.580598 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6280b34a-e741-4893-bca4-f1700f1d9535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:41.582075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6280b34a-e741-4893-bca4-f1700f1d9535 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:41.587877 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d376c8-9e29-4637-97b9-19ecb3e27470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:41.588894 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d376c8-9e29-4637-97b9-19ecb3e27470 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:41.589544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d376c8-9e29-4637-97b9-19ecb3e27470 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:41.589975 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d376c8-9e29-4637-97b9-19ecb3e27470 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:41.591143 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d376c8-9e29-4637-97b9-19ecb3e27470 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:41.591612 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d376c8-9e29-4637-97b9-19ecb3e27470 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:41.593206 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d376c8-9e29-4637-97b9-19ecb3e27470 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:41.599631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-6280b34a-e741-4893-bca4-f1700f1d9535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:41.599631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:41.599631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:41.599631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:41.599631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:41.599631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:41.599631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:41.599631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:41.599631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:41.599631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:41.599631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:41.599631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:41.599631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:41.599631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:41.599631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:41.600718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:41.606702 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:41.606702 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:41.606702 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:41.606702 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:41.606702 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 284/563] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:41.606702 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280b34a-e741-4893-bca4-f1700f1d9535 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:41.622413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 280/564] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:04:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:41.624228 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73579285-cb1d-4e43-9e9c-b8833f092f22 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:41.624698 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73579285-cb1d-4e43-9e9c-b8833f092f22 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:41.625169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73579285-cb1d-4e43-9e9c-b8833f092f22 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:41.630963 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 281/565] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:41.632181 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73579285-cb1d-4e43-9e9c-b8833f092f22 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:41.645586 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:41.716812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280b34a-e741-4893-bca4-f1700f1d9535 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:41.747108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73579285-cb1d-4e43-9e9c-b8833f092f22 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:41.753806 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc561b43-eda0-41d6-b0be-3e63105a39c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:41.754901 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc561b43-eda0-41d6-b0be-3e63105a39c3 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:41.755194 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc561b43-eda0-41d6-b0be-3e63105a39c3 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:41.755469 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc561b43-eda0-41d6-b0be-3e63105a39c3 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:41.756194 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc561b43-eda0-41d6-b0be-3e63105a39c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:41.756834 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc561b43-eda0-41d6-b0be-3e63105a39c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:41.758550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc561b43-eda0-41d6-b0be-3e63105a39c3 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:41.768619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:41.781507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 285/566] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:41.781507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc561b43-eda0-41d6-b0be-3e63105a39c3 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:41.785407 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93beece3-7735-4de4-abda-ac40fe239604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:41.786156 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beece3-7735-4de4-abda-ac40fe239604 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:41.786481 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beece3-7735-4de4-abda-ac40fe239604 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:41.786760 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beece3-7735-4de4-abda-ac40fe239604 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:41.787500 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93beece3-7735-4de4-abda-ac40fe239604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:41.787808 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93beece3-7735-4de4-abda-ac40fe239604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:41.789204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93beece3-7735-4de4-abda-ac40fe239604 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:41.806773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 282/567] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:41.808251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5348a-a62b-4f86-b254-7c07484b8248 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:41.808531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5348a-a62b-4f86-b254-7c07484b8248 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:41.808821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5348a-a62b-4f86-b254-7c07484b8248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:41.825175 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-30c5348a-a62b-4f86-b254-7c07484b8248 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:41.826401 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-30c5348a-a62b-4f86-b254-7c07484b8248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:41.889371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc561b43-eda0-41d6-b0be-3e63105a39c3 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:41.897720 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:41.903300 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 283/568] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:41.904292 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5348a-a62b-4f86-b254-7c07484b8248 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:41.937090 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429e193-c1c3-4e19-b59b-45b5c0c387a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:41.937903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429e193-c1c3-4e19-b59b-45b5c0c387a6 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:41.938188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429e193-c1c3-4e19-b59b-45b5c0c387a6 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:41.938497 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429e193-c1c3-4e19-b59b-45b5c0c387a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:41.962880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429e193-c1c3-4e19-b59b-45b5c0c387a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:41.964408 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429e193-c1c3-4e19-b59b-45b5c0c387a6 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:41.983853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-5429e193-c1c3-4e19-b59b-45b5c0c387a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:41.983853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:41.983853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:41.983853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:41.983853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:41.983853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:41.983853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:41.983853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:41.983853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:41.983853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:41.983853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:41.983853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:41.983853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:41.983853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:41.983853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:41.985054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:41.986182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:41.986182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:41.986182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:41.986182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:41.986182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 286/569] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:41.986182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429e193-c1c3-4e19-b59b-45b5c0c387a6 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:42.018938 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5348a-a62b-4f86-b254-7c07484b8248 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:42.041823 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:42.057399 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6f58a-d64b-4d1b-b0ad-2f4b68253121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:42.058214 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6f58a-d64b-4d1b-b0ad-2f4b68253121 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:42.058528 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6f58a-d64b-4d1b-b0ad-2f4b68253121 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:42.058800 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6f58a-d64b-4d1b-b0ad-2f4b68253121 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:42.065687 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6f58a-d64b-4d1b-b0ad-2f4b68253121 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:42.067178 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6f58a-d64b-4d1b-b0ad-2f4b68253121 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:42.080918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-ede6f58a-d64b-4d1b-b0ad-2f4b68253121 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:42.080918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:42.080918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:42.080918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:42.080918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:42.080918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:42.080918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:42.080918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:42.080918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:42.080918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:42.080918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:42.080918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:42.080918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:42.080918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:42.080918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:42.082074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:42.083270 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:42.083270 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:42.083270 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:42.083270 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:42.083270 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:42.083270 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:42.083270 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:42.083270 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:42.083270 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 284/570] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:42.083270 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6f58a-d64b-4d1b-b0ad-2f4b68253121 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:42.097658 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429e193-c1c3-4e19-b59b-45b5c0c387a6 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:42.132122 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bf063-b1e0-40d1-b243-430de0c5d278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:42.132918 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bf063-b1e0-40d1-b243-430de0c5d278 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:42.133194 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bf063-b1e0-40d1-b243-430de0c5d278 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:42.133463 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bf063-b1e0-40d1-b243-430de0c5d278 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:42.140487 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55bf063-b1e0-40d1-b243-430de0c5d278 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:42.141430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:42.141933 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55bf063-b1e0-40d1-b243-430de0c5d278 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-c55bf063-b1e0-40d1-b243-430de0c5d278 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:42.156760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:42.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:42.159017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:42.159017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:42.159017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:42.159017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:42.159017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:42.159017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:42.159017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:42.159017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:42.159017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:04:42.159017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:42.159017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 287/571] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:42.159017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bf063-b1e0-40d1-b243-430de0c5d278 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:42.187932 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6f58a-d64b-4d1b-b0ad-2f4b68253121 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:42.222443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1505730-a7cb-4210-871a-d71bdbdf3e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:42.223241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1505730-a7cb-4210-871a-d71bdbdf3e38 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:42.223550 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1505730-a7cb-4210-871a-d71bdbdf3e38 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:42.223787 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1505730-a7cb-4210-871a-d71bdbdf3e38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:42.234606 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1505730-a7cb-4210-871a-d71bdbdf3e38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-f1505730-a7cb-4210-871a-d71bdbdf3e38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:42.242288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:42.243475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:42.244676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:42.244676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:42.244676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:42.244676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:42.244676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:42.244676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:42.244676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:42.244676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:42.244676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:42.244676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:42.244676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:42.244676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:42.244676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 285/572] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:42.245572 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f689d1-6f59-4823-84ba-03b16a0e2820 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:42.245572 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f689d1-6f59-4823-84ba-03b16a0e2820 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:42.245734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f689d1-6f59-4823-84ba-03b16a0e2820 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:42.246920 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 286/573] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:42.247602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f689d1-6f59-4823-84ba-03b16a0e2820 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:42.263826 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bf063-b1e0-40d1-b243-430de0c5d278 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:42.275233 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:42.299704 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b3779-1898-4476-a626-0a93cb176aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:42.300392 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b3779-1898-4476-a626-0a93cb176aee None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:42.300685 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b3779-1898-4476-a626-0a93cb176aee None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:42.300968 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b3779-1898-4476-a626-0a93cb176aee None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:42.301687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b3779-1898-4476-a626-0a93cb176aee None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:42.301985 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b3779-1898-4476-a626-0a93cb176aee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:42.303391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b3779-1898-4476-a626-0a93cb176aee None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:42.315029 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 288/574] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:42.315946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b3779-1898-4476-a626-0a93cb176aee None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:42.355267 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f689d1-6f59-4823-84ba-03b16a0e2820 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:42.374556 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:42.390281 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95523f-4b4b-43a7-8583-73949a2768db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:42.390947 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95523f-4b4b-43a7-8583-73949a2768db None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:42.391225 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95523f-4b4b-43a7-8583-73949a2768db None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:42.391495 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95523f-4b4b-43a7-8583-73949a2768db None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:42.392274 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95523f-4b4b-43a7-8583-73949a2768db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:42.392581 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95523f-4b4b-43a7-8583-73949a2768db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:42.394002 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95523f-4b4b-43a7-8583-73949a2768db None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:42.406506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 287/575] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:42.407339 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95523f-4b4b-43a7-8583-73949a2768db None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:42.415373 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b3779-1898-4476-a626-0a93cb176aee None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:42.447844 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5be39-07f7-4fc0-8ffb-f4a5d74efda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:42.448526 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5be39-07f7-4fc0-8ffb-f4a5d74efda2 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:42.448828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5be39-07f7-4fc0-8ffb-f4a5d74efda2 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:42.449082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5be39-07f7-4fc0-8ffb-f4a5d74efda2 None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:42.449921 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a5be39-07f7-4fc0-8ffb-f4a5d74efda2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:42.450245 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a5be39-07f7-4fc0-8ffb-f4a5d74efda2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:42.451691 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a5be39-07f7-4fc0-8ffb-f4a5d74efda2 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:42.468805 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 289/576] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 20:04:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:42.470381 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749be3c-2020-4343-a9d6-3c5d8f722990 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:42.470669 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749be3c-2020-4343-a9d6-3c5d8f722990 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:42.470942 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749be3c-2020-4343-a9d6-3c5d8f722990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:42.487554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-a749be3c-2020-4343-a9d6-3c5d8f722990 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:42.488740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-a749be3c-2020-4343-a9d6-3c5d8f722990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:42.510309 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95523f-4b4b-43a7-8583-73949a2768db None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:42.511955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:42.543947 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1449d0-0f8f-4874-810c-66a9164c8786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:42.544643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1449d0-0f8f-4874-810c-66a9164c8786 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:42.544916 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1449d0-0f8f-4874-810c-66a9164c8786 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:42.545185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1449d0-0f8f-4874-810c-66a9164c8786 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:42.545927 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1449d0-0f8f-4874-810c-66a9164c8786 None admin] RBAC: Adding query filter params (name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:42.546217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1449d0-0f8f-4874-810c-66a9164c8786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:42.547634 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1449d0-0f8f-4874-810c-66a9164c8786 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:42.555728 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 288/577] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:42.566423 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 290/578] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:42.566621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1449d0-0f8f-4874-810c-66a9164c8786 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:42.566741 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749be3c-2020-4343-a9d6-3c5d8f722990 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:42.611661 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:42.656766 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749be3c-2020-4343-a9d6-3c5d8f722990 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:42.668936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1449d0-0f8f-4874-810c-66a9164c8786 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:42.692761 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65469c9d-3e35-4c46-ad86-e760a62a9449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:42.693430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65469c9d-3e35-4c46-ad86-e760a62a9449 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:42.693716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65469c9d-3e35-4c46-ad86-e760a62a9449 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:42.693998 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65469c9d-3e35-4c46-ad86-e760a62a9449 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:42.694883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65469c9d-3e35-4c46-ad86-e760a62a9449 None admin] RBAC: Adding query filter params (role.id=bdaf8732b34c43a38465efde7e18a060, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=87b30300be1a41128ff88e12e260f63b) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:42.695201 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65469c9d-3e35-4c46-ad86-e760a62a9449 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:42.696652 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65469c9d-3e35-4c46-ad86-e760a62a9449 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:42.702256 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f74326-be09-4adf-9ae9-05baa6def64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:42.702436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 291/579] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:04:42 2022] GET /identity/v3/role_assignments?user.id=87b30300be1a41128ff88e12e260f63b&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=bdaf8732b34c43a38465efde7e18a060 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:42.703078 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f74326-be09-4adf-9ae9-05baa6def64c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:42.703195 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65469c9d-3e35-4c46-ad86-e760a62a9449 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:42.703363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f74326-be09-4adf-9ae9-05baa6def64c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:42.703705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f74326-be09-4adf-9ae9-05baa6def64c None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:42.713008 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f74326-be09-4adf-9ae9-05baa6def64c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:42.714571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f74326-be09-4adf-9ae9-05baa6def64c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:42.730223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-f5f74326-be09-4adf-9ae9-05baa6def64c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:42.730223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:42.730223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:42.730223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:42.730223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:42.730223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:42.730223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:42.730223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:42.730223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:42.730223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:42.730223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:42.730223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:42.730223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:42.730223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:42.730223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:42.731338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:42.732279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:42.732279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:42.732279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:42.732279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:42.732279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:42.732279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:42.732279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:42.732279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:42.732279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 289/580] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:42.732826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f74326-be09-4adf-9ae9-05baa6def64c None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:42.738567 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:42.826889 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65469c9d-3e35-4c46-ad86-e760a62a9449 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:42.850131 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f74326-be09-4adf-9ae9-05baa6def64c None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:42.851846 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:42.866683 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071e50c-6f94-4fb7-8f58-c165e451ecb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:42.867806 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071e50c-6f94-4fb7-8f58-c165e451ecb9 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:42.868251 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071e50c-6f94-4fb7-8f58-c165e451ecb9 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:42.868696 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071e50c-6f94-4fb7-8f58-c165e451ecb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:42.880081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071e50c-6f94-4fb7-8f58-c165e451ecb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:42.881828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071e50c-6f94-4fb7-8f58-c165e451ecb9 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:42.888384 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6463d33-6ceb-4156-b933-d24b6a514c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:42.889197 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6463d33-6ceb-4156-b933-d24b6a514c7e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:42.889476 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6463d33-6ceb-4156-b933-d24b6a514c7e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:42.889749 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6463d33-6ceb-4156-b933-d24b6a514c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:42.900440 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6463d33-6ceb-4156-b933-d24b6a514c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:42.901943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6463d33-6ceb-4156-b933-d24b6a514c7e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:42.903013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-6071e50c-6f94-4fb7-8f58-c165e451ecb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:42.903013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:42.903013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:42.903013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:42.903013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:42.903013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:42.903013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:42.903013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:42.903013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:42.903013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:42.903013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:42.903013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:42.903013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:42.903013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:42.903013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:42.903936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:42.904917 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:42.904917 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:42.904917 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:42.904917 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:42.904917 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 292/581] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:42.905209 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071e50c-6f94-4fb7-8f58-c165e451ecb9 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:42.925794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-b6463d33-6ceb-4156-b933-d24b6a514c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:42.925794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:42.925794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:42.925794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:42.925794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:42.925794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:42.925794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:42.925794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:42.925794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:42.925794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:42.925794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:42.925794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:42.925794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:42.925794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:42.925794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:42.926860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:42.933247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:42.933247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:42.933247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:42.933247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:42.933247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 290/582] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:42.933247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6463d33-6ceb-4156-b933-d24b6a514c7e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:42.990291 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:43.018536 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071e50c-6f94-4fb7-8f58-c165e451ecb9 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:43.035412 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6463d33-6ceb-4156-b933-d24b6a514c7e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:43.055828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5d248-ae20-47d7-a094-a43238b0b8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:43.057387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5d248-ae20-47d7-a094-a43238b0b8cd None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:43.058155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5d248-ae20-47d7-a094-a43238b0b8cd None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:43.058625 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5d248-ae20-47d7-a094-a43238b0b8cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:43.070231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da4510-96a3-4f47-b2fa-a14e8b49da0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:43.070924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da4510-96a3-4f47-b2fa-a14e8b49da0e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:43.071206 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da4510-96a3-4f47-b2fa-a14e8b49da0e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:43.071499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da4510-96a3-4f47-b2fa-a14e8b49da0e None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:43.072268 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da4510-96a3-4f47-b2fa-a14e8b49da0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:43.072615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5d248-ae20-47d7-a094-a43238b0b8cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:43.072828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da4510-96a3-4f47-b2fa-a14e8b49da0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:43.074777 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da4510-96a3-4f47-b2fa-a14e8b49da0e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-5fc5d248-ae20-47d7-a094-a43238b0b8cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:43.082031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:43.083249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:43.084340 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:43.084340 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:43.084340 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:43.084340 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:43.084340 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:43.084340 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:43.084340 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:43.084340 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:43.084340 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:43.084340 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:43.084340 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:43.084340 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:43.087318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 293/583] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:43.088404 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5d248-ae20-47d7-a094-a43238b0b8cd None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:43.088933 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 291/584] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:43.089817 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da4510-96a3-4f47-b2fa-a14e8b49da0e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:43.109814 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:43.201742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da4510-96a3-4f47-b2fa-a14e8b49da0e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:43.205019 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5d248-ae20-47d7-a094-a43238b0b8cd None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:43.239103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308fd40-74f9-415a-957b-1a9217fc87f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:43.239505 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4312cfd-c796-45c6-b36c-70113ec11eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:43.239779 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308fd40-74f9-415a-957b-1a9217fc87f9 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:43.240063 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308fd40-74f9-415a-957b-1a9217fc87f9 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:43.240167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4312cfd-c796-45c6-b36c-70113ec11eb0 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:43.240332 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308fd40-74f9-415a-957b-1a9217fc87f9 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:43.240436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4312cfd-c796-45c6-b36c-70113ec11eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:43.240694 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4312cfd-c796-45c6-b36c-70113ec11eb0 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:43.241051 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308fd40-74f9-415a-957b-1a9217fc87f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:43.241357 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308fd40-74f9-415a-957b-1a9217fc87f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:43.241461 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4312cfd-c796-45c6-b36c-70113ec11eb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:43.241709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4312cfd-c796-45c6-b36c-70113ec11eb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:43.242778 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308fd40-74f9-415a-957b-1a9217fc87f9 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:43.243106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4312cfd-c796-45c6-b36c-70113ec11eb0 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:43.244826 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:43.255914 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 292/585] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:43.256533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308fd40-74f9-415a-957b-1a9217fc87f9 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:43.265547 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 294/586] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:43.265547 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4312cfd-c796-45c6-b36c-70113ec11eb0 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:43.336105 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:43.364562 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308fd40-74f9-415a-957b-1a9217fc87f9 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:43.368164 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4312cfd-c796-45c6-b36c-70113ec11eb0 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:43.400832 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76d473-681a-477e-a5ed-091e88e68018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:43.401723 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76d473-681a-477e-a5ed-091e88e68018 None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:43.402042 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76d473-681a-477e-a5ed-091e88e68018 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:43.402250 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76d473-681a-477e-a5ed-091e88e68018 None admin] PATH_INFO: `/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/378b046698c04b7b88194a9ffe5b045a/roles/bdaf8732b34c43a38465efde7e18a060` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:43.404408 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd93654-faa8-4a22-8ec8-8bbad64150ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:43.405082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd93654-faa8-4a22-8ec8-8bbad64150ff None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:43.405362 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd93654-faa8-4a22-8ec8-8bbad64150ff None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:43.405639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd93654-faa8-4a22-8ec8-8bbad64150ff None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:43.406418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd93654-faa8-4a22-8ec8-8bbad64150ff None admin] RBAC: Adding query filter params (name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:43.406723 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd93654-faa8-4a22-8ec8-8bbad64150ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:43.408098 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd93654-faa8-4a22-8ec8-8bbad64150ff None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:43.416709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 295/587] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:43.418287 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76d473-681a-477e-a5ed-091e88e68018 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba22fdc274b74521847e5fceb2d307b1, user_id=378b046698c04b7b88194a9ffe5b045a, role_id=bdaf8732b34c43a38465efde7e18a060)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:43.419762 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76d473-681a-477e-a5ed-091e88e68018 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:43.421779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd93654-faa8-4a22-8ec8-8bbad64150ff None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:43.434485 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 293/588] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:04:43 2022] PUT /identity/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/378b046698c04b7b88194a9ffe5b045a/roles/bdaf8732b34c43a38465efde7e18a060 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:04:43.435824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76d473-681a-477e-a5ed-091e88e68018 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:43.502835 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:43.557287 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76d473-681a-477e-a5ed-091e88e68018 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:43.580823 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd93654-faa8-4a22-8ec8-8bbad64150ff None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:43.588861 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:43.606871 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bdaa6-fe64-43ed-8463-f9d3ead603ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:43.608114 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bdaa6-fe64-43ed-8463-f9d3ead603ac None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:43.608694 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bdaa6-fe64-43ed-8463-f9d3ead603ac None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:43.609290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bdaa6-fe64-43ed-8463-f9d3ead603ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:43.618625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bdaa6-fe64-43ed-8463-f9d3ead603ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:43.620616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bdaa6-fe64-43ed-8463-f9d3ead603ac None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:43.627189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5a0cd-0706-4d4d-933f-c4a558858f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:43.628012 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5a0cd-0706-4d4d-933f-c4a558858f56 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:43.628311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5a0cd-0706-4d4d-933f-c4a558858f56 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:43.628625 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5a0cd-0706-4d4d-933f-c4a558858f56 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:43.637063 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5a0cd-0706-4d4d-933f-c4a558858f56 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:43.638148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-369bdaa6-fe64-43ed-8463-f9d3ead603ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:43.638148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:43.638148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:43.638148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:43.638148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:43.638148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:43.638148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:43.638148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:43.638148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:43.638148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:43.638148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:43.638148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:43.638148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:43.638148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:43.638148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:43.639101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:43.640035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:43.640035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:43.640035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:43.640035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:43.640035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:43.640035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:43.640035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:43.640035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:43.640503 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5a0cd-0706-4d4d-933f-c4a558858f56 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:43.642539 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 294/589] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:43.643853 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bdaa6-fe64-43ed-8463-f9d3ead603ac None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-69f5a0cd-0706-4d4d-933f-c4a558858f56 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:43.653703 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:43.654840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:43.655854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:43.655854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:43.655854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:43.655854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:43.655854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:43.655854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:43.655854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:43.655854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:43.655854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:04:43.655854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:43.659783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 296/590] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:43.660524 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5a0cd-0706-4d4d-933f-c4a558858f56 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:43.784472 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bdaa6-fe64-43ed-8463-f9d3ead603ac None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:43.806916 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5a0cd-0706-4d4d-933f-c4a558858f56 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:43.828955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa156a74-9d29-44b4-a753-1d99163d789d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:43.829770 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa156a74-9d29-44b4-a753-1d99163d789d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:43.830049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa156a74-9d29-44b4-a753-1d99163d789d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:43.830361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa156a74-9d29-44b4-a753-1d99163d789d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:43.831168 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:43.843514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa156a74-9d29-44b4-a753-1d99163d789d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:43.845344 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa156a74-9d29-44b4-a753-1d99163d789d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:43.855240 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19693a-2148-4a00-8c73-f5a2a2181ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:43.856487 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19693a-2148-4a00-8c73-f5a2a2181ff8 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:43.856989 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19693a-2148-4a00-8c73-f5a2a2181ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:43.857486 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19693a-2148-4a00-8c73-f5a2a2181ff8 None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:43.858504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19693a-2148-4a00-8c73-f5a2a2181ff8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:43.859030 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19693a-2148-4a00-8c73-f5a2a2181ff8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:43.860670 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19693a-2148-4a00-8c73-f5a2a2181ff8 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:43.864247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-fa156a74-9d29-44b4-a753-1d99163d789d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:43.864247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:43.864247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:43.864247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:43.864247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:43.864247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:43.864247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:43.864247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:43.864247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:43.864247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:43.864247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:43.864247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:43.864247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:43.864247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:43.864247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:43.865191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:43.866748 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:43.866748 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:43.866748 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:43.866748 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:43.866748 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 295/591] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:43.866748 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa156a74-9d29-44b4-a753-1d99163d789d None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:43.879236 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 297/592] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:43.880865 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19693a-2148-4a00-8c73-f5a2a2181ff8 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:43.911957 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:43.994625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa156a74-9d29-44b4-a753-1d99163d789d None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:44.012014 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19693a-2148-4a00-8c73-f5a2a2181ff8 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:44.039506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8baf6f-8a62-4519-9b47-c827dfdc64ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:44.040754 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8baf6f-8a62-4519-9b47-c827dfdc64ed None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:44.041451 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8baf6f-8a62-4519-9b47-c827dfdc64ed None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:44.042326 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8baf6f-8a62-4519-9b47-c827dfdc64ed None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:44.043649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8baf6f-8a62-4519-9b47-c827dfdc64ed None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:44.044313 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8baf6f-8a62-4519-9b47-c827dfdc64ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:44.046675 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8baf6f-8a62-4519-9b47-c827dfdc64ed None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:44.053180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570260f-88a0-43f2-b88a-68924afab36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:44.053857 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570260f-88a0-43f2-b88a-68924afab36e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:44.054137 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570260f-88a0-43f2-b88a-68924afab36e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:44.054454 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570260f-88a0-43f2-b88a-68924afab36e None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:44.055183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570260f-88a0-43f2-b88a-68924afab36e None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:44.055486 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570260f-88a0-43f2-b88a-68924afab36e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:44.057371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570260f-88a0-43f2-b88a-68924afab36e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:44.070208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 296/593] 10.222.0.84 () {56 vars in 1139 bytes} [Thu May 19 20:04:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:44.071509 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8baf6f-8a62-4519-9b47-c827dfdc64ed None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:44.071729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 298/594] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:44.077973 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570260f-88a0-43f2-b88a-68924afab36e None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:44.126211 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:44.189651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:44.192018 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570260f-88a0-43f2-b88a-68924afab36e None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:44.199649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8baf6f-8a62-4519-9b47-c827dfdc64ed None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:44.228374 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab04f11-7e33-49ab-85bf-447cd47db59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:44.229513 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab04f11-7e33-49ab-85bf-447cd47db59e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:44.229957 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab04f11-7e33-49ab-85bf-447cd47db59e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:44.230311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab04f11-7e33-49ab-85bf-447cd47db59e None admin] PATH_INFO: `/v3/users/glance` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:44.239695 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab04f11-7e33-49ab-85bf-447cd47db59e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:44.241318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab04f11-7e33-49ab-85bf-447cd47db59e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:44.245424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef117cd-fcc8-4721-826f-5a7f3f232383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:44.246593 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef117cd-fcc8-4721-826f-5a7f3f232383 None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:44.247049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef117cd-fcc8-4721-826f-5a7f3f232383 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:44.247480 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef117cd-fcc8-4721-826f-5a7f3f232383 None admin] PATH_INFO: `/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/c97675d166b84a88aed79d26499a0ed7/roles/bdaf8732b34c43a38465efde7e18a060` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-cab04f11-7e33-49ab-85bf-447cd47db59e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:44.256305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:44.257563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:44.258839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:44.258839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:44.258839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:44.258839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:44.258839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:44.258839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:44.258839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:44.258839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:44.258839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:04:44.258839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:44.265627 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 299/595] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:44.265627 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab04f11-7e33-49ab-85bf-447cd47db59e None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:44.274397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef117cd-fcc8-4721-826f-5a7f3f232383 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba22fdc274b74521847e5fceb2d307b1, user_id=c97675d166b84a88aed79d26499a0ed7, role_id=bdaf8732b34c43a38465efde7e18a060)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:44.276551 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef117cd-fcc8-4721-826f-5a7f3f232383 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:44.293279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 297/596] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:04:44 2022] PUT /identity/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/c97675d166b84a88aed79d26499a0ed7/roles/bdaf8732b34c43a38465efde7e18a060 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:04:44.294547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef117cd-fcc8-4721-826f-5a7f3f232383 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:44.370293 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:44.376413 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab04f11-7e33-49ab-85bf-447cd47db59e None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:44.413205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dc878-bbb4-4fde-94ed-2f9c6be0f3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:44.414572 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dc878-bbb4-4fde-94ed-2f9c6be0f3c0 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:44.414572 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dc878-bbb4-4fde-94ed-2f9c6be0f3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:44.414931 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dc878-bbb4-4fde-94ed-2f9c6be0f3c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:44.426994 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2dc878-bbb4-4fde-94ed-2f9c6be0f3c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:44.427663 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef117cd-fcc8-4721-826f-5a7f3f232383 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:44.428740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2dc878-bbb4-4fde-94ed-2f9c6be0f3c0 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:44.447699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-dc2dc878-bbb4-4fde-94ed-2f9c6be0f3c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:44.447699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:44.447699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:44.447699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:44.447699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:44.447699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:44.447699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:44.447699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:44.447699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:44.447699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:44.447699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:44.447699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:44.447699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:44.447699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:44.447699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:44.449436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:44.450704 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:44.450704 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:44.450704 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:44.450704 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:44.451881 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 300/597] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:44.459611 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dc878-bbb4-4fde-94ed-2f9c6be0f3c0 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:44.504872 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e7bd4-f5b0-4bde-a16e-88c35fc82386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:44.505807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e7bd4-f5b0-4bde-a16e-88c35fc82386 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:44.506186 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e7bd4-f5b0-4bde-a16e-88c35fc82386 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:44.506575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e7bd4-f5b0-4bde-a16e-88c35fc82386 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:44.507546 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e7bd4-f5b0-4bde-a16e-88c35fc82386 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:44.507957 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e7bd4-f5b0-4bde-a16e-88c35fc82386 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:44.510861 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e7bd4-f5b0-4bde-a16e-88c35fc82386 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:44.563399 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 298/598] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 20:04:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:44.565762 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e7bd4-f5b0-4bde-a16e-88c35fc82386 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:44.580508 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:44.631772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dc878-bbb4-4fde-94ed-2f9c6be0f3c0 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:44.672796 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2091c7-01dc-405e-a0e9-0d1f8b528dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:44.673568 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2091c7-01dc-405e-a0e9-0d1f8b528dca None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:44.673916 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2091c7-01dc-405e-a0e9-0d1f8b528dca None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:44.674272 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2091c7-01dc-405e-a0e9-0d1f8b528dca None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:44.675186 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2091c7-01dc-405e-a0e9-0d1f8b528dca None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:44.675566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2091c7-01dc-405e-a0e9-0d1f8b528dca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:44.677154 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2091c7-01dc-405e-a0e9-0d1f8b528dca None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:44.690831 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 301/599] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:44.696506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2091c7-01dc-405e-a0e9-0d1f8b528dca None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:44.697184 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e7bd4-f5b0-4bde-a16e-88c35fc82386 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:44.714379 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:44.749201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f48adc-bd0a-4164-9ff2-2226b2b33ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:44.750290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f48adc-bd0a-4164-9ff2-2226b2b33ac8 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:44.750756 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f48adc-bd0a-4164-9ff2-2226b2b33ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:44.751184 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f48adc-bd0a-4164-9ff2-2226b2b33ac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:44.777670 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f48adc-bd0a-4164-9ff2-2226b2b33ac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:44.781693 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f48adc-bd0a-4164-9ff2-2226b2b33ac8 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:44.820246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-b8f48adc-bd0a-4164-9ff2-2226b2b33ac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:44.820246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:44.820246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:44.820246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:44.820246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:44.820246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:44.820246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:44.820246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:44.820246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:44.820246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:44.820246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:44.820246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:44.820246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:44.820246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:44.820246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:44.821446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:44.822730 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:44.822730 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:44.822730 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:44.822730 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:44.830426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 299/600] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:44.835125 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f48adc-bd0a-4164-9ff2-2226b2b33ac8 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:44.925823 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2091c7-01dc-405e-a0e9-0d1f8b528dca None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:44.936954 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:44.976848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb35926-3d2e-4d31-bdde-9eb32e76b2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:44.978011 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb35926-3d2e-4d31-bdde-9eb32e76b2ca None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:44.978609 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb35926-3d2e-4d31-bdde-9eb32e76b2ca None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:44.978988 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb35926-3d2e-4d31-bdde-9eb32e76b2ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:44.986561 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb35926-3d2e-4d31-bdde-9eb32e76b2ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:44.988130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb35926-3d2e-4d31-bdde-9eb32e76b2ca None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:45.003244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-7cb35926-3d2e-4d31-bdde-9eb32e76b2ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:45.003244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:45.003244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:45.003244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:45.003244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:45.003244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:45.003244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:45.003244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:45.003244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:45.003244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:45.003244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:45.003244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:45.003244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:45.003244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:45.003244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:45.004259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:45.005246 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:45.005246 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:45.005246 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:45.005246 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:45.005246 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:45.005246 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:45.005246 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:45.005246 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:45.006812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f48adc-bd0a-4164-9ff2-2226b2b33ac8 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:45.009262 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 302/601] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:45.018132 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb405e16-accf-4398-b1c0-ac41fe1eac04 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:45.018132 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb405e16-accf-4398-b1c0-ac41fe1eac04 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:45.018132 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb405e16-accf-4398-b1c0-ac41fe1eac04 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:45.018132 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 303/602] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:45.018132 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb405e16-accf-4398-b1c0-ac41fe1eac04 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:45.047062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc69c70-10c1-445f-84cc-90c1822fa500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:45.047763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc69c70-10c1-445f-84cc-90c1822fa500 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:45.048039 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc69c70-10c1-445f-84cc-90c1822fa500 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:45.048325 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc69c70-10c1-445f-84cc-90c1822fa500 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:45.049033 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc69c70-10c1-445f-84cc-90c1822fa500 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:45.049338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc69c70-10c1-445f-84cc-90c1822fa500 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:45.050864 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc69c70-10c1-445f-84cc-90c1822fa500 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:45.064212 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 300/603] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:45.065610 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796174af-ae9c-4280-9992-2d5ddb8eee31 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:45.065894 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796174af-ae9c-4280-9992-2d5ddb8eee31 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:45.066293 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796174af-ae9c-4280-9992-2d5ddb8eee31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:45.084688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-796174af-ae9c-4280-9992-2d5ddb8eee31 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:45.086002 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-796174af-ae9c-4280-9992-2d5ddb8eee31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:45.097977 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:45.136510 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb405e16-accf-4398-b1c0-ac41fe1eac04 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:45.176822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 301/604] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:45.178287 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796174af-ae9c-4280-9992-2d5ddb8eee31 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:45.183785 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4d240-8fc9-428d-b9ba-232f8fc5f37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:45.185281 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4d240-8fc9-428d-b9ba-232f8fc5f37b None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:45.190682 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4d240-8fc9-428d-b9ba-232f8fc5f37b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:45.191145 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4d240-8fc9-428d-b9ba-232f8fc5f37b None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:45.192112 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4d240-8fc9-428d-b9ba-232f8fc5f37b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:45.192591 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4d240-8fc9-428d-b9ba-232f8fc5f37b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:45.194154 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4d240-8fc9-428d-b9ba-232f8fc5f37b None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:45.212655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 304/605] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:45.214219 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4d240-8fc9-428d-b9ba-232f8fc5f37b None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:45.248148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:45.314986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796174af-ae9c-4280-9992-2d5ddb8eee31 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:45.331628 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4d240-8fc9-428d-b9ba-232f8fc5f37b None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:45.360886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48927246-d878-47ee-9eb8-2047b00f4490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:45.361596 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:45.362814 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48927246-d878-47ee-9eb8-2047b00f4490 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:45.363391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48927246-d878-47ee-9eb8-2047b00f4490 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:45.364493 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48927246-d878-47ee-9eb8-2047b00f4490 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:45.371256 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a9eef-0e8b-46fb-8335-b0b4298da757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:45.372300 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a9eef-0e8b-46fb-8335-b0b4298da757 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:45.372692 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48927246-d878-47ee-9eb8-2047b00f4490 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:45.372835 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a9eef-0e8b-46fb-8335-b0b4298da757 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:45.373142 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a9eef-0e8b-46fb-8335-b0b4298da757 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:45.374635 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48927246-d878-47ee-9eb8-2047b00f4490 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:45.384265 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a9eef-0e8b-46fb-8335-b0b4298da757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:45.391069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-48927246-d878-47ee-9eb8-2047b00f4490 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:45.391069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:45.391069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:45.391069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:45.391069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:45.391069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:45.391069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:45.391069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:45.391069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:45.391069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:45.391069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:45.391069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:45.391069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:45.391069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:45.391069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:45.392236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:45.393396 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:45.393396 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:45.393396 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:45.393396 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:45.393396 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:45.393396 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:45.393396 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:45.393396 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-dd6a9eef-0e8b-46fb-8335-b0b4298da757 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:45.394371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:45.395506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:45.396575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:45.396575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:45.396575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:45.396575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:45.396575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:45.396575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:45.396575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:45.396575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:45.396575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:45.396575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:45.396575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:45.396575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:45.398684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 302/607] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:45.399732 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48927246-d878-47ee-9eb8-2047b00f4490 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:45.402614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 305/607] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:45.404364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a9eef-0e8b-46fb-8335-b0b4298da757 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:45.514403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48927246-d878-47ee-9eb8-2047b00f4490 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:45.517567 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:45.527372 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a9eef-0e8b-46fb-8335-b0b4298da757 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:45.557520 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ff95e-af74-4bf6-9928-688b01aafc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:45.558876 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ff95e-af74-4bf6-9928-688b01aafc9a None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:45.559300 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ff95e-af74-4bf6-9928-688b01aafc9a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:45.559835 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ff95e-af74-4bf6-9928-688b01aafc9a None admin] PATH_INFO: `/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/e2c5489b02604fcbae8dd5eff816d39d/roles/bdaf8732b34c43a38465efde7e18a060` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:45.576901 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a2daa-8da9-404c-9f73-3c9fa50ef838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:45.578218 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a2daa-8da9-404c-9f73-3c9fa50ef838 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:45.578683 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a2daa-8da9-404c-9f73-3c9fa50ef838 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:45.579370 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a2daa-8da9-404c-9f73-3c9fa50ef838 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:45.580296 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a2daa-8da9-404c-9f73-3c9fa50ef838 None admin] RBAC: Adding query filter params (name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:45.580762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a2daa-8da9-404c-9f73-3c9fa50ef838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:45.581491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ff95e-af74-4bf6-9928-688b01aafc9a None admin] RBAC: Authorizing `identity:create_grant(project_id=ba22fdc274b74521847e5fceb2d307b1, user_id=e2c5489b02604fcbae8dd5eff816d39d, role_id=bdaf8732b34c43a38465efde7e18a060)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:45.582509 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a2daa-8da9-404c-9f73-3c9fa50ef838 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:45.583245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ff95e-af74-4bf6-9928-688b01aafc9a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:45.598304 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 306/608] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:45.598304 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a2daa-8da9-404c-9f73-3c9fa50ef838 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:45.608532 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 303/609] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:04:45 2022] PUT /identity/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/e2c5489b02604fcbae8dd5eff816d39d/roles/bdaf8732b34c43a38465efde7e18a060 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:04:45.609740 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ff95e-af74-4bf6-9928-688b01aafc9a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:45.690813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:45.757758 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a2daa-8da9-404c-9f73-3c9fa50ef838 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:45.768799 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ff95e-af74-4bf6-9928-688b01aafc9a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:45.802174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9f5d3-56fc-433b-9966-71c957a7105b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:45.804074 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9f5d3-56fc-433b-9966-71c957a7105b None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:45.805326 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9f5d3-56fc-433b-9966-71c957a7105b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:45.805877 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9f5d3-56fc-433b-9966-71c957a7105b None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:45.808452 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9f5d3-56fc-433b-9966-71c957a7105b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:45.810191 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9f5d3-56fc-433b-9966-71c957a7105b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:45.811694 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9f5d3-56fc-433b-9966-71c957a7105b None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:45.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 307/610] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:45.848199 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95c439-a4ba-4466-9014-c3558eff2b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:45.849037 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95c439-a4ba-4466-9014-c3558eff2b0a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:45.849317 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95c439-a4ba-4466-9014-c3558eff2b0a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:45.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95c439-a4ba-4466-9014-c3558eff2b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:45.852188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9f5d3-56fc-433b-9966-71c957a7105b None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:45.876097 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95c439-a4ba-4466-9014-c3558eff2b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:45.879146 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95c439-a4ba-4466-9014-c3558eff2b0a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:45.916448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-9f95c439-a4ba-4466-9014-c3558eff2b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:45.916448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:45.916448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:45.916448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:45.916448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:45.916448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:45.916448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:45.916448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:45.916448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:45.916448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:45.916448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:45.916448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:45.916448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:45.916448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:45.916448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:45.917771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:45.918718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:45.918718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:45.918718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:45.918718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:45.918718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 304/611] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:45.925766 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95c439-a4ba-4466-9014-c3558eff2b0a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:45.936262 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:46.026454 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9f5d3-56fc-433b-9966-71c957a7105b None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:46.037765 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:46.066491 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bb991-07aa-4b3a-ace9-c0d551bcb150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:46.067395 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bb991-07aa-4b3a-ace9-c0d551bcb150 None admin] REQUEST_METHOD: `PUT` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:46.067678 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bb991-07aa-4b3a-ace9-c0d551bcb150 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:46.067955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bb991-07aa-4b3a-ace9-c0d551bcb150 None admin] PATH_INFO: `/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/be8df5550506438eb334c649893eb4dd/roles/bdaf8732b34c43a38465efde7e18a060` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:46.078357 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95c439-a4ba-4466-9014-c3558eff2b0a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:46.083794 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bb991-07aa-4b3a-ace9-c0d551bcb150 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba22fdc274b74521847e5fceb2d307b1, user_id=be8df5550506438eb334c649893eb4dd, role_id=bdaf8732b34c43a38465efde7e18a060)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:46.085238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bb991-07aa-4b3a-ace9-c0d551bcb150 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:46.102076 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 308/612] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:04:45 2022] PUT /identity/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/be8df5550506438eb334c649893eb4dd/roles/bdaf8732b34c43a38465efde7e18a060 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:04:46.103509 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4ec22-3436-477a-84b9-f1f62248bf3b None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:46.103878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4ec22-3436-477a-84b9-f1f62248bf3b None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:46.104393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4ec22-3436-477a-84b9-f1f62248bf3b None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:46.106237 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 309/613] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:46.112490 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069629f5-ef2f-4f88-bae4-03da4776e500 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:46.112833 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069629f5-ef2f-4f88-bae4-03da4776e500 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:46.113059 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069629f5-ef2f-4f88-bae4-03da4776e500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:46.116864 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67361a8-bc7c-4263-b8a3-7cfb4c75a7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:46.117547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67361a8-bc7c-4263-b8a3-7cfb4c75a7f8 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:46.117833 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67361a8-bc7c-4263-b8a3-7cfb4c75a7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:46.118109 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67361a8-bc7c-4263-b8a3-7cfb4c75a7f8 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:46.119041 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67361a8-bc7c-4263-b8a3-7cfb4c75a7f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:46.119515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67361a8-bc7c-4263-b8a3-7cfb4c75a7f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:46.121326 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67361a8-bc7c-4263-b8a3-7cfb4c75a7f8 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:46.132337 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-069629f5-ef2f-4f88-bae4-03da4776e500 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:46.133759 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-069629f5-ef2f-4f88-bae4-03da4776e500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:46.134778 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 305/614] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:46.144649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67361a8-bc7c-4263-b8a3-7cfb4c75a7f8 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:46.213855 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 310/615] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:04:46.238958 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069629f5-ef2f-4f88-bae4-03da4776e500 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:46.268022 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:46.325370 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67361a8-bc7c-4263-b8a3-7cfb4c75a7f8 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:46.343318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:46.356885 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069629f5-ef2f-4f88-bae4-03da4776e500 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:46.370241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2d42c-0c54-4393-baa0-efc505a12d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:46.371124 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2d42c-0c54-4393-baa0-efc505a12d17 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:46.371410 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2d42c-0c54-4393-baa0-efc505a12d17 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:46.371685 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2d42c-0c54-4393-baa0-efc505a12d17 None admin] PATH_INFO: `/v3/users/swift` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:46.380567 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2d42c-0c54-4393-baa0-efc505a12d17 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:46.382058 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2d42c-0c54-4393-baa0-efc505a12d17 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-f1b2d42c-0c54-4393-baa0-efc505a12d17 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:46.397810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:46.398936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:46.399924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:46.399924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:46.399924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:46.399924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:46.399924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:46.399924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:46.399924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:46.399924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:46.399924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:04:46.399924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:46.400482 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25944560-5531-4ca3-bb23-1942564d8cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:46.400612 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 306/616] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:04:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:46.402183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25944560-5531-4ca3-bb23-1942564d8cd3 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:46.402503 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25944560-5531-4ca3-bb23-1942564d8cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:46.402784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25944560-5531-4ca3-bb23-1942564d8cd3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:46.405819 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2d42c-0c54-4393-baa0-efc505a12d17 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:46.427726 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25944560-5531-4ca3-bb23-1942564d8cd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-25944560-5531-4ca3-bb23-1942564d8cd3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:46.436134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:46.437151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:46.438073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:46.438073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:46.438073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:46.438073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:46.438073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:46.438073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:46.438073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:46.438073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:46.438073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:46.438073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:46.438073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:46.438073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:46.442438 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 311/617] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:46.451306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25944560-5531-4ca3-bb23-1942564d8cd3 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:46.519465 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2d42c-0c54-4393-baa0-efc505a12d17 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:46.558215 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:46.563702 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1807fdb-0f53-4abf-a694-3ab9e6cbd80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:46.564686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1807fdb-0f53-4abf-a694-3ab9e6cbd80f None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:46.565302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1807fdb-0f53-4abf-a694-3ab9e6cbd80f None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:46.567021 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1807fdb-0f53-4abf-a694-3ab9e6cbd80f None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:46.568315 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1807fdb-0f53-4abf-a694-3ab9e6cbd80f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:46.570232 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1807fdb-0f53-4abf-a694-3ab9e6cbd80f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:46.578377 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1807fdb-0f53-4abf-a694-3ab9e6cbd80f None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:46.613058 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 307/618] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 20:04:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:46.617657 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1807fdb-0f53-4abf-a694-3ab9e6cbd80f None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:46.663815 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25944560-5531-4ca3-bb23-1942564d8cd3 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:46.722896 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96097f45-72a6-406d-af0b-0df3f5fda09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:46.723583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96097f45-72a6-406d-af0b-0df3f5fda09e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:46.723897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96097f45-72a6-406d-af0b-0df3f5fda09e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:46.724147 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96097f45-72a6-406d-af0b-0df3f5fda09e None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:46.724912 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96097f45-72a6-406d-af0b-0df3f5fda09e None admin] RBAC: Adding query filter params (name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:46.725222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96097f45-72a6-406d-af0b-0df3f5fda09e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:46.726970 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96097f45-72a6-406d-af0b-0df3f5fda09e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:46.741013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 312/619] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:46.747022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96097f45-72a6-406d-af0b-0df3f5fda09e None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:46.781859 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:46.813161 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1807fdb-0f53-4abf-a694-3ab9e6cbd80f None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:46.850696 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59c4ab-428b-4e4e-a136-624764417eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:46.851953 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59c4ab-428b-4e4e-a136-624764417eff None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:46.852449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59c4ab-428b-4e4e-a136-624764417eff None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:46.852933 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59c4ab-428b-4e4e-a136-624764417eff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:46.864445 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee59c4ab-428b-4e4e-a136-624764417eff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:46.866452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee59c4ab-428b-4e4e-a136-624764417eff None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:46.886733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-ee59c4ab-428b-4e4e-a136-624764417eff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:46.886733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:46.886733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:46.886733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:46.886733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:46.886733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:46.886733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:46.886733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:46.886733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:46.886733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:46.886733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:46.886733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:46.886733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:46.886733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:46.886733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:46.887771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:46.888728 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:46.888728 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:46.888728 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:46.888728 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:46.891075 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 308/620] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:46.896289 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59c4ab-428b-4e4e-a136-624764417eff None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:46.954935 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96097f45-72a6-406d-af0b-0df3f5fda09e None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:46.958636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:46.998628 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2000042-2b47-45d8-89f8-884e4f5f8288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:46.999436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2000042-2b47-45d8-89f8-884e4f5f8288 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:46.999721 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2000042-2b47-45d8-89f8-884e4f5f8288 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:47.000128 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2000042-2b47-45d8-89f8-884e4f5f8288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:47.011696 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2000042-2b47-45d8-89f8-884e4f5f8288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:47.013280 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2000042-2b47-45d8-89f8-884e4f5f8288 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:47.032607 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59c4ab-428b-4e4e-a136-624764417eff None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:47.033962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-e2000042-2b47-45d8-89f8-884e4f5f8288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:47.033962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:47.033962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:47.033962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:47.033962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:47.033962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:47.033962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:47.033962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:47.033962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:47.033962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:47.033962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:47.033962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:47.033962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:47.033962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:47.033962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:47.034926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:47.036436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:47.036436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:47.036436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:47.036436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:47.036436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 313/621] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:47.036849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fffd5-b0a2-4354-82bf-78cbc2a680fe None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:47.037060 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fffd5-b0a2-4354-82bf-78cbc2a680fe None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:47.037350 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fffd5-b0a2-4354-82bf-78cbc2a680fe None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:47.044001 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 314/622] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:47.044001 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fffd5-b0a2-4354-82bf-78cbc2a680fe None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:47.079467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701a32f-a8ea-4384-9a24-90affaeecd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:47.080523 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701a32f-a8ea-4384-9a24-90affaeecd9e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:47.080985 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701a32f-a8ea-4384-9a24-90affaeecd9e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:47.081530 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701a32f-a8ea-4384-9a24-90affaeecd9e None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:47.082459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701a32f-a8ea-4384-9a24-90affaeecd9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:47.082944 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701a32f-a8ea-4384-9a24-90affaeecd9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:47.084646 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701a32f-a8ea-4384-9a24-90affaeecd9e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:47.098865 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 309/623] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:47.101927 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509328a6-09a0-486b-8d06-21f4c4f59db5 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:47.102417 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509328a6-09a0-486b-8d06-21f4c4f59db5 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:47.102888 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509328a6-09a0-486b-8d06-21f4c4f59db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:47.125500 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:47.132834 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-509328a6-09a0-486b-8d06-21f4c4f59db5 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:47.135034 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-509328a6-09a0-486b-8d06-21f4c4f59db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:47.169258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fffd5-b0a2-4354-82bf-78cbc2a680fe None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:47.206629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1cd4a-2ca4-444a-8b30-0dd2a2390e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:47.207712 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1cd4a-2ca4-444a-8b30-0dd2a2390e34 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:47.208243 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1cd4a-2ca4-444a-8b30-0dd2a2390e34 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:47.208728 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1cd4a-2ca4-444a-8b30-0dd2a2390e34 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:47.209712 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1cd4a-2ca4-444a-8b30-0dd2a2390e34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:47.210218 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1cd4a-2ca4-444a-8b30-0dd2a2390e34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:47.211985 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1cd4a-2ca4-444a-8b30-0dd2a2390e34 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:47.226645 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 315/624] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:47.227947 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1cd4a-2ca4-444a-8b30-0dd2a2390e34 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:47.229286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 310/625] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:47.234310 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509328a6-09a0-486b-8d06-21f4c4f59db5 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:47.274894 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:47.355279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1cd4a-2ca4-444a-8b30-0dd2a2390e34 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:47.368869 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509328a6-09a0-486b-8d06-21f4c4f59db5 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:47.381801 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:47.395093 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e39662-70b9-4b6e-919f-37af526fbde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:47.396335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e39662-70b9-4b6e-919f-37af526fbde1 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:47.396827 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e39662-70b9-4b6e-919f-37af526fbde1 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:47.397347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e39662-70b9-4b6e-919f-37af526fbde1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:47.405173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e39662-70b9-4b6e-919f-37af526fbde1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:47.407079 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e39662-70b9-4b6e-919f-37af526fbde1 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:47.411843 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677ee7e-a85c-4ede-88f1-ac971a74d76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:47.412648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677ee7e-a85c-4ede-88f1-ac971a74d76b None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:47.412922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677ee7e-a85c-4ede-88f1-ac971a74d76b None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:47.413191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677ee7e-a85c-4ede-88f1-ac971a74d76b None admin] PATH_INFO: `/v3/users/nova` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:47.420985 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677ee7e-a85c-4ede-88f1-ac971a74d76b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:47.422487 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677ee7e-a85c-4ede-88f1-ac971a74d76b None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:47.423706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-42e39662-70b9-4b6e-919f-37af526fbde1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:47.423706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:47.423706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:47.423706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:47.423706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:47.423706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:47.423706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:47.423706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:47.423706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:47.423706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:47.423706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:47.423706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:47.423706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:47.423706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:47.423706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:47.424661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:47.425610 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:47.425610 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:47.425610 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:47.425610 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:47.425610 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:47.425610 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:47.425610 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:47.425610 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:47.428193 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 316/626] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:47.429394 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e39662-70b9-4b6e-919f-37af526fbde1 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-e677ee7e-a85c-4ede-88f1-ac971a74d76b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:47.438633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:47.439731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:47.440775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:47.440775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:47.440775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:47.440775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:47.440775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:47.440775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:47.440775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:47.440775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:47.440775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:04:47.440775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:47.445359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 311/627] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:04:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:47.446426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677ee7e-a85c-4ede-88f1-ac971a74d76b None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:47.550782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e39662-70b9-4b6e-919f-37af526fbde1 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:47.560670 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:47.587220 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677ee7e-a85c-4ede-88f1-ac971a74d76b None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:47.600221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2479ad4-c87f-4e60-aedc-2608b2246632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:47.601320 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2479ad4-c87f-4e60-aedc-2608b2246632 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:47.601837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2479ad4-c87f-4e60-aedc-2608b2246632 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:47.602379 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2479ad4-c87f-4e60-aedc-2608b2246632 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:47.613911 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2479ad4-c87f-4e60-aedc-2608b2246632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-d2479ad4-c87f-4e60-aedc-2608b2246632 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:47.622150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:47.623284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:47.624187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:47.624187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:47.624187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:47.624187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:47.624187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:47.624187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:47.624187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:47.624187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:47.624187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:47.624187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:47.624187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:47.624187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:47.626905 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 317/628] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:47.627926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2479ad4-c87f-4e60-aedc-2608b2246632 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:47.629170 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0caa2f4-c5e2-4e4e-8aa7-aeb11a6aa4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:47.629857 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0caa2f4-c5e2-4e4e-8aa7-aeb11a6aa4c9 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:47.630138 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0caa2f4-c5e2-4e4e-8aa7-aeb11a6aa4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:47.630441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0caa2f4-c5e2-4e4e-8aa7-aeb11a6aa4c9 None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:47.631241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0caa2f4-c5e2-4e4e-8aa7-aeb11a6aa4c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:47.631549 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0caa2f4-c5e2-4e4e-8aa7-aeb11a6aa4c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:47.633331 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0caa2f4-c5e2-4e4e-8aa7-aeb11a6aa4c9 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:47.650389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 312/629] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:47.651613 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0caa2f4-c5e2-4e4e-8aa7-aeb11a6aa4c9 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:47.683723 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:47.742068 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2479ad4-c87f-4e60-aedc-2608b2246632 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:47.783949 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0caa2f4-c5e2-4e4e-8aa7-aeb11a6aa4c9 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:47.784159 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d04ed5-d1ae-48ea-9116-240e1e0ed9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:47.784764 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d04ed5-d1ae-48ea-9116-240e1e0ed9ac None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:47.785054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d04ed5-d1ae-48ea-9116-240e1e0ed9ac None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:47.785387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d04ed5-d1ae-48ea-9116-240e1e0ed9ac None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:47.786252 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d04ed5-d1ae-48ea-9116-240e1e0ed9ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:47.786655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d04ed5-d1ae-48ea-9116-240e1e0ed9ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:47.789366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d04ed5-d1ae-48ea-9116-240e1e0ed9ac None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:47.814326 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 318/630] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:04:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:47.815366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d04ed5-d1ae-48ea-9116-240e1e0ed9ac None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:47.822862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcb1ca-3d38-44db-8909-fad9291b38af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:47.823851 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcb1ca-3d38-44db-8909-fad9291b38af None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:47.824351 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcb1ca-3d38-44db-8909-fad9291b38af None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:47.824856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcb1ca-3d38-44db-8909-fad9291b38af None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:47.825906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcb1ca-3d38-44db-8909-fad9291b38af None admin] RBAC: Adding query filter params (name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:47.826460 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcb1ca-3d38-44db-8909-fad9291b38af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:47.828076 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcb1ca-3d38-44db-8909-fad9291b38af None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:47.838699 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 313/631] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:47.839942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcb1ca-3d38-44db-8909-fad9291b38af None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:47.856617 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:47.953778 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d04ed5-d1ae-48ea-9116-240e1e0ed9ac None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:47.955762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:47.958085 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcb1ca-3d38-44db-8909-fad9291b38af None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:47.993262 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eedf4-1e04-4cae-a542-0f80f3bedef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:47.993913 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eedf4-1e04-4cae-a542-0f80f3bedef9 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:47.994190 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eedf4-1e04-4cae-a542-0f80f3bedef9 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:47.994546 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eedf4-1e04-4cae-a542-0f80f3bedef9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:47.995567 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7356a-a85f-43bc-a85c-3269fc425572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:47.996642 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7356a-a85f-43bc-a85c-3269fc425572 None admin] REQUEST_METHOD: `PUT` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:47.997196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7356a-a85f-43bc-a85c-3269fc425572 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:47.997623 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7356a-a85f-43bc-a85c-3269fc425572 None admin] PATH_INFO: `/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/87b30300be1a41128ff88e12e260f63b/roles/bdaf8732b34c43a38465efde7e18a060` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:48.014564 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7356a-a85f-43bc-a85c-3269fc425572 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba22fdc274b74521847e5fceb2d307b1, user_id=87b30300be1a41128ff88e12e260f63b, role_id=bdaf8732b34c43a38465efde7e18a060)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:48.016282 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7356a-a85f-43bc-a85c-3269fc425572 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:48.038525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 319/632] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:04:47 2022] PUT /identity/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/87b30300be1a41128ff88e12e260f63b/roles/bdaf8732b34c43a38465efde7e18a060 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:04:48.040602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7356a-a85f-43bc-a85c-3269fc425572 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:48.115920 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44eedf4-1e04-4cae-a542-0f80f3bedef9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:48.119789 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44eedf4-1e04-4cae-a542-0f80f3bedef9 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:48.146237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:48.280004 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7356a-a85f-43bc-a85c-3269fc425572 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:48.305700 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 314/633] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:48.311304 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eedf4-1e04-4cae-a542-0f80f3bedef9 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:48.322969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bacaab-b27d-4765-91fa-bbadd16b2669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:48.324240 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bacaab-b27d-4765-91fa-bbadd16b2669 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:48.324687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bacaab-b27d-4765-91fa-bbadd16b2669 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:48.325135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bacaab-b27d-4765-91fa-bbadd16b2669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:48.336655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bacaab-b27d-4765-91fa-bbadd16b2669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:48.338454 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bacaab-b27d-4765-91fa-bbadd16b2669 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:48.358073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-86bacaab-b27d-4765-91fa-bbadd16b2669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:48.358073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:48.358073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:48.358073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:48.358073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:48.358073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:48.358073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:48.358073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:48.358073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:48.358073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:48.358073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:48.358073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:48.358073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:48.358073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:48.358073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:48.359368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:48.360347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:48.360347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:48.360347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:48.360347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:48.360604 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 320/634] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:48.366102 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bacaab-b27d-4765-91fa-bbadd16b2669 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:48.373124 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:48.448862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eedf4-1e04-4cae-a542-0f80f3bedef9 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:48.465302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:48.493240 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bacaab-b27d-4765-91fa-bbadd16b2669 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:48.502999 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9216e-49fa-4fd7-b82e-1677912418e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:48.503815 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9216e-49fa-4fd7-b82e-1677912418e5 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:48.504101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9216e-49fa-4fd7-b82e-1677912418e5 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:48.504381 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9216e-49fa-4fd7-b82e-1677912418e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:48.518628 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9216e-49fa-4fd7-b82e-1677912418e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:48.520522 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9216e-49fa-4fd7-b82e-1677912418e5 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:48.537440 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff1e74-fb3d-4a10-b223-09f6b91138d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:48.538149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff1e74-fb3d-4a10-b223-09f6b91138d1 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:48.538478 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff1e74-fb3d-4a10-b223-09f6b91138d1 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:48.538758 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff1e74-fb3d-4a10-b223-09f6b91138d1 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:48.539504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff1e74-fb3d-4a10-b223-09f6b91138d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:48.539818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff1e74-fb3d-4a10-b223-09f6b91138d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:48.541315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff1e74-fb3d-4a10-b223-09f6b91138d1 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:48.548241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-ecd9216e-49fa-4fd7-b82e-1677912418e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:48.548241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:48.548241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:48.548241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:48.548241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:48.548241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:48.548241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:48.548241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:48.548241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:48.548241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:48.548241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:48.548241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:48.548241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:48.548241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:48.548241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:48.550230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:48.552171 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:48.552171 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:48.552171 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:48.552171 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:48.556124 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 315/635] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:48.558924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61acdb-0137-4d16-8ed9-5031a347f26c None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:48.559376 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61acdb-0137-4d16-8ed9-5031a347f26c None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:48.559825 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61acdb-0137-4d16-8ed9-5031a347f26c None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:48.561315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 321/636] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:48.561651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 316/637] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:48.570387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff1e74-fb3d-4a10-b223-09f6b91138d1 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:48.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61acdb-0137-4d16-8ed9-5031a347f26c None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:48.649439 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:48.685397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61acdb-0137-4d16-8ed9-5031a347f26c None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:48.689639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff1e74-fb3d-4a10-b223-09f6b91138d1 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:48.722064 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080f253-c4e1-49a5-be6e-c0bf01e25776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:48.722900 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080f253-c4e1-49a5-be6e-c0bf01e25776 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:48.723179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080f253-c4e1-49a5-be6e-c0bf01e25776 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:48.723457 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080f253-c4e1-49a5-be6e-c0bf01e25776 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:48.727398 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dc36d-1bf1-4098-b72c-51066283c957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:48.728071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dc36d-1bf1-4098-b72c-51066283c957 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:48.728355 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dc36d-1bf1-4098-b72c-51066283c957 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:48.728718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dc36d-1bf1-4098-b72c-51066283c957 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:48.729416 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dc36d-1bf1-4098-b72c-51066283c957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:48.729720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dc36d-1bf1-4098-b72c-51066283c957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:48.730986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080f253-c4e1-49a5-be6e-c0bf01e25776 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:48.731201 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dc36d-1bf1-4098-b72c-51066283c957 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:48.732434 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080f253-c4e1-49a5-be6e-c0bf01e25776 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:48.748002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 322/638] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:48.748002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9216c8e-6d84-4eeb-825f-1dbf08932aeb None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:48.748002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9216c8e-6d84-4eeb-825f-1dbf08932aeb None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:48.748002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9216c8e-6d84-4eeb-825f-1dbf08932aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-1080f253-c4e1-49a5-be6e-c0bf01e25776 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:48.748413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:48.749346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:48.750415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:48.750415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:48.750415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:48.750415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:48.750415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:48.750415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:48.750415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:48.750415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:48.750415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:04:48.750415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:48.750415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 317/639] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:48.750415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d6e53-fae2-472d-b45d-4961a797e676 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:48.750415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d6e53-fae2-472d-b45d-4961a797e676 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:48.751183 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d6e53-fae2-472d-b45d-4961a797e676 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:48.752086 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 318/640] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:48.754413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d6e53-fae2-472d-b45d-4961a797e676 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:48.771050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-e9216c8e-6d84-4eeb-825f-1dbf08932aeb None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:48.772506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-e9216c8e-6d84-4eeb-825f-1dbf08932aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:48.779186 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:48.845454 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 323/641] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:48.846582 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9216c8e-6d84-4eeb-825f-1dbf08932aeb None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:48.879060 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d6e53-fae2-472d-b45d-4961a797e676 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:48.914994 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:48.915202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382e2cf-279e-427e-8dc0-c43a24c6c3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:48.916018 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382e2cf-279e-427e-8dc0-c43a24c6c3fb None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:48.916347 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382e2cf-279e-427e-8dc0-c43a24c6c3fb None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:48.916788 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382e2cf-279e-427e-8dc0-c43a24c6c3fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:48.924591 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382e2cf-279e-427e-8dc0-c43a24c6c3fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:48.926074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382e2cf-279e-427e-8dc0-c43a24c6c3fb None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:48.941222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-d382e2cf-279e-427e-8dc0-c43a24c6c3fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:48.941222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:48.941222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:48.941222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:48.941222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:48.941222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:48.941222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:48.941222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:48.941222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:48.941222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:48.941222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:48.941222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:48.941222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:48.941222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:48.941222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:48.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:48.943233 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:48.943233 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:48.943233 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:48.943233 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:48.943233 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:48.943233 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:48.943233 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:48.943233 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:48.943233 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 319/642] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:48.944544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4fd81-4bc8-4013-87a1-a945832a8577 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:48.944544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4fd81-4bc8-4013-87a1-a945832a8577 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:48.944544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4fd81-4bc8-4013-87a1-a945832a8577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:48.964457 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-91e4fd81-4bc8-4013-87a1-a945832a8577 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:48.966111 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-91e4fd81-4bc8-4013-87a1-a945832a8577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:48.968576 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9216c8e-6d84-4eeb-825f-1dbf08932aeb None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:49.005743 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084712e-9155-4e67-ad18-de9936ec60b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:49.007116 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084712e-9155-4e67-ad18-de9936ec60b6 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:49.007116 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084712e-9155-4e67-ad18-de9936ec60b6 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:49.007116 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084712e-9155-4e67-ad18-de9936ec60b6 None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:49.007845 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f084712e-9155-4e67-ad18-de9936ec60b6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:49.008161 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f084712e-9155-4e67-ad18-de9936ec60b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:49.009596 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f084712e-9155-4e67-ad18-de9936ec60b6 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:49.033131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 324/643] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 20:04:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:49.033131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084712e-9155-4e67-ad18-de9936ec60b6 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:49.044883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 320/644] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:49.046436 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4fd81-4bc8-4013-87a1-a945832a8577 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:49.057564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:49.150362 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084712e-9155-4e67-ad18-de9936ec60b6 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:49.155275 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4fd81-4bc8-4013-87a1-a945832a8577 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:49.165553 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:49.189791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a8b73-0b99-4d25-915e-578625ed3ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:49.190676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a8b73-0b99-4d25-915e-578625ed3ce2 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:49.191006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a8b73-0b99-4d25-915e-578625ed3ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:49.191280 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a8b73-0b99-4d25-915e-578625ed3ce2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:49.191544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61684d-bdbb-4858-ad7f-6393a2cf2eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:49.192210 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61684d-bdbb-4858-ad7f-6393a2cf2eb4 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:49.192491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61684d-bdbb-4858-ad7f-6393a2cf2eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:49.192764 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61684d-bdbb-4858-ad7f-6393a2cf2eb4 None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:49.193546 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61684d-bdbb-4858-ad7f-6393a2cf2eb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:49.194137 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61684d-bdbb-4858-ad7f-6393a2cf2eb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:49.195348 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61684d-bdbb-4858-ad7f-6393a2cf2eb4 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:49.202989 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a8b73-0b99-4d25-915e-578625ed3ce2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:49.212304 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 321/645] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:49.212304 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61684d-bdbb-4858-ad7f-6393a2cf2eb4 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-4e5a8b73-0b99-4d25-915e-578625ed3ce2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:49.212575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:49.213579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:49.214540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:49.214540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:49.214540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:49.214540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:49.214540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:49.214540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:49.214540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:49.214540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:49.214540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:49.214540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:49.214540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:49.214540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:49.214540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 325/646] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:49.214540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a8b73-0b99-4d25-915e-578625ed3ce2 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:49.294075 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:49.330359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a8b73-0b99-4d25-915e-578625ed3ce2 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:49.331443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61684d-bdbb-4858-ad7f-6393a2cf2eb4 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:49.367221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82608f7a-2869-4377-925e-9c683f10e04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:49.368032 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82608f7a-2869-4377-925e-9c683f10e04d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:49.368166 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee266d-0758-4b3e-94e6-31ff50e2a28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:49.368300 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82608f7a-2869-4377-925e-9c683f10e04d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:49.368578 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82608f7a-2869-4377-925e-9c683f10e04d None admin] PATH_INFO: `/v3/roles/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:49.368808 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee266d-0758-4b3e-94e6-31ff50e2a28d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:49.369091 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee266d-0758-4b3e-94e6-31ff50e2a28d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:49.369520 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee266d-0758-4b3e-94e6-31ff50e2a28d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:49.379982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82608f7a-2869-4377-925e-9c683f10e04d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-82608f7a-2869-4377-925e-9c683f10e04d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:49.387779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:49.388763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:49.389671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:49.389671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:49.389671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:49.389671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:49.389671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:49.389671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:49.389671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:49.389671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:49.389671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:49.389671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:49.389671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:49.389671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:49.389671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 326/647] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:49.390472 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82608f7a-2869-4377-925e-9c683f10e04d None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:49.424366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:49.453237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee266d-0758-4b3e-94e6-31ff50e2a28d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:49.454751 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee266d-0758-4b3e-94e6-31ff50e2a28d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:49.500364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82608f7a-2869-4377-925e-9c683f10e04d None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:49.525419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:49.534783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353dc18b-758a-4738-819c-360f5a8e5fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:49.535454 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dc18b-758a-4738-819c-360f5a8e5fc7 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:49.535854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dc18b-758a-4738-819c-360f5a8e5fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:49.536139 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dc18b-758a-4738-819c-360f5a8e5fc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:49.536985 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353dc18b-758a-4738-819c-360f5a8e5fc7 None admin] RBAC: Adding query filter params (role.id=bdaf8732b34c43a38465efde7e18a060, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=378b046698c04b7b88194a9ffe5b045a) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:49.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353dc18b-758a-4738-819c-360f5a8e5fc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:49.538771 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353dc18b-758a-4738-819c-360f5a8e5fc7 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:49.546517 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 327/648] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:04:49 2022] GET /identity/v3/role_assignments?user.id=378b046698c04b7b88194a9ffe5b045a&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=bdaf8732b34c43a38465efde7e18a060 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:49.547554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353dc18b-758a-4738-819c-360f5a8e5fc7 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:49.581931 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 322/649] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:49.581931 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee266d-0758-4b3e-94e6-31ff50e2a28d None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:49.712294 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:49.723687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353dc18b-758a-4738-819c-360f5a8e5fc7 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:49.733359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee266d-0758-4b3e-94e6-31ff50e2a28d None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:49.762207 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5c1b6-ebb2-4942-8795-6b037d699829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:49.763329 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5c1b6-ebb2-4942-8795-6b037d699829 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:49.763820 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5c1b6-ebb2-4942-8795-6b037d699829 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:49.764301 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5c1b6-ebb2-4942-8795-6b037d699829 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:49.765322 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5c1b6-ebb2-4942-8795-6b037d699829 None admin] RBAC: Adding query filter params (name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:49.765839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5c1b6-ebb2-4942-8795-6b037d699829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:49.774467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5c1b6-ebb2-4942-8795-6b037d699829 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:49.781177 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2905f1-3754-4aae-928d-7f60dd88648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:49.781177 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2905f1-3754-4aae-928d-7f60dd88648c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:49.781177 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2905f1-3754-4aae-928d-7f60dd88648c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:49.781177 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2905f1-3754-4aae-928d-7f60dd88648c None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:49.781567 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2905f1-3754-4aae-928d-7f60dd88648c None admin] RBAC: Adding query filter params (name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:49.781567 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2905f1-3754-4aae-928d-7f60dd88648c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:49.782371 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2905f1-3754-4aae-928d-7f60dd88648c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:49.789351 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 328/650] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:49.790574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5c1b6-ebb2-4942-8795-6b037d699829 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:49.792510 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 323/651] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:49.798863 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2905f1-3754-4aae-928d-7f60dd88648c None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:49.833759 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:49.923784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5c1b6-ebb2-4942-8795-6b037d699829 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:49.940834 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2905f1-3754-4aae-928d-7f60dd88648c None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:49.973193 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f6cfd-9fa2-4854-b773-8f780d0ad551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:49.973929 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f6cfd-9fa2-4854-b773-8f780d0ad551 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:49.974210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f6cfd-9fa2-4854-b773-8f780d0ad551 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:49.974517 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f6cfd-9fa2-4854-b773-8f780d0ad551 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:49.977558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09741bb-699f-4012-8f41-65c026f7d56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:49.978646 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09741bb-699f-4012-8f41-65c026f7d56e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:49.979142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09741bb-699f-4012-8f41-65c026f7d56e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:49.979653 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09741bb-699f-4012-8f41-65c026f7d56e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:49.986944 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f6cfd-9fa2-4854-b773-8f780d0ad551 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:49.988934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f6cfd-9fa2-4854-b773-8f780d0ad551 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:49.993773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09741bb-699f-4012-8f41-65c026f7d56e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:49.995392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09741bb-699f-4012-8f41-65c026f7d56e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:50.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-855f6cfd-9fa2-4854-b773-8f780d0ad551 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:50.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:50.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:50.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:50.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:50.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:50.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:50.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:50.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:50.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:50.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:50.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:50.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:50.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:50.011898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:50.012907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:50.013877 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:50.013877 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:50.013877 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:50.013877 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:50.014123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-a09741bb-699f-4012-8f41-65c026f7d56e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:50.014123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:50.014123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:50.014123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:50.014123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:50.014123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:50.014123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:50.014123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:50.014123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:50.014123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:50.014123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:50.014123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:50.014123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:50.014123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:50.014123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:50.015029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:50.015977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:50.015977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:50.015977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:50.015977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:50.015977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 324/652] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:50.015977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09741bb-699f-4012-8f41-65c026f7d56e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:50.018931 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 329/653] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:50.024376 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f6cfd-9fa2-4854-b773-8f780d0ad551 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:50.042611 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:50.119539 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:50.141995 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09741bb-699f-4012-8f41-65c026f7d56e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:50.149617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f6cfd-9fa2-4854-b773-8f780d0ad551 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:50.186853 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2da49f-9d1c-422a-ad65-551831229188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:50.187227 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6d143-02a1-45e2-818b-7c05571a99f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:50.188022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6d143-02a1-45e2-818b-7c05571a99f6 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:50.188372 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6d143-02a1-45e2-818b-7c05571a99f6 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:50.188650 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2da49f-9d1c-422a-ad65-551831229188 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:50.188770 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6d143-02a1-45e2-818b-7c05571a99f6 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:50.189210 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2da49f-9d1c-422a-ad65-551831229188 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:50.189564 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b6d143-02a1-45e2-818b-7c05571a99f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:50.189857 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2da49f-9d1c-422a-ad65-551831229188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:50.189964 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b6d143-02a1-45e2-818b-7c05571a99f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:50.191407 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b6d143-02a1-45e2-818b-7c05571a99f6 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:50.203581 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2da49f-9d1c-422a-ad65-551831229188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:50.204657 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 330/654] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:50.205695 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2da49f-9d1c-422a-ad65-551831229188 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:50.205814 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6d143-02a1-45e2-818b-7c05571a99f6 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:50.231004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-0f2da49f-9d1c-422a-ad65-551831229188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:50.231004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:50.231004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:50.231004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:50.231004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:50.231004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:50.231004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:50.231004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:50.231004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:50.231004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:50.231004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:50.231004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:50.231004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:50.231004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:50.231004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:50.232003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:50.233054 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:50.233054 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:50.233054 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:50.233054 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:50.235587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 325/655] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:50.236759 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2da49f-9d1c-422a-ad65-551831229188 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:50.297824 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:50.330249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6d143-02a1-45e2-818b-7c05571a99f6 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:50.361138 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2da49f-9d1c-422a-ad65-551831229188 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:50.367594 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1672e-181b-41fa-8bb2-e2a3b33220b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:50.368475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1672e-181b-41fa-8bb2-e2a3b33220b6 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:50.368909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1672e-181b-41fa-8bb2-e2a3b33220b6 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:50.369388 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1672e-181b-41fa-8bb2-e2a3b33220b6 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:50.370384 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1672e-181b-41fa-8bb2-e2a3b33220b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:50.370903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1672e-181b-41fa-8bb2-e2a3b33220b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:50.372645 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1672e-181b-41fa-8bb2-e2a3b33220b6 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:50.389565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 331/656] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:50.390828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1672e-181b-41fa-8bb2-e2a3b33220b6 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:50.398842 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff38220-ea59-4b9e-acf4-0d3e35985c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:50.399654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff38220-ea59-4b9e-acf4-0d3e35985c5c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:50.399934 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff38220-ea59-4b9e-acf4-0d3e35985c5c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:50.400210 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff38220-ea59-4b9e-acf4-0d3e35985c5c None admin] PATH_INFO: `/v3/users/glance` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:50.403381 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:50.410507 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff38220-ea59-4b9e-acf4-0d3e35985c5c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:50.412540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff38220-ea59-4b9e-acf4-0d3e35985c5c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-1ff38220-ea59-4b9e-acf4-0d3e35985c5c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:50.428755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:50.430015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:50.431286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:50.431286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:50.431286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:50.431286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:50.431286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:50.431286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:50.431286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:50.431286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:50.431286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:04:50.431286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:50.435104 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 326/657] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:50.436797 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff38220-ea59-4b9e-acf4-0d3e35985c5c None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:50.500181 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1672e-181b-41fa-8bb2-e2a3b33220b6 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:50.531773 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:50.538363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec3642-d0ee-4b13-bdf7-de4786bdbff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:50.539047 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec3642-d0ee-4b13-bdf7-de4786bdbff5 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:50.539341 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec3642-d0ee-4b13-bdf7-de4786bdbff5 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:50.539607 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec3642-d0ee-4b13-bdf7-de4786bdbff5 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:50.540343 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec3642-d0ee-4b13-bdf7-de4786bdbff5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:50.540650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec3642-d0ee-4b13-bdf7-de4786bdbff5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:50.542119 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec3642-d0ee-4b13-bdf7-de4786bdbff5 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:50.553256 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff38220-ea59-4b9e-acf4-0d3e35985c5c None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:50.560299 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 332/658] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:50.561808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0b4aa-236c-4c1f-bddc-b59bed4c05d8 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:50.562162 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0b4aa-236c-4c1f-bddc-b59bed4c05d8 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:50.562518 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0b4aa-236c-4c1f-bddc-b59bed4c05d8 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:50.566614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 333/659] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:50.566614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0b4aa-236c-4c1f-bddc-b59bed4c05d8 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:50.594489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249e946-9dd3-4e22-a22c-480b1ee6c1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:50.595319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249e946-9dd3-4e22-a22c-480b1ee6c1f0 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:50.595596 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249e946-9dd3-4e22-a22c-480b1ee6c1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:50.595868 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249e946-9dd3-4e22-a22c-480b1ee6c1f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:50.602823 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7249e946-9dd3-4e22-a22c-480b1ee6c1f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:50.604316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7249e946-9dd3-4e22-a22c-480b1ee6c1f0 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:50.619072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-7249e946-9dd3-4e22-a22c-480b1ee6c1f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:50.619072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:50.619072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:50.619072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:50.619072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:50.619072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:50.619072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:50.619072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:50.619072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:50.619072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:50.619072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:50.619072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:50.619072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:50.619072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:50.619072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:50.620089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:50.621032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:50.621032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:50.621032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:50.621032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:50.621032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:50.621032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:50.621032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:50.621032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:50.621032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 327/660] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:50.622404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249e946-9dd3-4e22-a22c-480b1ee6c1f0 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:50.682550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0b4aa-236c-4c1f-bddc-b59bed4c05d8 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:50.684430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:50.718042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57c1e1-05a4-47ad-b24f-eb5ad58cb901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:50.718872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57c1e1-05a4-47ad-b24f-eb5ad58cb901 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:50.719557 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57c1e1-05a4-47ad-b24f-eb5ad58cb901 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:50.719958 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57c1e1-05a4-47ad-b24f-eb5ad58cb901 None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:50.720885 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57c1e1-05a4-47ad-b24f-eb5ad58cb901 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:50.721280 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57c1e1-05a4-47ad-b24f-eb5ad58cb901 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:50.722803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57c1e1-05a4-47ad-b24f-eb5ad58cb901 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:50.733105 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249e946-9dd3-4e22-a22c-480b1ee6c1f0 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:50.740761 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 334/661] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 20:04:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:50.742018 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7ba20-805e-47e9-9689-1e96e19dd0d2 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:50.742359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7ba20-805e-47e9-9689-1e96e19dd0d2 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:50.742694 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7ba20-805e-47e9-9689-1e96e19dd0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:50.759268 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-43f7ba20-805e-47e9-9689-1e96e19dd0d2 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:50.760523 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-43f7ba20-805e-47e9-9689-1e96e19dd0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:50.774520 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea6b24-4ff0-4fa7-ba80-976b41369cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:50.775324 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea6b24-4ff0-4fa7-ba80-976b41369cab None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:50.775684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea6b24-4ff0-4fa7-ba80-976b41369cab None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:50.775960 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea6b24-4ff0-4fa7-ba80-976b41369cab None admin] PATH_INFO: `/v3/users/neutron` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:50.783190 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea6b24-4ff0-4fa7-ba80-976b41369cab None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:50.784636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea6b24-4ff0-4fa7-ba80-976b41369cab None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-56ea6b24-4ff0-4fa7-ba80-976b41369cab None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:50.799082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:50.800115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:50.806043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:50.806043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:50.806043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:50.806043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:50.806043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:50.806043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:50.806043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:50.806043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:50.806043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:04:50.806043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:50.806043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 328/662] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:50.806043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea6b24-4ff0-4fa7-ba80-976b41369cab None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:50.811987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:50.838393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 335/663] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:50.839446 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7ba20-805e-47e9-9689-1e96e19dd0d2 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:50.914785 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea6b24-4ff0-4fa7-ba80-976b41369cab None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:50.934440 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:50.953898 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5923fe-906d-47e0-b566-534a91d199fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:50.954606 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5923fe-906d-47e0-b566-534a91d199fd None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:50.954890 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5923fe-906d-47e0-b566-534a91d199fd None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:50.955169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5923fe-906d-47e0-b566-534a91d199fd None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:50.955957 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5923fe-906d-47e0-b566-534a91d199fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:50.956261 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5923fe-906d-47e0-b566-534a91d199fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:50.957665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7ba20-805e-47e9-9689-1e96e19dd0d2 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:50.957779 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5923fe-906d-47e0-b566-534a91d199fd None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:50.975669 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 329/664] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:50.975669 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5923fe-906d-47e0-b566-534a91d199fd None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:51.002128 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b1bda-ebc6-45f1-9cd3-a0d04ca811ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:51.003352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b1bda-ebc6-45f1-9cd3-a0d04ca811ca None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:51.003789 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b1bda-ebc6-45f1-9cd3-a0d04ca811ca None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:51.004339 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b1bda-ebc6-45f1-9cd3-a0d04ca811ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:51.055933 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:51.079892 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b1bda-ebc6-45f1-9cd3-a0d04ca811ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:51.081371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b1bda-ebc6-45f1-9cd3-a0d04ca811ca None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:51.081480 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5923fe-906d-47e0-b566-534a91d199fd None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:51.116271 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b381a-9ae5-4dda-92a3-04518f5743b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:51.116927 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b381a-9ae5-4dda-92a3-04518f5743b6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:51.117243 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b381a-9ae5-4dda-92a3-04518f5743b6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:51.117471 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b381a-9ae5-4dda-92a3-04518f5743b6 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:51.118334 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b381a-9ae5-4dda-92a3-04518f5743b6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:51.118650 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b381a-9ae5-4dda-92a3-04518f5743b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:51.120324 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b381a-9ae5-4dda-92a3-04518f5743b6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:51.142103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 330/665] 10.222.0.84 () {56 vars in 1139 bytes} [Thu May 19 20:04:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:51.142103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b381a-9ae5-4dda-92a3-04518f5743b6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:51.173996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:51.202018 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 336/666] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:51.203345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b1bda-ebc6-45f1-9cd3-a0d04ca811ca None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:51.243945 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b381a-9ae5-4dda-92a3-04518f5743b6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:51.278393 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cf139-cd18-4661-ba27-9ad82c029dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:51.279508 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cf139-cd18-4661-ba27-9ad82c029dc4 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:51.279907 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cf139-cd18-4661-ba27-9ad82c029dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:51.280250 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cf139-cd18-4661-ba27-9ad82c029dc4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:51.286625 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:51.291690 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741cf139-cd18-4661-ba27-9ad82c029dc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-741cf139-cd18-4661-ba27-9ad82c029dc4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:51.299828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:51.301969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:51.304227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:51.304227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:51.304227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:51.304227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:51.304227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:51.304227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:51.304227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:51.304227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:51.304227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:51.304227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:51.304227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:51.304227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:51.304227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 331/667] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:51.304227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cf139-cd18-4661-ba27-9ad82c029dc4 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:51.318589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b1bda-ebc6-45f1-9cd3-a0d04ca811ca None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:51.357264 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847d552-2068-49ff-b1ab-dfcfd2b5f3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:51.358041 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847d552-2068-49ff-b1ab-dfcfd2b5f3b7 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:51.358416 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847d552-2068-49ff-b1ab-dfcfd2b5f3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:51.358756 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847d552-2068-49ff-b1ab-dfcfd2b5f3b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:51.359664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847d552-2068-49ff-b1ab-dfcfd2b5f3b7 None admin] RBAC: Adding query filter params (role.id=bdaf8732b34c43a38465efde7e18a060, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=c97675d166b84a88aed79d26499a0ed7) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:51.360356 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847d552-2068-49ff-b1ab-dfcfd2b5f3b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:51.361471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847d552-2068-49ff-b1ab-dfcfd2b5f3b7 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:51.374294 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 337/668] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:04:51 2022] GET /identity/v3/role_assignments?user.id=c97675d166b84a88aed79d26499a0ed7&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=bdaf8732b34c43a38465efde7e18a060 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:51.374294 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847d552-2068-49ff-b1ab-dfcfd2b5f3b7 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:51.427390 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:51.437517 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cf139-cd18-4661-ba27-9ad82c029dc4 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:51.486029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60430008-c254-4ebe-96b9-390296137659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:51.486912 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60430008-c254-4ebe-96b9-390296137659 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:51.487208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60430008-c254-4ebe-96b9-390296137659 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:51.487481 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60430008-c254-4ebe-96b9-390296137659 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:51.496369 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847d552-2068-49ff-b1ab-dfcfd2b5f3b7 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:51.549320 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c11cf-ff44-448b-9e2f-1f883a8b4660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:51.550464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c11cf-ff44-448b-9e2f-1f883a8b4660 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:51.550846 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c11cf-ff44-448b-9e2f-1f883a8b4660 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:51.551286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c11cf-ff44-448b-9e2f-1f883a8b4660 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:51.563678 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c11cf-ff44-448b-9e2f-1f883a8b4660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:51.565699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c11cf-ff44-448b-9e2f-1f883a8b4660 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:51.568287 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:51.580048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60430008-c254-4ebe-96b9-390296137659 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:51.581814 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60430008-c254-4ebe-96b9-390296137659 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:51.589839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-627c11cf-ff44-448b-9e2f-1f883a8b4660 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:51.589839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:51.589839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:51.589839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:51.589839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:51.589839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:51.589839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:51.589839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:51.589839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:51.589839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:51.589839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:51.589839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:51.589839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:51.589839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:51.589839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:51.591022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:51.592086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:51.592086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:51.592086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:51.592086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:51.593193 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 338/669] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:51.594081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c11cf-ff44-448b-9e2f-1f883a8b4660 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:51.706693 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:51.718322 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 332/670] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:51.718322 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60430008-c254-4ebe-96b9-390296137659 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:51.731979 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c11cf-ff44-448b-9e2f-1f883a8b4660 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:51.776575 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e06de9-c160-4f86-ba34-def69f6d1aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:51.777264 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e06de9-c160-4f86-ba34-def69f6d1aae None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:51.777691 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e06de9-c160-4f86-ba34-def69f6d1aae None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:51.777973 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e06de9-c160-4f86-ba34-def69f6d1aae None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:51.779107 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e06de9-c160-4f86-ba34-def69f6d1aae None admin] RBAC: Adding query filter params (name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:51.779421 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e06de9-c160-4f86-ba34-def69f6d1aae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:51.781267 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e06de9-c160-4f86-ba34-def69f6d1aae None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:51.802534 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 339/671] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:51.803694 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e06de9-c160-4f86-ba34-def69f6d1aae None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:51.836012 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60430008-c254-4ebe-96b9-390296137659 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:51.862626 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:51.880629 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c18d9d-9b17-4d1b-a8a3-b0f7a9256dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:51.881327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c18d9d-9b17-4d1b-a8a3-b0f7a9256dcd None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:51.881619 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c18d9d-9b17-4d1b-a8a3-b0f7a9256dcd None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:51.881903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c18d9d-9b17-4d1b-a8a3-b0f7a9256dcd None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:51.882681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c18d9d-9b17-4d1b-a8a3-b0f7a9256dcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:51.882997 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c18d9d-9b17-4d1b-a8a3-b0f7a9256dcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:51.884446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c18d9d-9b17-4d1b-a8a3-b0f7a9256dcd None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:51.907119 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 333/672] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:51.908161 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c18d9d-9b17-4d1b-a8a3-b0f7a9256dcd None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:51.921117 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e06de9-c160-4f86-ba34-def69f6d1aae None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:51.958159 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277553f5-90aa-4627-9068-6fe539c806bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:51.959252 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277553f5-90aa-4627-9068-6fe539c806bc None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:51.959752 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277553f5-90aa-4627-9068-6fe539c806bc None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:51.960286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277553f5-90aa-4627-9068-6fe539c806bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:51.973249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277553f5-90aa-4627-9068-6fe539c806bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:51.974973 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277553f5-90aa-4627-9068-6fe539c806bc None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:51.980519 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:51.994795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-277553f5-90aa-4627-9068-6fe539c806bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:51.994795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:51.994795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:51.994795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:51.994795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:51.994795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:51.994795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:51.994795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:51.994795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:51.994795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:51.994795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:51.994795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:51.994795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:51.994795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:51.994795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:51.996013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:51.997190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:51.997190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:51.997190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:51.997190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:52.003578 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 340/673] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:52.004738 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277553f5-90aa-4627-9068-6fe539c806bc None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:52.027138 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c18d9d-9b17-4d1b-a8a3-b0f7a9256dcd None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:52.062543 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5fffc-b3e5-47d7-a837-655fa28c77ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:52.063383 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5fffc-b3e5-47d7-a837-655fa28c77ca None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:52.063661 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5fffc-b3e5-47d7-a837-655fa28c77ca None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:52.063934 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5fffc-b3e5-47d7-a837-655fa28c77ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:52.075920 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5fffc-b3e5-47d7-a837-655fa28c77ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:52.077396 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5fffc-b3e5-47d7-a837-655fa28c77ca None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:52.095654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-e7b5fffc-b3e5-47d7-a837-655fa28c77ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:52.095654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:52.095654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:52.095654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:52.095654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:52.095654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:52.095654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:52.095654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:52.095654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:52.095654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:52.095654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:52.095654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:52.095654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:52.095654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:52.095654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:52.096636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:52.097697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:52.097697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:52.097697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:52.097697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:52.097697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 334/674] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:52.098007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5fffc-b3e5-47d7-a837-655fa28c77ca None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:52.121748 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277553f5-90aa-4627-9068-6fe539c806bc None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:52.130861 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:52.159653 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cc34d-f59a-4a5f-94a6-ea770f067f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:52.160460 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cc34d-f59a-4a5f-94a6-ea770f067f34 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:52.160740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cc34d-f59a-4a5f-94a6-ea770f067f34 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:52.161010 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cc34d-f59a-4a5f-94a6-ea770f067f34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:52.168024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cc34d-f59a-4a5f-94a6-ea770f067f34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:52.169490 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cc34d-f59a-4a5f-94a6-ea770f067f34 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:52.184089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-ba8cc34d-f59a-4a5f-94a6-ea770f067f34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:52.184089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:52.184089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:52.184089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:52.184089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:52.184089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:52.184089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:52.184089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:52.184089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:52.184089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:52.184089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:52.184089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:52.184089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:52.184089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:52.184089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:52.185166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:52.186366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:52.186366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:52.186366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:52.186366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:52.186366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:52.186366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:52.186366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:52.186366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:52.189898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 341/675] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:52.191183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2963b-4557-48ca-a433-4e69fa73e21d None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:52.191472 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2963b-4557-48ca-a433-4e69fa73e21d None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:52.191752 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2963b-4557-48ca-a433-4e69fa73e21d None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:52.193168 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 342/676] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:52.193711 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2963b-4557-48ca-a433-4e69fa73e21d None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:52.216760 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5fffc-b3e5-47d7-a837-655fa28c77ca None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:52.224904 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:52.255101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd73b65-5586-437f-acbe-d8662dcd840a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:52.255922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd73b65-5586-437f-acbe-d8662dcd840a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:52.256204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd73b65-5586-437f-acbe-d8662dcd840a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:52.256475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd73b65-5586-437f-acbe-d8662dcd840a None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:52.257200 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd73b65-5586-437f-acbe-d8662dcd840a None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:52.257503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd73b65-5586-437f-acbe-d8662dcd840a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:52.258916 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd73b65-5586-437f-acbe-d8662dcd840a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:52.273450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 335/677] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:52.274304 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd73b65-5586-437f-acbe-d8662dcd840a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:52.307419 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2963b-4557-48ca-a433-4e69fa73e21d None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:52.345096 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10531d16-c272-48b7-88f3-0cf6eaf09cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:52.346088 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10531d16-c272-48b7-88f3-0cf6eaf09cf4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:52.346343 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10531d16-c272-48b7-88f3-0cf6eaf09cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:52.346680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10531d16-c272-48b7-88f3-0cf6eaf09cf4 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:52.347415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10531d16-c272-48b7-88f3-0cf6eaf09cf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:52.347886 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10531d16-c272-48b7-88f3-0cf6eaf09cf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:52.349739 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10531d16-c272-48b7-88f3-0cf6eaf09cf4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:52.361403 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 343/678] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:52.362642 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc68993-a358-4ca7-b8ba-bc0a7c62ed5b None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:52.362930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc68993-a358-4ca7-b8ba-bc0a7c62ed5b None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:52.363205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc68993-a358-4ca7-b8ba-bc0a7c62ed5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:52.380876 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-ebc68993-a358-4ca7-b8ba-bc0a7c62ed5b None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:52.382148 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-ebc68993-a358-4ca7-b8ba-bc0a7c62ed5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:52.387165 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd73b65-5586-437f-acbe-d8662dcd840a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:52.406995 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:52.421368 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567b24f-0531-48ce-9f9e-eb25164bc87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:52.422051 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567b24f-0531-48ce-9f9e-eb25164bc87e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:52.422373 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567b24f-0531-48ce-9f9e-eb25164bc87e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:52.422640 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567b24f-0531-48ce-9f9e-eb25164bc87e None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:52.423429 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567b24f-0531-48ce-9f9e-eb25164bc87e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:52.423739 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567b24f-0531-48ce-9f9e-eb25164bc87e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:52.425193 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567b24f-0531-48ce-9f9e-eb25164bc87e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:52.438164 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 336/679] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:52.438833 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567b24f-0531-48ce-9f9e-eb25164bc87e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:52.466428 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 344/680] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:52.466428 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc68993-a358-4ca7-b8ba-bc0a7c62ed5b None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:52.471798 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:52.550048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567b24f-0531-48ce-9f9e-eb25164bc87e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:52.588953 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7f956-bfad-491d-a371-48ef113fd322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:52.589780 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7f956-bfad-491d-a371-48ef113fd322 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:52.590062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7f956-bfad-491d-a371-48ef113fd322 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:52.590233 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc68993-a358-4ca7-b8ba-bc0a7c62ed5b None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:52.590372 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7f956-bfad-491d-a371-48ef113fd322 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:52.598826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7f956-bfad-491d-a371-48ef113fd322 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:52.600222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7f956-bfad-491d-a371-48ef113fd322 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:52.615771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-1cb7f956-bfad-491d-a371-48ef113fd322 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:52.615771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:52.615771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:52.615771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:52.615771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:52.615771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:52.615771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:52.615771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:52.615771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:52.615771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:52.615771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:52.615771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:52.615771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:52.615771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:52.615771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:52.616742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:52.617724 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:52.617724 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:52.617724 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:52.617724 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:52.617724 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:52.617724 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:52.617724 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:52.617724 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:52.617724 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 337/681] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:52.618242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7f956-bfad-491d-a371-48ef113fd322 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:52.633539 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f48328-6072-403f-bb78-b8f0bff70358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:52.634404 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f48328-6072-403f-bb78-b8f0bff70358 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:52.634668 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f48328-6072-403f-bb78-b8f0bff70358 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:52.634997 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f48328-6072-403f-bb78-b8f0bff70358 None admin] PATH_INFO: `/v3/users/swift` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:52.642704 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f48328-6072-403f-bb78-b8f0bff70358 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:52.644183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f48328-6072-403f-bb78-b8f0bff70358 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-81f48328-6072-403f-bb78-b8f0bff70358 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:52.659259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:52.660279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:52.666614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:52.666614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:52.666614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:52.666614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:52.666614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:52.666614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:52.666614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:52.666614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:52.666614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:04:52.666614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:52.666614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 345/682] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:04:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:52.666614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f48328-6072-403f-bb78-b8f0bff70358 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:52.688190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:52.710416 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:52.727719 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7f956-bfad-491d-a371-48ef113fd322 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:52.762962 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b760cd-5b3a-419b-a02f-1c9703f24b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:52.763628 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b760cd-5b3a-419b-a02f-1c9703f24b3e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:52.763905 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b760cd-5b3a-419b-a02f-1c9703f24b3e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:52.764174 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b760cd-5b3a-419b-a02f-1c9703f24b3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:52.765001 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b760cd-5b3a-419b-a02f-1c9703f24b3e None admin] RBAC: Adding query filter params (role.id=bdaf8732b34c43a38465efde7e18a060, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=be8df5550506438eb334c649893eb4dd) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:52.765310 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b760cd-5b3a-419b-a02f-1c9703f24b3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:52.766787 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b760cd-5b3a-419b-a02f-1c9703f24b3e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:52.782302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 338/683] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:04:52 2022] GET /identity/v3/role_assignments?user.id=be8df5550506438eb334c649893eb4dd&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=bdaf8732b34c43a38465efde7e18a060 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:52.782302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b760cd-5b3a-419b-a02f-1c9703f24b3e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:52.785568 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f48328-6072-403f-bb78-b8f0bff70358 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:52.838295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4d4b6-2c7b-4ea8-bdf6-c7baf56160f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:52.839253 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4d4b6-2c7b-4ea8-bdf6-c7baf56160f0 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:52.839696 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4d4b6-2c7b-4ea8-bdf6-c7baf56160f0 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:52.840073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4d4b6-2c7b-4ea8-bdf6-c7baf56160f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:52.853853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4d4b6-2c7b-4ea8-bdf6-c7baf56160f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-3ec4d4b6-2c7b-4ea8-bdf6-c7baf56160f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:52.862948 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:52.864215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:52.865114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:52.865114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:52.865114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:52.865114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:52.865114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:52.865114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:52.865114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:52.865114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:52.865114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:52.865114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:52.865114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:04:52.865114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:52.865114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 346/684] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:04:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:52.865991 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4d4b6-2c7b-4ea8-bdf6-c7baf56160f0 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:52.908739 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b760cd-5b3a-419b-a02f-1c9703f24b3e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:52.945919 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40c22d-dce8-4b08-b405-926c85f68e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:52.946621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40c22d-dce8-4b08-b405-926c85f68e49 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:52.946899 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40c22d-dce8-4b08-b405-926c85f68e49 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:52.947168 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40c22d-dce8-4b08-b405-926c85f68e49 None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:52.948005 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40c22d-dce8-4b08-b405-926c85f68e49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:52.948487 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40c22d-dce8-4b08-b405-926c85f68e49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:52.950547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40c22d-dce8-4b08-b405-926c85f68e49 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:52.963603 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:52.970421 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 339/685] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:52.971576 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40c22d-dce8-4b08-b405-926c85f68e49 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:52.986870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:52.999114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4d4b6-2c7b-4ea8-bdf6-c7baf56160f0 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:53.042676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949cdd4-4a9b-4a40-ba91-857fa3900a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:53.043471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949cdd4-4a9b-4a40-ba91-857fa3900a66 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:53.043823 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949cdd4-4a9b-4a40-ba91-857fa3900a66 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:53.044171 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949cdd4-4a9b-4a40-ba91-857fa3900a66 None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:53.045785 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949cdd4-4a9b-4a40-ba91-857fa3900a66 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:53.045785 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949cdd4-4a9b-4a40-ba91-857fa3900a66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:53.047013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949cdd4-4a9b-4a40-ba91-857fa3900a66 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:53.066585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 347/686] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 20:04:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:53.068029 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949cdd4-4a9b-4a40-ba91-857fa3900a66 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:53.104252 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40c22d-dce8-4b08-b405-926c85f68e49 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:53.141080 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcfc62-e2ae-44f3-b2ae-9a1a921d6508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:53.141806 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcfc62-e2ae-44f3-b2ae-9a1a921d6508 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:53.142032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcfc62-e2ae-44f3-b2ae-9a1a921d6508 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:53.142325 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcfc62-e2ae-44f3-b2ae-9a1a921d6508 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:53.143127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedcfc62-e2ae-44f3-b2ae-9a1a921d6508 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:53.143361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedcfc62-e2ae-44f3-b2ae-9a1a921d6508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:53.144794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedcfc62-e2ae-44f3-b2ae-9a1a921d6508 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:53.158010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 340/687] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:53.158915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcfc62-e2ae-44f3-b2ae-9a1a921d6508 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:53.189037 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949cdd4-4a9b-4a40-ba91-857fa3900a66 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:53.227811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27c671-c2f5-4e78-9b67-5eea489c9505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:53.229290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27c671-c2f5-4e78-9b67-5eea489c9505 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:53.229929 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27c671-c2f5-4e78-9b67-5eea489c9505 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:53.230506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27c671-c2f5-4e78-9b67-5eea489c9505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:53.231216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:53.232500 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27c671-c2f5-4e78-9b67-5eea489c9505 None admin] RBAC: Adding query filter params (role.id=bdaf8732b34c43a38465efde7e18a060, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=e2c5489b02604fcbae8dd5eff816d39d) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:53.233050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27c671-c2f5-4e78-9b67-5eea489c9505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:53.234907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27c671-c2f5-4e78-9b67-5eea489c9505 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:53.238988 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:53.243630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 348/688] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:04:53 2022] GET /identity/v3/role_assignments?user.id=e2c5489b02604fcbae8dd5eff816d39d&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=bdaf8732b34c43a38465efde7e18a060 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:53.244713 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27c671-c2f5-4e78-9b67-5eea489c9505 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:53.279142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcfc62-e2ae-44f3-b2ae-9a1a921d6508 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:53.342390 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75020186-2957-4751-bcbc-cd3a8f8123ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:53.343048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75020186-2957-4751-bcbc-cd3a8f8123ff None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:53.343327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75020186-2957-4751-bcbc-cd3a8f8123ff None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:53.343599 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75020186-2957-4751-bcbc-cd3a8f8123ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:53.362050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27c671-c2f5-4e78-9b67-5eea489c9505 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:53.400832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8755c-486c-4739-b7d0-b79e7133636d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:53.402049 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8755c-486c-4739-b7d0-b79e7133636d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:53.402622 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8755c-486c-4739-b7d0-b79e7133636d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:53.403129 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8755c-486c-4739-b7d0-b79e7133636d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:53.416377 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8755c-486c-4739-b7d0-b79e7133636d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:53.418422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8755c-486c-4739-b7d0-b79e7133636d None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:53.438338 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-e7b8755c-486c-4739-b7d0-b79e7133636d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:53.438338 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:53.438338 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:53.438338 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:53.438338 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:53.438338 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:53.438338 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:53.438338 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:53.438338 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:53.438338 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:53.438338 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:53.438338 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:53.438338 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:53.438338 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:53.438338 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:53.439430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:53.440377 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:53.440377 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:53.440377 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:53.440377 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:53.442885 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 349/689] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:53.449682 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8755c-486c-4739-b7d0-b79e7133636d None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:53.487845 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75020186-2957-4751-bcbc-cd3a8f8123ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:53.490518 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75020186-2957-4751-bcbc-cd3a8f8123ff None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:53.573405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:53.626112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:53.642582 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8755c-486c-4739-b7d0-b79e7133636d None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:53.675756 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 341/690] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:53.677846 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75020186-2957-4751-bcbc-cd3a8f8123ff None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:53.693956 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89ee6e-8301-4d46-815c-3024f75e5d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:53.695436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89ee6e-8301-4d46-815c-3024f75e5d63 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:53.696142 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89ee6e-8301-4d46-815c-3024f75e5d63 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:53.696600 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89ee6e-8301-4d46-815c-3024f75e5d63 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:53.697903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89ee6e-8301-4d46-815c-3024f75e5d63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:53.698770 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89ee6e-8301-4d46-815c-3024f75e5d63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:53.700368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89ee6e-8301-4d46-815c-3024f75e5d63 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:53.722225 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 350/691] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:53.724476 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89ee6e-8301-4d46-815c-3024f75e5d63 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:53.802081 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75020186-2957-4751-bcbc-cd3a8f8123ff None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:53.840605 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb452d4a-a1fe-4230-8d9c-8a18aa7f4a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:53.841864 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb452d4a-a1fe-4230-8d9c-8a18aa7f4a49 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:53.842472 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb452d4a-a1fe-4230-8d9c-8a18aa7f4a49 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:53.842950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb452d4a-a1fe-4230-8d9c-8a18aa7f4a49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:53.855109 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb452d4a-a1fe-4230-8d9c-8a18aa7f4a49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:53.856939 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb452d4a-a1fe-4230-8d9c-8a18aa7f4a49 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:53.857170 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89ee6e-8301-4d46-815c-3024f75e5d63 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:53.874113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:53.878575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-eb452d4a-a1fe-4230-8d9c-8a18aa7f4a49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:53.878575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:53.878575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:53.878575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:53.878575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:53.878575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:53.878575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:53.878575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:53.878575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:53.878575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:53.878575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:53.878575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:53.878575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:53.878575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:53.878575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:53.879643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:53.880597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:53.880597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:53.880597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:53.880597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:53.887913 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 342/692] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:53.889676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe362cb-7570-454d-99d8-2d00bfc26718 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:53.890281 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe362cb-7570-454d-99d8-2d00bfc26718 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:53.890759 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe362cb-7570-454d-99d8-2d00bfc26718 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:53.892776 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 343/693] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:53.893811 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe362cb-7570-454d-99d8-2d00bfc26718 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:53.896119 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a351d98-5ebd-4118-9af2-9b9f5dde867c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:53.896920 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a351d98-5ebd-4118-9af2-9b9f5dde867c None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:53.897203 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a351d98-5ebd-4118-9af2-9b9f5dde867c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:53.897758 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a351d98-5ebd-4118-9af2-9b9f5dde867c None admin] PATH_INFO: `/v3/users/nova` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:53.907556 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a351d98-5ebd-4118-9af2-9b9f5dde867c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:53.909051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a351d98-5ebd-4118-9af2-9b9f5dde867c None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-1a351d98-5ebd-4118-9af2-9b9f5dde867c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:53.926888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:53.927986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:53.929056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:53.929056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:53.929056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:53.929056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:53.929056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:53.929056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:53.929056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:53.929056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:53.929056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:04:53.929056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:53.929870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:53.936679 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 351/694] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:04:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:53.938165 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc168701-51b8-44e3-9891-661cc8c90c26 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:53.938569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc168701-51b8-44e3-9891-661cc8c90c26 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:53.938968 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc168701-51b8-44e3-9891-661cc8c90c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:53.956354 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-dc168701-51b8-44e3-9891-661cc8c90c26 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:53.957967 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-dc168701-51b8-44e3-9891-661cc8c90c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:54.018505 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe362cb-7570-454d-99d8-2d00bfc26718 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:54.032011 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 352/695] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:54.033362 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc168701-51b8-44e3-9891-661cc8c90c26 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:54.055377 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce93e6-164c-4381-92da-a49f03f9afe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:54.056214 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce93e6-164c-4381-92da-a49f03f9afe2 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:54.056584 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce93e6-164c-4381-92da-a49f03f9afe2 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:54.056972 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce93e6-164c-4381-92da-a49f03f9afe2 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:54.057835 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce93e6-164c-4381-92da-a49f03f9afe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:54.058237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce93e6-164c-4381-92da-a49f03f9afe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:54.059785 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce93e6-164c-4381-92da-a49f03f9afe2 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:54.077796 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 344/696] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:54.078886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce93e6-164c-4381-92da-a49f03f9afe2 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:54.173195 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc168701-51b8-44e3-9891-661cc8c90c26 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:54.181607 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:54.194638 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:54.203268 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce93e6-164c-4381-92da-a49f03f9afe2 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:54.225544 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e4f59-cd83-439a-bb91-1d78d4b9e6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:54.226541 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e4f59-cd83-439a-bb91-1d78d4b9e6fb None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:54.226829 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e4f59-cd83-439a-bb91-1d78d4b9e6fb None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:54.227107 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e4f59-cd83-439a-bb91-1d78d4b9e6fb None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:54.228223 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e4f59-cd83-439a-bb91-1d78d4b9e6fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:54.228668 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e4f59-cd83-439a-bb91-1d78d4b9e6fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:54.230976 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e4f59-cd83-439a-bb91-1d78d4b9e6fb None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:54.248938 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8cf65-9995-47f4-a49d-5f6e2c0479a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:54.250086 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8cf65-9995-47f4-a49d-5f6e2c0479a8 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:54.250599 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8cf65-9995-47f4-a49d-5f6e2c0479a8 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:54.251106 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8cf65-9995-47f4-a49d-5f6e2c0479a8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:54.251980 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8cf65-9995-47f4-a49d-5f6e2c0479a8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:54.253587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8cf65-9995-47f4-a49d-5f6e2c0479a8 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:54.256123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 353/697] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:04:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:54.256980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e4f59-cd83-439a-bb91-1d78d4b9e6fb None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-baf8cf65-9995-47f4-a49d-5f6e2c0479a8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:54.269092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:54.270207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:54.271188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:54.271188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:54.271188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:54.271188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:54.271188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:54.271188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:54.271188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:54.271188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:54.271188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:54.271188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:54.271188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:54.271188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:04:54.271188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:04:54.271188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:04:54.271188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:54.274053 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 345/698] 10.222.0.84 () {56 vars in 1129 bytes} [Thu May 19 20:04:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:54.275079 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8cf65-9995-47f4-a49d-5f6e2c0479a8 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:54.381277 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e4f59-cd83-439a-bb91-1d78d4b9e6fb None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:54.397880 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8cf65-9995-47f4-a49d-5f6e2c0479a8 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:54.417909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a821d5-4839-4390-bb2e-4d878304c384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:54.419094 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a821d5-4839-4390-bb2e-4d878304c384 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:54.419612 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a821d5-4839-4390-bb2e-4d878304c384 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:54.420111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a821d5-4839-4390-bb2e-4d878304c384 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:54.427714 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a821d5-4839-4390-bb2e-4d878304c384 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:54.429408 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a821d5-4839-4390-bb2e-4d878304c384 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:54.441967 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87716e1-934e-473e-94bf-554753ea739e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:54.442658 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87716e1-934e-473e-94bf-554753ea739e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:54.442942 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87716e1-934e-473e-94bf-554753ea739e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:54.443220 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87716e1-934e-473e-94bf-554753ea739e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:54.445232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-f9a821d5-4839-4390-bb2e-4d878304c384 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:54.445232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:54.445232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:54.445232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:54.445232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:54.445232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:54.445232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:54.445232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:54.445232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:54.445232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:54.445232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:54.445232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:54.445232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:54.445232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:54.445232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:54.446190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:54.447471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:54.447471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:54.447471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:54.447471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:54.447471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:54.447471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:54.447471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:54.447471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:54.449847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 354/699] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:54.450972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a821d5-4839-4390-bb2e-4d878304c384 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:54.466631 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:54.477051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:54.534733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87716e1-934e-473e-94bf-554753ea739e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:54.536629 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87716e1-934e-473e-94bf-554753ea739e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:54.584983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a821d5-4839-4390-bb2e-4d878304c384 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:54.634773 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7701a8c-60fa-4a6e-a847-ebdb2a0f4e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:54.635456 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7701a8c-60fa-4a6e-a847-ebdb2a0f4e12 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:54.635742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7701a8c-60fa-4a6e-a847-ebdb2a0f4e12 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:54.636029 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7701a8c-60fa-4a6e-a847-ebdb2a0f4e12 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:54.637058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7701a8c-60fa-4a6e-a847-ebdb2a0f4e12 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:54.637373 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7701a8c-60fa-4a6e-a847-ebdb2a0f4e12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:54.639162 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7701a8c-60fa-4a6e-a847-ebdb2a0f4e12 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:54.649245 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 355/700] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 20:04:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:54.649245 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7701a8c-60fa-4a6e-a847-ebdb2a0f4e12 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:54.659985 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 346/701] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:04:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:04:54.661109 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87716e1-934e-473e-94bf-554753ea739e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:54.720639 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:54.753114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:54.771280 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7701a8c-60fa-4a6e-a847-ebdb2a0f4e12 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:54.792569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87716e1-934e-473e-94bf-554753ea739e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:54.816157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673abee1-a11d-46d6-9aad-5381a5f3a573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:54.816837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673abee1-a11d-46d6-9aad-5381a5f3a573 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:54.817122 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673abee1-a11d-46d6-9aad-5381a5f3a573 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:54.817399 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673abee1-a11d-46d6-9aad-5381a5f3a573 None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:54.818480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673abee1-a11d-46d6-9aad-5381a5f3a573 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:54.819391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673abee1-a11d-46d6-9aad-5381a5f3a573 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:54.821126 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673abee1-a11d-46d6-9aad-5381a5f3a573 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:54.831043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b80812-dedd-4974-90d8-85040702ddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:54.832399 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b80812-dedd-4974-90d8-85040702ddfe None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:54.832911 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b80812-dedd-4974-90d8-85040702ddfe None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:54.833388 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b80812-dedd-4974-90d8-85040702ddfe None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:54.834358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b80812-dedd-4974-90d8-85040702ddfe None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:54.835016 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b80812-dedd-4974-90d8-85040702ddfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:54.836679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b80812-dedd-4974-90d8-85040702ddfe None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:54.837176 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 356/702] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:54.838151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673abee1-a11d-46d6-9aad-5381a5f3a573 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:54.850465 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 347/703] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 20:04:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:54.851841 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b80812-dedd-4974-90d8-85040702ddfe None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:54.961663 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673abee1-a11d-46d6-9aad-5381a5f3a573 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:55.004501 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37743ce-48c5-4327-b733-2e08acf53953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:55.005363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37743ce-48c5-4327-b733-2e08acf53953 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:55.005651 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37743ce-48c5-4327-b733-2e08acf53953 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:55.005923 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37743ce-48c5-4327-b733-2e08acf53953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:55.018590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37743ce-48c5-4327-b733-2e08acf53953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:55.020083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37743ce-48c5-4327-b733-2e08acf53953 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:55.023304 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:55.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-c37743ce-48c5-4327-b733-2e08acf53953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:55.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:55.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:55.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:55.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:55.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:55.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:55.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:55.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:55.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:55.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:55.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:55.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:55.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:55.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:55.048183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:55.049457 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:55.049457 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:55.049457 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:55.049457 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:55.051955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 357/704] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:55.061513 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37743ce-48c5-4327-b733-2e08acf53953 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:55.099069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b80812-dedd-4974-90d8-85040702ddfe None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:55.131944 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:55.159677 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f00ef-f31a-4d99-996c-4501a7dcd629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:55.160372 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f00ef-f31a-4d99-996c-4501a7dcd629 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:55.160653 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f00ef-f31a-4d99-996c-4501a7dcd629 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:55.160928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f00ef-f31a-4d99-996c-4501a7dcd629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:55.161970 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f00ef-f31a-4d99-996c-4501a7dcd629 None admin] RBAC: Adding query filter params (role.id=bdaf8732b34c43a38465efde7e18a060, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=87b30300be1a41128ff88e12e260f63b) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:55.162307 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f00ef-f31a-4d99-996c-4501a7dcd629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:55.163774 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f00ef-f31a-4d99-996c-4501a7dcd629 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:55.170854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 348/705] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:04:54 2022] GET /identity/v3/role_assignments?user.id=87b30300be1a41128ff88e12e260f63b&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=bdaf8732b34c43a38465efde7e18a060 => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:55.218298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37743ce-48c5-4327-b733-2e08acf53953 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:55.260397 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a5fd8-f514-49cd-aa28-d44e1bde9fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:55.261246 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a5fd8-f514-49cd-aa28-d44e1bde9fb2 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:55.261431 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a5fd8-f514-49cd-aa28-d44e1bde9fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:55.261700 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a5fd8-f514-49cd-aa28-d44e1bde9fb2 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:55.262527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a5fd8-f514-49cd-aa28-d44e1bde9fb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:55.262825 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a5fd8-f514-49cd-aa28-d44e1bde9fb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:55.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea065ea-7e2e-4a77-a23b-3accab5ea522 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:55.264245 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a5fd8-f514-49cd-aa28-d44e1bde9fb2 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:55.264679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea065ea-7e2e-4a77-a23b-3accab5ea522 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:55.265164 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea065ea-7e2e-4a77-a23b-3accab5ea522 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:55.267368 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 349/706] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:04:55.272780 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb0327-0371-459c-bd1a-111c84ea7049 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:55.273063 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb0327-0371-459c-bd1a-111c84ea7049 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:55.273345 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb0327-0371-459c-bd1a-111c84ea7049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:55.288632 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 358/707] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:55.292435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-77cb0327-0371-459c-bd1a-111c84ea7049 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:55.293753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-77cb0327-0371-459c-bd1a-111c84ea7049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:55.294951 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a5fd8-f514-49cd-aa28-d44e1bde9fb2 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:55.381544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 350/708] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:04:55.401051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:55.422222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb0327-0371-459c-bd1a-111c84ea7049 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:55.458856 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a5fd8-f514-49cd-aa28-d44e1bde9fb2 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:55.499763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec0903-122c-46c5-9658-58d97af535cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:55.500588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec0903-122c-46c5-9658-58d97af535cb None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:55.500869 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec0903-122c-46c5-9658-58d97af535cb None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:55.501143 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec0903-122c-46c5-9658-58d97af535cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:55.511001 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec0903-122c-46c5-9658-58d97af535cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:55.512382 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec0903-122c-46c5-9658-58d97af535cb None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:55.530345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-e5ec0903-122c-46c5-9658-58d97af535cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:55.530345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:55.530345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:55.530345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:55.530345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:55.530345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:55.530345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:55.530345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:55.530345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:55.530345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:55.530345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:55.530345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:55.530345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:04:55.530345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:04:55.530345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:55.531352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:55.532285 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:04:55.532285 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:04:55.532285 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:04:55.532285 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:04:55.532285 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:04:55.532285 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:04:55.532285 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:04:55.532285 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:55.536275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 359/709] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:55.537491 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec0903-122c-46c5-9658-58d97af535cb None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:55.542805 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb0327-0371-459c-bd1a-111c84ea7049 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:55.551010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:55.582909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02692e4f-7964-4d22-970f-9691ac3ade2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:55.584020 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02692e4f-7964-4d22-970f-9691ac3ade2a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:55.584453 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02692e4f-7964-4d22-970f-9691ac3ade2a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:55.584908 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02692e4f-7964-4d22-970f-9691ac3ade2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:55.597149 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02692e4f-7964-4d22-970f-9691ac3ade2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:55.599117 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02692e4f-7964-4d22-970f-9691ac3ade2a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:55.620127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-02692e4f-7964-4d22-970f-9691ac3ade2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:55.620127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:55.620127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:55.620127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:55.620127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:55.620127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:55.620127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:55.620127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:55.620127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:55.620127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:55.620127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:55.620127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:55.620127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:55.620127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:55.620127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:55.621361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:55.622528 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:55.622528 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:55.622528 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:55.622528 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:55.624596 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 351/710] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:55.630380 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02692e4f-7964-4d22-970f-9691ac3ade2a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:55.647528 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec0903-122c-46c5-9658-58d97af535cb None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:55.664720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:55.684282 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e8ac9-964d-47f3-9a49-283ffd390b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:55.684952 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e8ac9-964d-47f3-9a49-283ffd390b47 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:55.685269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e8ac9-964d-47f3-9a49-283ffd390b47 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:55.685490 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e8ac9-964d-47f3-9a49-283ffd390b47 None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:55.686307 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e8ac9-964d-47f3-9a49-283ffd390b47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:55.686902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e8ac9-964d-47f3-9a49-283ffd390b47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:55.688056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e8ac9-964d-47f3-9a49-283ffd390b47 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:55.705275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 360/711] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:04:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:55.706778 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e8ac9-964d-47f3-9a49-283ffd390b47 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:55.748577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02692e4f-7964-4d22-970f-9691ac3ade2a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:55.807824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0abc91-35be-408d-8d0d-5459851ca110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:55.808506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0abc91-35be-408d-8d0d-5459851ca110 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:55.808786 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0abc91-35be-408d-8d0d-5459851ca110 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:55.809055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0abc91-35be-408d-8d0d-5459851ca110 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:55.809786 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0abc91-35be-408d-8d0d-5459851ca110 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:55.810086 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0abc91-35be-408d-8d0d-5459851ca110 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:55.812877 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0abc91-35be-408d-8d0d-5459851ca110 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:55.840619 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 352/712] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:55.844827 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0abc91-35be-408d-8d0d-5459851ca110 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:55.850074 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e8ac9-964d-47f3-9a49-283ffd390b47 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:55.886674 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:55.917602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48026600-4c9e-48b2-9145-60be15832786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:55.918286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48026600-4c9e-48b2-9145-60be15832786 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:55.918572 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48026600-4c9e-48b2-9145-60be15832786 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:55.918841 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48026600-4c9e-48b2-9145-60be15832786 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:55.919679 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48026600-4c9e-48b2-9145-60be15832786 None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=378b046698c04b7b88194a9ffe5b045a) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:55.919988 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48026600-4c9e-48b2-9145-60be15832786 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:55.921463 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48026600-4c9e-48b2-9145-60be15832786 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:55.932459 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 361/713] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:04:55 2022] GET /identity/v3/role_assignments?user.id=378b046698c04b7b88194a9ffe5b045a&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:55.997435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0abc91-35be-408d-8d0d-5459851ca110 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:56.027962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b1af7-49fd-44e5-8f13-dca53528b7ff None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:56.028554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b1af7-49fd-44e5-8f13-dca53528b7ff None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:56.028554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b1af7-49fd-44e5-8f13-dca53528b7ff None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:56.030404 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 362/714] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:56.036856 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c20f5-af80-4fe7-bf18-0a8513fb4f14 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:56.037133 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c20f5-af80-4fe7-bf18-0a8513fb4f14 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:56.037413 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c20f5-af80-4fe7-bf18-0a8513fb4f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:56.038187 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3234d0-abe3-4f3a-b0d0-5c0eace59644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:56.039304 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3234d0-abe3-4f3a-b0d0-5c0eace59644 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:56.039795 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3234d0-abe3-4f3a-b0d0-5c0eace59644 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:56.040276 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3234d0-abe3-4f3a-b0d0-5c0eace59644 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:56.041414 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3234d0-abe3-4f3a-b0d0-5c0eace59644 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:56.043284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3234d0-abe3-4f3a-b0d0-5c0eace59644 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:56.066828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 353/715] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:04:55 2022] POST /identity/v3/users => generated 254 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:04:56.069104 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-cc8c20f5-af80-4fe7-bf18-0a8513fb4f14 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:56.074369 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-cc8c20f5-af80-4fe7-bf18-0a8513fb4f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:56.111318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:56.152193 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 363/716] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:56.195324 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3234d0-abe3-4f3a-b0d0-5c0eace59644 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:56.295966 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:56.328584 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3234d0-abe3-4f3a-b0d0-5c0eace59644 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:56.362751 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c76227-2acf-430f-9f1d-15f9c4312e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:56.363605 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76227-2acf-430f-9f1d-15f9c4312e7b None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:56.363884 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76227-2acf-430f-9f1d-15f9c4312e7b None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:56.364156 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76227-2acf-430f-9f1d-15f9c4312e7b None admin] PATH_INFO: `/v3/services/network` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:56.364864 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c76227-2acf-430f-9f1d-15f9c4312e7b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:56.366301 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c76227-2acf-430f-9f1d-15f9c4312e7b None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-36c76227-2acf-430f-9f1d-15f9c4312e7b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:56.379489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:56.380686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:56.381600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:56.381600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:56.381600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:56.381600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:56.381600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:56.381600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:56.381600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:56.381600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:56.381600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:56.381600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:56.381600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:04:56.381600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:04:56.381600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:04:56.381600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:56.381600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 354/717] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:04:56 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:56.390227 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c20f5-af80-4fe7-bf18-0a8513fb4f14 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:56.494381 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c20f5-af80-4fe7-bf18-0a8513fb4f14 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:56.530227 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca4ced-4dd4-4e32-81a5-331d59300cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:56.530928 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca4ced-4dd4-4e32-81a5-331d59300cba None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:56.531204 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca4ced-4dd4-4e32-81a5-331d59300cba None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:56.531478 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca4ced-4dd4-4e32-81a5-331d59300cba None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:56.532196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca4ced-4dd4-4e32-81a5-331d59300cba None admin] RBAC: Adding query filter params (name=network) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:56.532500 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca4ced-4dd4-4e32-81a5-331d59300cba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:56.533872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca4ced-4dd4-4e32-81a5-331d59300cba None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:56.541511 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 364/718] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:04:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:56.549649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c76227-2acf-430f-9f1d-15f9c4312e7b None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:56.702160 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c76227-2acf-430f-9f1d-15f9c4312e7b None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:56.763486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d961f-f6c1-4c78-9332-691c2a0cad61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:56.764167 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d961f-f6c1-4c78-9332-691c2a0cad61 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:56.764452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d961f-f6c1-4c78-9332-691c2a0cad61 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:56.764732 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d961f-f6c1-4c78-9332-691c2a0cad61 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:56.765448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d961f-f6c1-4c78-9332-691c2a0cad61 None admin] RBAC: Adding query filter params (type=network) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:56.765762 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d961f-f6c1-4c78-9332-691c2a0cad61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:56.767180 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d961f-f6c1-4c78-9332-691c2a0cad61 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:56.778899 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 355/719] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:04:56 2022] GET /identity/v3/services?type=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:57.848569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a60b2b-4e28-4785-ace7-525d6485ce23 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:57.849598 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a60b2b-4e28-4785-ace7-525d6485ce23 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:57.850092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a60b2b-4e28-4785-ace7-525d6485ce23 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:57.852036 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 365/720] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:57.857795 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbff50-ca94-4ed7-aba9-050d070cb62e None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:57.858327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbff50-ca94-4ed7-aba9-050d070cb62e None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:57.858416 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbff50-ca94-4ed7-aba9-050d070cb62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:57.863623 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:57.881968 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-80cbff50-ca94-4ed7-aba9-050d070cb62e None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:57.883205 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-80cbff50-ca94-4ed7-aba9-050d070cb62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:57.985015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 356/721] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:58.019063 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a60b2b-4e28-4785-ace7-525d6485ce23 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:58.025508 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59b500-23c9-4b89-b100-8db1e010f80c None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:58.026550 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59b500-23c9-4b89-b100-8db1e010f80c None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:58.026848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59b500-23c9-4b89-b100-8db1e010f80c None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:58.028501 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 357/722] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:04:58.033629 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7a90-42fc-4958-820c-26578775aee5 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:58.033917 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7a90-42fc-4958-820c-26578775aee5 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:58.034198 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7a90-42fc-4958-820c-26578775aee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:58.073490 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-93ce7a90-42fc-4958-820c-26578775aee5 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:58.074743 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-93ce7a90-42fc-4958-820c-26578775aee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:58.086632 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:58.184526 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a60b2b-4e28-4785-ace7-525d6485ce23 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:58.229623 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 358/723] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:58.235524 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38b5ed-89c0-42cd-a363-8966d2550b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:58.236201 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38b5ed-89c0-42cd-a363-8966d2550b78 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:58.236482 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38b5ed-89c0-42cd-a363-8966d2550b78 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:58.236750 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38b5ed-89c0-42cd-a363-8966d2550b78 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:58.237475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38b5ed-89c0-42cd-a363-8966d2550b78 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:58.242922 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38b5ed-89c0-42cd-a363-8966d2550b78 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:58.264849 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce7a90-42fc-4958-820c-26578775aee5 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:58.286972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 366/724] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:04:58 2022] POST /identity/v3/services => generated 251 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:04:58.361914 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:58.413531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce7a90-42fc-4958-820c-26578775aee5 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:58.469632 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6b47b-4e10-45ae-8586-c1f903107ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:58.470385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6b47b-4e10-45ae-8586-c1f903107ec7 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:58.470660 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6b47b-4e10-45ae-8586-c1f903107ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:58.470927 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6b47b-4e10-45ae-8586-c1f903107ec7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:58.482807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6b47b-4e10-45ae-8586-c1f903107ec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-e7f6b47b-4e10-45ae-8586-c1f903107ec7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:58.490459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:58.491427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:58.492352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:58.492352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:58.492352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:58.492352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:58.492352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:58.492352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:58.492352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:58.492352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:58.492352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:58.492352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:58.492352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:04:58.492352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:58.496571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 359/725] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:04:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:04:58.497987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38b5ed-89c0-42cd-a363-8966d2550b78 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:58.637241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:58.645601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38b5ed-89c0-42cd-a363-8966d2550b78 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:58.688964 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee01e72-1060-460e-86f1-b62620ca082e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:58.689669 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee01e72-1060-460e-86f1-b62620ca082e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:58.689956 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee01e72-1060-460e-86f1-b62620ca082e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:58.690230 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee01e72-1060-460e-86f1-b62620ca082e None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:58.691034 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee01e72-1060-460e-86f1-b62620ca082e None admin] RBAC: Adding query filter params (name=admin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:58.691343 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee01e72-1060-460e-86f1-b62620ca082e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:58.692737 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee01e72-1060-460e-86f1-b62620ca082e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:58.702684 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 367/726] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:04:58.708356 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6b47b-4e10-45ae-8586-c1f903107ec7 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:58.926499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6b47b-4e10-45ae-8586-c1f903107ec7 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:58.955397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:58.981055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a9485-4fd8-43f8-b169-ef68deb456ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:58.981920 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a9485-4fd8-43f8-b169-ef68deb456ce None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:58.982197 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a9485-4fd8-43f8-b169-ef68deb456ce None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:58.982490 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a9485-4fd8-43f8-b169-ef68deb456ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:59.046538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60402325-46bf-4491-84ee-ff21d6f7cc8a None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:59.047075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60402325-46bf-4491-84ee-ff21d6f7cc8a None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:59.047075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60402325-46bf-4491-84ee-ff21d6f7cc8a None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:59.048876 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 368/727] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:59.055979 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3ea5d-f848-4efa-86a9-8cfeaae36bdb None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:59.056334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3ea5d-f848-4efa-86a9-8cfeaae36bdb None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:59.056528 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3ea5d-f848-4efa-86a9-8cfeaae36bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:59.095706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-b4b3ea5d-f848-4efa-86a9-8cfeaae36bdb None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:59.096932 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-b4b3ea5d-f848-4efa-86a9-8cfeaae36bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:59.129267 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a9485-4fd8-43f8-b169-ef68deb456ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:59.130890 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a9485-4fd8-43f8-b169-ef68deb456ce None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:59.150327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-d73a9485-4fd8-43f8-b169-ef68deb456ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:59.150327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:59.150327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:59.150327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:59.150327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:59.150327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:59.150327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:59.150327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:59.150327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:59.150327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:59.150327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:59.150327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:59.150327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:04:59.150327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:04:59.150327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:59.151915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:59.152891 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:04:59.152891 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:04:59.152891 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:04:59.152891 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:59.153432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 360/728] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 446 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:59.154828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa860e2e-5bd4-4c22-bf7b-b7aba4ff3e29 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:59.155111 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa860e2e-5bd4-4c22-bf7b-b7aba4ff3e29 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:59.155386 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa860e2e-5bd4-4c22-bf7b-b7aba4ff3e29 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:59.156882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 361/729] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:59.160424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa860e2e-5bd4-4c22-bf7b-b7aba4ff3e29 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:59.210700 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 369/730] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:59.211941 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a263a-2721-4dea-8c99-09a268cc7291 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:59.212228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a263a-2721-4dea-8c99-09a268cc7291 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:59.212506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a263a-2721-4dea-8c99-09a268cc7291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:59.231921 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-175a263a-2721-4dea-8c99-09a268cc7291 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:59.233210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-175a263a-2721-4dea-8c99-09a268cc7291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:59.278073 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa860e2e-5bd4-4c22-bf7b-b7aba4ff3e29 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:59.304441 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 370/731] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:59.305546 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a263a-2721-4dea-8c99-09a268cc7291 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:59.324010 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:59.335278 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b4b01-ed4b-49be-9dbe-801af7b7c3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:59.335965 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b4b01-ed4b-49be-9dbe-801af7b7c3bc None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:59.336246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b4b01-ed4b-49be-9dbe-801af7b7c3bc None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:59.336521 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b4b01-ed4b-49be-9dbe-801af7b7c3bc None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:59.337290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b4b01-ed4b-49be-9dbe-801af7b7c3bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:59.337593 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b4b01-ed4b-49be-9dbe-801af7b7c3bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:59.339038 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b4b01-ed4b-49be-9dbe-801af7b7c3bc None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:59.352431 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 362/732] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:59.353276 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b4b01-ed4b-49be-9dbe-801af7b7c3bc None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:59.385252 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:59.420030 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a263a-2721-4dea-8c99-09a268cc7291 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:59.456798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4386a-f33e-41af-82f0-9d01ad7f4e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:59.457530 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4386a-f33e-41af-82f0-9d01ad7f4e48 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:59.457811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4386a-f33e-41af-82f0-9d01ad7f4e48 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:59.458085 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4386a-f33e-41af-82f0-9d01ad7f4e48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:59.466923 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b4b01-ed4b-49be-9dbe-801af7b7c3bc None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:59.469857 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4386a-f33e-41af-82f0-9d01ad7f4e48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-3ef4386a-f33e-41af-82f0-9d01ad7f4e48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:59.477548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:59.478791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:59.479872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:59.479872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:59.479872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:59.479872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:59.479872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:59.479872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:59.479872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:59.479872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:59.479872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:59.479872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:59.479872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:04:59.479872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:59.479872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 371/733] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:04:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:59.480608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4386a-f33e-41af-82f0-9d01ad7f4e48 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:59.507523 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970448d-897c-454a-b7b0-d8d955cfeb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:59.508737 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970448d-897c-454a-b7b0-d8d955cfeb39 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:59.509225 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970448d-897c-454a-b7b0-d8d955cfeb39 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:59.509795 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970448d-897c-454a-b7b0-d8d955cfeb39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:59.521364 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970448d-897c-454a-b7b0-d8d955cfeb39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-3970448d-897c-454a-b7b0-d8d955cfeb39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:04:59.529547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:59.530588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:59.531503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:59.531503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:59.531503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:59.531503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:59.531503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:04:59.531503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:04:59.531503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:04:59.531503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:04:59.531503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:04:59.531503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:04:59.531503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:04:59.531503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:04:59.534283 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 363/734] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:04:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:59.535301 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970448d-897c-454a-b7b0-d8d955cfeb39 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:59.563139 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:59.590007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4386a-f33e-41af-82f0-9d01ad7f4e48 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:59.627508 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb43610-dabe-45b1-b003-a220d40aa2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:59.628317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb43610-dabe-45b1-b003-a220d40aa2b9 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:59.628597 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb43610-dabe-45b1-b003-a220d40aa2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:59.628870 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb43610-dabe-45b1-b003-a220d40aa2b9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:59.633490 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:59.637713 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb43610-dabe-45b1-b003-a220d40aa2b9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:59.639367 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb43610-dabe-45b1-b003-a220d40aa2b9 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:59.646446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970448d-897c-454a-b7b0-d8d955cfeb39 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-deb43610-dabe-45b1-b003-a220d40aa2b9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:04:59.654051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:04:59.655290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:04:59.656308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:04:59.656308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:04:59.656308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:04:59.656308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:04:59.656308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:04:59.656308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:04:59.656308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:04:59.656308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:04:59.656308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:04:59.656308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:04:59.656308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 372/735] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:04:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:04:59.656308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb43610-dabe-45b1-b003-a220d40aa2b9 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:59.694928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7217f52-11f6-4396-9a31-0b41261275f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:59.695876 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7217f52-11f6-4396-9a31-0b41261275f1 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:59.696343 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7217f52-11f6-4396-9a31-0b41261275f1 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:59.696773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7217f52-11f6-4396-9a31-0b41261275f1 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:59.698716 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7217f52-11f6-4396-9a31-0b41261275f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:59.699189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7217f52-11f6-4396-9a31-0b41261275f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:59.701627 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7217f52-11f6-4396-9a31-0b41261275f1 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:59.715551 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 364/736] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:04:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:59.715551 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2ba6f-cf28-4d34-8a34-b520eaa0964f None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:59.715551 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2ba6f-cf28-4d34-8a34-b520eaa0964f None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:59.715551 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2ba6f-cf28-4d34-8a34-b520eaa0964f None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:59.715551 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 365/737] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:04:59.715551 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2ba6f-cf28-4d34-8a34-b520eaa0964f None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:59.777459 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb43610-dabe-45b1-b003-a220d40aa2b9 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:59.812480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a7577-c718-48a3-8a35-e920b4e76c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:59.813169 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a7577-c718-48a3-8a35-e920b4e76c71 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:59.813462 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a7577-c718-48a3-8a35-e920b4e76c71 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:59.813734 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a7577-c718-48a3-8a35-e920b4e76c71 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:59.814534 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a7577-c718-48a3-8a35-e920b4e76c71 None admin] RBAC: Adding query filter params (name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:59.814932 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a7577-c718-48a3-8a35-e920b4e76c71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:59.816230 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a7577-c718-48a3-8a35-e920b4e76c71 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:59.818586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:59.830360 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 373/738] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:04:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:59.830360 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a7577-c718-48a3-8a35-e920b4e76c71 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:04:59.830681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2ba6f-cf28-4d34-8a34-b520eaa0964f None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:59.867237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7cf6c-7987-4848-9215-e5f5320a3790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:04:59.868210 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7cf6c-7987-4848-9215-e5f5320a3790 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:59.868489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7cf6c-7987-4848-9215-e5f5320a3790 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:59.868759 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7cf6c-7987-4848-9215-e5f5320a3790 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:59.869668 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7cf6c-7987-4848-9215-e5f5320a3790 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:04:59.869983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7cf6c-7987-4848-9215-e5f5320a3790 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:04:59.871700 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7cf6c-7987-4848-9215-e5f5320a3790 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:04:59.890003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 366/739] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 20:04:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:04:59.891272 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de42ec6-c10c-43c9-bfb8-ebb6340e2230 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:04:59.891555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de42ec6-c10c-43c9-bfb8-ebb6340e2230 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:04:59.891824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de42ec6-c10c-43c9-bfb8-ebb6340e2230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:04:59.908022 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-1de42ec6-c10c-43c9-bfb8-ebb6340e2230 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:04:59.909195 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-1de42ec6-c10c-43c9-bfb8-ebb6340e2230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:04:59.912773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:04:59.951080 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a7577-c718-48a3-8a35-e920b4e76c71 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:04:59.984437 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 367/740] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:04:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:04:59.994937 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de42ec6-c10c-43c9-bfb8-ebb6340e2230 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:00.001963 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b6c64-0b95-45d9-ab57-e4ae8a29b5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:00.002956 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b6c64-0b95-45d9-ab57-e4ae8a29b5fe None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:00.003392 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b6c64-0b95-45d9-ab57-e4ae8a29b5fe None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:00.003673 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b6c64-0b95-45d9-ab57-e4ae8a29b5fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:00.019123 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b6c64-0b95-45d9-ab57-e4ae8a29b5fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:00.020846 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b6c64-0b95-45d9-ab57-e4ae8a29b5fe None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:00.041334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-2a0b6c64-0b95-45d9-ab57-e4ae8a29b5fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:00.041334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:00.041334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:00.041334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:00.041334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:00.041334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:00.041334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:00.041334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:00.041334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:00.041334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:00.041334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:00.041334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:00.041334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:00.041334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:00.041334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:00.042808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:00.043780 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:00.043780 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:00.043780 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:00.043780 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:00.047192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 374/741] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:00.048038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b6c64-0b95-45d9-ab57-e4ae8a29b5fe None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:00.094677 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de42ec6-c10c-43c9-bfb8-ebb6340e2230 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:00.100806 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:00.129734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d0934-50bd-495b-a994-12fd4c34b053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:00.130569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d0934-50bd-495b-a994-12fd4c34b053 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:00.130860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d0934-50bd-495b-a994-12fd4c34b053 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:00.131133 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d0934-50bd-495b-a994-12fd4c34b053 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:00.139380 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:00.141618 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d0934-50bd-495b-a994-12fd4c34b053 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:00.143255 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d0934-50bd-495b-a994-12fd4c34b053 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:00.156477 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b6c64-0b95-45d9-ab57-e4ae8a29b5fe None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:00.161438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-766d0934-50bd-495b-a994-12fd4c34b053 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:00.161438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:00.161438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:00.161438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:00.161438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:00.161438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:00.161438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:00.161438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:00.161438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:00.161438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:00.161438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:00.161438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:00.161438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:00.161438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:00.161438 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:00.162588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:00.163560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:00.163560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:00.163560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:00.163560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:00.163560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 368/742] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:00.163560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d0934-50bd-495b-a994-12fd4c34b053 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:00.197778 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f076f-7664-45a4-a379-d97d71d49e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:00.198464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f076f-7664-45a4-a379-d97d71d49e20 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:00.198731 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f076f-7664-45a4-a379-d97d71d49e20 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:00.199000 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f076f-7664-45a4-a379-d97d71d49e20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:00.271962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f076f-7664-45a4-a379-d97d71d49e20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:00.273412 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f076f-7664-45a4-a379-d97d71d49e20 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:00.277743 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d0934-50bd-495b-a994-12fd4c34b053 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:00.313414 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462c55f-6aa0-4fa8-99d0-ea991dc303b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:00.314459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462c55f-6aa0-4fa8-99d0-ea991dc303b0 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:00.314969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462c55f-6aa0-4fa8-99d0-ea991dc303b0 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:00.315313 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462c55f-6aa0-4fa8-99d0-ea991dc303b0 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:00.316051 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462c55f-6aa0-4fa8-99d0-ea991dc303b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:00.317447 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462c55f-6aa0-4fa8-99d0-ea991dc303b0 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:00.325662 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:00.342789 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 369/743] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:05:00 2022] POST /identity/v3/services => generated 240 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:05:00.344805 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462c55f-6aa0-4fa8-99d0-ea991dc303b0 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:00.388504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 375/744] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:05:00 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:05:00.389343 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f076f-7664-45a4-a379-d97d71d49e20 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:00.428983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:00.505304 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f076f-7664-45a4-a379-d97d71d49e20 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:00.506685 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462c55f-6aa0-4fa8-99d0-ea991dc303b0 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:00.551702 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642fb0d-b00b-463c-addc-027a631db350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:00.552586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642fb0d-b00b-463c-addc-027a631db350 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:00.552989 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642fb0d-b00b-463c-addc-027a631db350 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:00.553363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642fb0d-b00b-463c-addc-027a631db350 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:00.554201 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642fb0d-b00b-463c-addc-027a631db350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:00.554638 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642fb0d-b00b-463c-addc-027a631db350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:00.556108 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642fb0d-b00b-463c-addc-027a631db350 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:00.566162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad03fd-4f9a-427f-9c29-517f22d3a438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:00.566379 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad03fd-4f9a-427f-9c29-517f22d3a438 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:00.566744 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad03fd-4f9a-427f-9c29-517f22d3a438 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:00.567087 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad03fd-4f9a-427f-9c29-517f22d3a438 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:00.567919 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad03fd-4f9a-427f-9c29-517f22d3a438 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:00.568296 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad03fd-4f9a-427f-9c29-517f22d3a438 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:00.569823 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad03fd-4f9a-427f-9c29-517f22d3a438 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:00.575935 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 376/745] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:00.577067 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642fb0d-b00b-463c-addc-027a631db350 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:00.589430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 370/746] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:00.590296 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad03fd-4f9a-427f-9c29-517f22d3a438 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:00.593200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:00.696066 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642fb0d-b00b-463c-addc-027a631db350 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:00.726619 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad03fd-4f9a-427f-9c29-517f22d3a438 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:00.737493 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:00.738038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37fb8a-b9b1-43af-8421-e13a98d15d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:00.739103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37fb8a-b9b1-43af-8421-e13a98d15d2a None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:00.739609 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37fb8a-b9b1-43af-8421-e13a98d15d2a None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:00.740082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37fb8a-b9b1-43af-8421-e13a98d15d2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:00.751972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e37fb8a-b9b1-43af-8421-e13a98d15d2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:00.753760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e37fb8a-b9b1-43af-8421-e13a98d15d2a None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:00.763677 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee448b-aee1-4f68-8677-d368b762a9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:00.764504 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee448b-aee1-4f68-8677-d368b762a9af None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:00.764862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee448b-aee1-4f68-8677-d368b762a9af None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:00.765041 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee448b-aee1-4f68-8677-d368b762a9af None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:00.772565 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee448b-aee1-4f68-8677-d368b762a9af None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:00.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-9e37fb8a-b9b1-43af-8421-e13a98d15d2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:00.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:00.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:00.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:00.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:00.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:00.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:00.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:00.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:00.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:00.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:00.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:00.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:00.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:00.773873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:00.774847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:00.775829 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:00.775829 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:00.775829 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:00.775829 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:00.776075 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee448b-aee1-4f68-8677-d368b762a9af None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:00.782670 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 377/747] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:00.783880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37fb8a-b9b1-43af-8421-e13a98d15d2a None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-2dee448b-aee1-4f68-8677-d368b762a9af None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:05:00.792577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:00.793883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:00.794962 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:00.794962 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:00.794962 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:05:00.794962 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:05:00.794962 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:05:00.794962 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:05:00.794962 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:05:00.794962 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:05:00.794962 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:05:00.794962 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:00.798814 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 371/748] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:05:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:00.799880 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee448b-aee1-4f68-8677-d368b762a9af None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:00.858054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:00.900600 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37fb8a-b9b1-43af-8421-e13a98d15d2a None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:00.904439 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee448b-aee1-4f68-8677-d368b762a9af None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:00.937277 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a587d31-bf72-4f86-b95f-5555e1d9aabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:00.938791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a587d31-bf72-4f86-b95f-5555e1d9aabe None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:00.939280 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf9f32-f997-4e42-ac38-161939ed0f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:00.939428 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a587d31-bf72-4f86-b95f-5555e1d9aabe None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:00.939900 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a587d31-bf72-4f86-b95f-5555e1d9aabe None admin] PATH_INFO: `/v3/users/nova` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:00.940021 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf9f32-f997-4e42-ac38-161939ed0f6f None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:00.940219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf9f32-f997-4e42-ac38-161939ed0f6f None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:00.940495 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf9f32-f997-4e42-ac38-161939ed0f6f None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:00.941224 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf9f32-f997-4e42-ac38-161939ed0f6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:00.941539 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf9f32-f997-4e42-ac38-161939ed0f6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:00.942966 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf9f32-f997-4e42-ac38-161939ed0f6f None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:00.950653 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a587d31-bf72-4f86-b95f-5555e1d9aabe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:00.952496 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a587d31-bf72-4f86-b95f-5555e1d9aabe None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:00.959440 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 372/749] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:00.959440 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf9f32-f997-4e42-ac38-161939ed0f6f None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-7a587d31-bf72-4f86-b95f-5555e1d9aabe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:05:00.979038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:00.980208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:00.981219 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:00.981219 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:00.981219 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:05:00.981219 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:05:00.981219 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:05:00.981219 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:05:00.981219 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:05:00.981219 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:05:00.981219 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:05:00.981219 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:00.988391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 378/750] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:05:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:00.989346 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a587d31-bf72-4f86-b95f-5555e1d9aabe None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:00.995075 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:01.077223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf9f32-f997-4e42-ac38-161939ed0f6f None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:01.114502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98662c4-92e7-464d-bde4-553c2876dc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:01.115093 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98662c4-92e7-464d-bde4-553c2876dc57 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:01.115368 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98662c4-92e7-464d-bde4-553c2876dc57 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:01.115629 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98662c4-92e7-464d-bde4-553c2876dc57 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:01.116464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98662c4-92e7-464d-bde4-553c2876dc57 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:01.116770 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98662c4-92e7-464d-bde4-553c2876dc57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:01.118202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98662c4-92e7-464d-bde4-553c2876dc57 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:01.138511 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 373/751] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 20:05:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:01.140997 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6939d0f-d717-4e9e-871f-7ead1439f547 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:01.141771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6939d0f-d717-4e9e-871f-7ead1439f547 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:01.142364 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6939d0f-d717-4e9e-871f-7ead1439f547 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:01.144440 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 374/752] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:01.145578 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6939d0f-d717-4e9e-871f-7ead1439f547 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:01.219713 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a587d31-bf72-4f86-b95f-5555e1d9aabe None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:01.244380 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:01.250156 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:01.257316 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22e03d-e26e-430f-b481-101e20a248d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:01.258134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22e03d-e26e-430f-b481-101e20a248d8 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:01.258444 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22e03d-e26e-430f-b481-101e20a248d8 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:01.258775 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22e03d-e26e-430f-b481-101e20a248d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:01.265076 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6939d0f-d717-4e9e-871f-7ead1439f547 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:01.266487 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22e03d-e26e-430f-b481-101e20a248d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:01.268056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22e03d-e26e-430f-b481-101e20a248d8 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:01.283319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-6d22e03d-e26e-430f-b481-101e20a248d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:01.283319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:01.283319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:01.283319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:01.283319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:01.283319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:01.283319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:01.283319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:01.283319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:01.283319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:01.283319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:01.283319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:01.283319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:05:01.283319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:05:01.283319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:01.284493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:01.285644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:05:01.285644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:05:01.285644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:05:01.285644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:05:01.285644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:05:01.285644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:05:01.285644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:01.285644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:01.286481 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 379/753] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:01.287065 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22e03d-e26e-430f-b481-101e20a248d8 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:01.310531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f6089-868b-481b-aeb9-fe15e2ae09a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:01.311304 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6089-868b-481b-aeb9-fe15e2ae09a1 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:01.311652 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6089-868b-481b-aeb9-fe15e2ae09a1 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:01.311994 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6089-868b-481b-aeb9-fe15e2ae09a1 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:01.312938 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f6089-868b-481b-aeb9-fe15e2ae09a1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:01.313246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f6089-868b-481b-aeb9-fe15e2ae09a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:01.314737 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f6089-868b-481b-aeb9-fe15e2ae09a1 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:01.334094 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 375/754] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:05:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:01.335493 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccacab1-9238-4ef5-a389-6009bea6300e None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:01.335783 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccacab1-9238-4ef5-a389-6009bea6300e None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:01.336058 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccacab1-9238-4ef5-a389-6009bea6300e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:01.354509 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-3ccacab1-9238-4ef5-a389-6009bea6300e None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:01.355761 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-3ccacab1-9238-4ef5-a389-6009bea6300e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:01.398095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22e03d-e26e-430f-b481-101e20a248d8 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:01.425325 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 376/755] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:01.431678 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccacab1-9238-4ef5-a389-6009bea6300e None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:01.435758 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe266fe2-2434-407b-8f37-2ac67d3171d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:01.436485 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe266fe2-2434-407b-8f37-2ac67d3171d3 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:01.436836 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe266fe2-2434-407b-8f37-2ac67d3171d3 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:01.437178 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe266fe2-2434-407b-8f37-2ac67d3171d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:01.487672 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:01.506656 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe266fe2-2434-407b-8f37-2ac67d3171d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:01.508117 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe266fe2-2434-407b-8f37-2ac67d3171d3 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:01.530607 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:01.563610 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccacab1-9238-4ef5-a389-6009bea6300e None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:01.606585 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c157b03-8fbe-472f-b53d-9c15675be84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:01.607313 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c157b03-8fbe-472f-b53d-9c15675be84a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:01.607560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c157b03-8fbe-472f-b53d-9c15675be84a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:01.607833 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c157b03-8fbe-472f-b53d-9c15675be84a None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:01.608642 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c157b03-8fbe-472f-b53d-9c15675be84a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:01.608954 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c157b03-8fbe-472f-b53d-9c15675be84a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:01.610451 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c157b03-8fbe-472f-b53d-9c15675be84a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:01.611240 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 380/756] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:05:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:05:01.614165 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe266fe2-2434-407b-8f37-2ac67d3171d3 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:01.626219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 377/757] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:05:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:01.627126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c157b03-8fbe-472f-b53d-9c15675be84a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:01.704715 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:01.724678 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe266fe2-2434-407b-8f37-2ac67d3171d3 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:01.736886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c157b03-8fbe-472f-b53d-9c15675be84a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:01.760904 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76682887-f110-4f6c-8db5-44aedbcafa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:01.761720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76682887-f110-4f6c-8db5-44aedbcafa25 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:01.762003 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76682887-f110-4f6c-8db5-44aedbcafa25 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:01.762303 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76682887-f110-4f6c-8db5-44aedbcafa25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:01.771983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60353a46-65db-4aeb-85c5-523040e2a6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:01.772802 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60353a46-65db-4aeb-85c5-523040e2a6f9 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:01.773084 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60353a46-65db-4aeb-85c5-523040e2a6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:01.773341 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76682887-f110-4f6c-8db5-44aedbcafa25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:01.773448 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60353a46-65db-4aeb-85c5-523040e2a6f9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:01.774000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60353a46-65db-4aeb-85c5-523040e2a6f9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:01.774838 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76682887-f110-4f6c-8db5-44aedbcafa25 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:01.775472 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60353a46-65db-4aeb-85c5-523040e2a6f9 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:01.777654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-60353a46-65db-4aeb-85c5-523040e2a6f9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:01.789311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:01.795697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:01.796648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 378/758] 10.222.0.84 () {56 vars in 1129 bytes} [Thu May 19 20:05:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:01.797542 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60353a46-65db-4aeb-85c5-523040e2a6f9 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:01.797615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-76682887-f110-4f6c-8db5-44aedbcafa25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:01.797615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:01.797615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:01.797615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:01.797615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:01.797615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:01.797615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:01.797615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:01.797615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:01.797615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:01.797615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:01.797615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:01.797615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:01.797615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:01.797615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:01.803336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:01.804278 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:01.804278 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:01.804278 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:01.804278 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:01.804278 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 381/759] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:01.804278 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76682887-f110-4f6c-8db5-44aedbcafa25 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:01.911879 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76682887-f110-4f6c-8db5-44aedbcafa25 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:01.919737 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60353a46-65db-4aeb-85c5-523040e2a6f9 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:01.948350 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f9551-d1cb-4b45-9ddf-9a569eb10464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:01.949281 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f9551-d1cb-4b45-9ddf-9a569eb10464 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:01.949773 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f9551-d1cb-4b45-9ddf-9a569eb10464 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:01.950143 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f9551-d1cb-4b45-9ddf-9a569eb10464 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:01.951111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f9551-d1cb-4b45-9ddf-9a569eb10464 None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=87b30300be1a41128ff88e12e260f63b) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:01.951578 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f9551-d1cb-4b45-9ddf-9a569eb10464 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:01.953058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f9551-d1cb-4b45-9ddf-9a569eb10464 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:01.954464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d607d-03a7-4cda-9bf6-1e3bb2415e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:01.955176 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:01.955294 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d607d-03a7-4cda-9bf6-1e3bb2415e89 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:01.955538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d607d-03a7-4cda-9bf6-1e3bb2415e89 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:01.956197 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d607d-03a7-4cda-9bf6-1e3bb2415e89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:01.960735 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 382/760] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:05:01 2022] GET /identity/v3/role_assignments?user.id=87b30300be1a41128ff88e12e260f63b&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:01.961622 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f9551-d1cb-4b45-9ddf-9a569eb10464 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:01.971594 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d607d-03a7-4cda-9bf6-1e3bb2415e89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:01.973735 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d607d-03a7-4cda-9bf6-1e3bb2415e89 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:01.993986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-e24d607d-03a7-4cda-9bf6-1e3bb2415e89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:01.993986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:01.993986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:01.993986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:01.993986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:01.993986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:01.993986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:01.993986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:01.993986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:01.993986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:01.993986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:01.993986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:01.993986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:01.993986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:01.993986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:01.995625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:01.996605 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:01.996605 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:01.996605 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:01.996605 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:01.996605 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 379/761] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:01.996605 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d607d-03a7-4cda-9bf6-1e3bb2415e89 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:02.055279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:02.111578 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f9551-d1cb-4b45-9ddf-9a569eb10464 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:02.125312 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d607d-03a7-4cda-9bf6-1e3bb2415e89 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:02.160350 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e9372-4605-4972-967c-de8b7474ae78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:02.161383 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e9372-4605-4972-967c-de8b7474ae78 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:02.162286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e9372-4605-4972-967c-de8b7474ae78 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:02.162657 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e9372-4605-4972-967c-de8b7474ae78 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:02.163752 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e9372-4605-4972-967c-de8b7474ae78 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:02.164379 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e9372-4605-4972-967c-de8b7474ae78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:02.166995 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e9372-4605-4972-967c-de8b7474ae78 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:02.171034 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24897e9f-b3fa-406b-8c71-c8de996958c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:02.171913 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24897e9f-b3fa-406b-8c71-c8de996958c4 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:02.172202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24897e9f-b3fa-406b-8c71-c8de996958c4 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:02.172537 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24897e9f-b3fa-406b-8c71-c8de996958c4 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:02.172980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 383/762] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 20:05:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:02.173931 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e9372-4605-4972-967c-de8b7474ae78 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:02.179218 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24897e9f-b3fa-406b-8c71-c8de996958c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:02.179878 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24897e9f-b3fa-406b-8c71-c8de996958c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:02.181984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24897e9f-b3fa-406b-8c71-c8de996958c4 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:02.201195 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 380/763] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:02.202144 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24897e9f-b3fa-406b-8c71-c8de996958c4 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:02.263312 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:02.308495 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e9372-4605-4972-967c-de8b7474ae78 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:02.309156 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24897e9f-b3fa-406b-8c71-c8de996958c4 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:02.341826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:02.347990 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c243d-9722-4b28-8644-da9636400ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:02.349666 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c243d-9722-4b28-8644-da9636400ea5 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:02.350465 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3f0a5-85cd-4318-935a-c863f73cd656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:02.350875 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c243d-9722-4b28-8644-da9636400ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:02.351619 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3f0a5-85cd-4318-935a-c863f73cd656 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:02.351988 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c243d-9722-4b28-8644-da9636400ea5 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:02.352821 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c243d-9722-4b28-8644-da9636400ea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:02.353201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3f0a5-85cd-4318-935a-c863f73cd656 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:02.353767 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3f0a5-85cd-4318-935a-c863f73cd656 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:02.354200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c243d-9722-4b28-8644-da9636400ea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:02.355040 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3f0a5-85cd-4318-935a-c863f73cd656 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:02.356541 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c243d-9722-4b28-8644-da9636400ea5 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:02.356939 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3f0a5-85cd-4318-935a-c863f73cd656 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:02.358486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3f0a5-85cd-4318-935a-c863f73cd656 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:02.369548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 381/764] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 20:05:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:02.370751 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3f0a5-85cd-4318-935a-c863f73cd656 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:02.377982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 384/765] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:02.379121 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c243d-9722-4b28-8644-da9636400ea5 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:02.477986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3f0a5-85cd-4318-935a-c863f73cd656 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:02.490633 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c243d-9722-4b28-8644-da9636400ea5 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:02.515580 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d8c6c-34eb-4a5a-8168-76fa20d414d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:02.516442 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d8c6c-34eb-4a5a-8168-76fa20d414d1 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:02.516840 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d8c6c-34eb-4a5a-8168-76fa20d414d1 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:02.518600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d8c6c-34eb-4a5a-8168-76fa20d414d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:02.519352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:02.531950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d8c6c-34eb-4a5a-8168-76fa20d414d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:02.533816 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb077892-4ba5-454a-8e55-dd0a1634e45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:02.534239 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d8c6c-34eb-4a5a-8168-76fa20d414d1 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:02.534750 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb077892-4ba5-454a-8e55-dd0a1634e45d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:02.535133 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb077892-4ba5-454a-8e55-dd0a1634e45d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:02.535532 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb077892-4ba5-454a-8e55-dd0a1634e45d None admin] PATH_INFO: `/v3/endpoints` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:02.536464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb077892-4ba5-454a-8e55-dd0a1634e45d None admin] RBAC: Adding query filter params (interface=public, service_id=7dde62c243ae4e09be13329632373f46, region_id=RegionOne) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:02.536884 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb077892-4ba5-454a-8e55-dd0a1634e45d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:02.538657 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb077892-4ba5-454a-8e55-dd0a1634e45d None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:02.549381 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 385/766] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:05:02 2022] GET /identity/v3/endpoints?service_id=7dde62c243ae4e09be13329632373f46&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:02.550506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb077892-4ba5-454a-8e55-dd0a1634e45d None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:02.553201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-801d8c6c-34eb-4a5a-8168-76fa20d414d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:02.553201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:02.553201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:02.553201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:02.553201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:02.553201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:02.553201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:02.553201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:02.553201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:02.553201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:02.553201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:02.553201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:02.553201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:05:02.553201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:05:02.553201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:02.554449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:02.555651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:05:02.555651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:05:02.555651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:05:02.555651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:05:02.555651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:05:02.555651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:05:02.555651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:02.555651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:02.559726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 382/767] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:02.560837 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d8c6c-34eb-4a5a-8168-76fa20d414d1 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:02.616989 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:02.676840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb077892-4ba5-454a-8e55-dd0a1634e45d None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:02.710004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d8c6c-34eb-4a5a-8168-76fa20d414d1 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:02.714941 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac113e-fd33-4a80-a337-cef1295594cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:02.715885 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac113e-fd33-4a80-a337-cef1295594cc None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:02.716272 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac113e-fd33-4a80-a337-cef1295594cc None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:02.716653 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac113e-fd33-4a80-a337-cef1295594cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:02.724726 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac113e-fd33-4a80-a337-cef1295594cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:02.726194 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac113e-fd33-4a80-a337-cef1295594cc None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:02.740878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-d7ac113e-fd33-4a80-a337-cef1295594cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:02.740878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:02.740878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:02.740878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:02.740878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:02.740878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:02.740878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:02.740878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:02.740878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:02.740878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:02.740878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:02.740878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:02.740878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:05:02.740878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:05:02.740878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:02.741851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:02.742840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:05:02.742840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:05:02.742840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:05:02.742840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:05:02.742840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:05:02.742840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:05:02.742840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:02.742840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:02.742840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 386/768] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:02.744361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac113e-fd33-4a80-a337-cef1295594cc None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:02.754376 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db8028-4eb3-49a2-9a21-f85ec0df6743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:02.755056 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db8028-4eb3-49a2-9a21-f85ec0df6743 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:02.755394 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db8028-4eb3-49a2-9a21-f85ec0df6743 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:02.755673 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db8028-4eb3-49a2-9a21-f85ec0df6743 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:02.756363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db8028-4eb3-49a2-9a21-f85ec0df6743 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:02.757755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db8028-4eb3-49a2-9a21-f85ec0df6743 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:02.770638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 383/769] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:05:02 2022] GET /identity/v3/services => generated 772 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:02.772837 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5697ef-860e-4502-81fb-702a70f0f2bc None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:02.773125 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5697ef-860e-4502-81fb-702a70f0f2bc None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:02.773568 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5697ef-860e-4502-81fb-702a70f0f2bc None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:02.777284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 384/770] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:02.777284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5697ef-860e-4502-81fb-702a70f0f2bc None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:02.814430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:02.892378 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac113e-fd33-4a80-a337-cef1295594cc None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:02.897525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:02.902785 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5697ef-860e-4502-81fb-702a70f0f2bc None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:02.935747 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e06725-e4d5-4acc-a578-0f9584753b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:02.936876 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e06725-e4d5-4acc-a578-0f9584753b1f None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:02.937317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e06725-e4d5-4acc-a578-0f9584753b1f None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:02.937784 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e06725-e4d5-4acc-a578-0f9584753b1f None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:02.938804 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e06725-e4d5-4acc-a578-0f9584753b1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:02.939277 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e06725-e4d5-4acc-a578-0f9584753b1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:02.940886 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e06725-e4d5-4acc-a578-0f9584753b1f None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:02.942237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10c20d-22dc-4fa5-8193-cf1ca6307095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:02.943001 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10c20d-22dc-4fa5-8193-cf1ca6307095 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:02.943464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10c20d-22dc-4fa5-8193-cf1ca6307095 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:02.943627 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10c20d-22dc-4fa5-8193-cf1ca6307095 None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:02.944522 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10c20d-22dc-4fa5-8193-cf1ca6307095 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:02.944891 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10c20d-22dc-4fa5-8193-cf1ca6307095 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:02.946523 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10c20d-22dc-4fa5-8193-cf1ca6307095 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:02.956366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 387/771] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:05:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:02.958081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221fa25-22c6-42f6-ad9b-9fcf3d2b0293 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:02.958578 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221fa25-22c6-42f6-ad9b-9fcf3d2b0293 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:02.959045 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221fa25-22c6-42f6-ad9b-9fcf3d2b0293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:02.966285 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 385/772] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:05:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:02.966285 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10c20d-22dc-4fa5-8193-cf1ca6307095 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:02.982837 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-e221fa25-22c6-42f6-ad9b-9fcf3d2b0293 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:02.983932 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-e221fa25-22c6-42f6-ad9b-9fcf3d2b0293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:03.062398 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 388/773] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:03.063709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221fa25-22c6-42f6-ad9b-9fcf3d2b0293 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:03.084545 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10c20d-22dc-4fa5-8193-cf1ca6307095 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:03.126645 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:03.132234 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bea245-63d1-4085-9fa4-4b7e0368c16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:03.133291 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bea245-63d1-4085-9fa4-4b7e0368c16e None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:03.133585 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bea245-63d1-4085-9fa4-4b7e0368c16e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:03.133868 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bea245-63d1-4085-9fa4-4b7e0368c16e None admin] PATH_INFO: `/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/378b046698c04b7b88194a9ffe5b045a/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:03.136396 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bea245-63d1-4085-9fa4-4b7e0368c16e None admin] RBAC: Authorizing `identity:create_grant(project_id=ba22fdc274b74521847e5fceb2d307b1, user_id=378b046698c04b7b88194a9ffe5b045a, role_id=b34e4ad4fe7b4a60bd2fc669e2b31a66)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:03.137879 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bea245-63d1-4085-9fa4-4b7e0368c16e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:03.153446 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 386/774] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:05:02 2022] PUT /identity/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/378b046698c04b7b88194a9ffe5b045a/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:05:03.154460 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bea245-63d1-4085-9fa4-4b7e0368c16e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:03.179647 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:03.190164 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221fa25-22c6-42f6-ad9b-9fcf3d2b0293 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:03.228755 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acae879-3d88-43cb-b11e-78ddf58807f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:03.229951 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acae879-3d88-43cb-b11e-78ddf58807f5 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:03.230477 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acae879-3d88-43cb-b11e-78ddf58807f5 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:03.230982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acae879-3d88-43cb-b11e-78ddf58807f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:03.232039 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acae879-3d88-43cb-b11e-78ddf58807f5 None admin] RBAC: Adding query filter params (role.id=bdaf8732b34c43a38465efde7e18a060, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=bdf915a5e0aa42a89e575ce1e8a65e2a) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:03.232556 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acae879-3d88-43cb-b11e-78ddf58807f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:03.234210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acae879-3d88-43cb-b11e-78ddf58807f5 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:03.240941 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 389/775] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:05:03 2022] GET /identity/v3/role_assignments?user.id=bdf915a5e0aa42a89e575ce1e8a65e2a&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=bdaf8732b34c43a38465efde7e18a060 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:03.287317 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bea245-63d1-4085-9fa4-4b7e0368c16e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:03.330594 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9128270-dfed-480e-8c34-d9e1e769d32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:03.331408 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9128270-dfed-480e-8c34-d9e1e769d32d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:03.331690 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9128270-dfed-480e-8c34-d9e1e769d32d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:03.332016 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9128270-dfed-480e-8c34-d9e1e769d32d None admin] PATH_INFO: `/v3/services/network` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:03.332701 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9128270-dfed-480e-8c34-d9e1e769d32d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:03.334152 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9128270-dfed-480e-8c34-d9e1e769d32d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-d9128270-dfed-480e-8c34-d9e1e769d32d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:03.348237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:03.349397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:03.350848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:03.350848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:03.350848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:03.350848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:03.350848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:03.350848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:03.350848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:03.350848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:03.350848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:03.350848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:03.350848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:03.350848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:03.350848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:05:03.350848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:03.350848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 387/776] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:03 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:03.358630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acae879-3d88-43cb-b11e-78ddf58807f5 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:03.491030 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:03.494627 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acae879-3d88-43cb-b11e-78ddf58807f5 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:03.529961 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da483cf-d6f7-41de-81ac-be3eaacaa4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:03.530666 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da483cf-d6f7-41de-81ac-be3eaacaa4f5 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:03.530944 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da483cf-d6f7-41de-81ac-be3eaacaa4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:03.531217 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da483cf-d6f7-41de-81ac-be3eaacaa4f5 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:03.531934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da483cf-d6f7-41de-81ac-be3eaacaa4f5 None admin] RBAC: Adding query filter params (name=network) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:03.532238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da483cf-d6f7-41de-81ac-be3eaacaa4f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:03.533608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da483cf-d6f7-41de-81ac-be3eaacaa4f5 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:03.541553 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 390/777] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:05:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:03.545472 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9128270-dfed-480e-8c34-d9e1e769d32d None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:03.665629 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:03.689913 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9128270-dfed-480e-8c34-d9e1e769d32d None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:03.741598 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1b586-e730-4cc8-9a03-3cba81e4e097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:03.742780 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1b586-e730-4cc8-9a03-3cba81e4e097 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:03.743275 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1b586-e730-4cc8-9a03-3cba81e4e097 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:03.743749 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1b586-e730-4cc8-9a03-3cba81e4e097 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:03.745156 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1b586-e730-4cc8-9a03-3cba81e4e097 None admin] RBAC: Adding query filter params (type=network) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:03.745697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1b586-e730-4cc8-9a03-3cba81e4e097 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:03.748271 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1b586-e730-4cc8-9a03-3cba81e4e097 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:03.758909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 388/778] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:05:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:03.764659 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da483cf-d6f7-41de-81ac-be3eaacaa4f5 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:03.906141 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da483cf-d6f7-41de-81ac-be3eaacaa4f5 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:03.959194 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff38a4-ebe5-4de9-9f44-411a832739f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:03.959799 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff38a4-ebe5-4de9-9f44-411a832739f1 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:03.960124 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff38a4-ebe5-4de9-9f44-411a832739f1 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:03.960404 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff38a4-ebe5-4de9-9f44-411a832739f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:03.961193 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff38a4-ebe5-4de9-9f44-411a832739f1 None admin] RBAC: Adding query filter params (interface=public, service_id=02f58a5d1f084c468caa078c6f03b5db, region_id=RegionOne) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:03.961505 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff38a4-ebe5-4de9-9f44-411a832739f1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:03.963070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff38a4-ebe5-4de9-9f44-411a832739f1 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:03.971383 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 391/779] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:05:03 2022] GET /identity/v3/endpoints?service_id=02f58a5d1f084c468caa078c6f03b5db&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:03.975889 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1b586-e730-4cc8-9a03-3cba81e4e097 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:04.126521 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1b586-e730-4cc8-9a03-3cba81e4e097 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:04.175492 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346e891-b7f1-4ecd-a54a-2e669d61e539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:04.176566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346e891-b7f1-4ecd-a54a-2e669d61e539 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:04.177052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346e891-b7f1-4ecd-a54a-2e669d61e539 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:04.177593 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346e891-b7f1-4ecd-a54a-2e669d61e539 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:04.178641 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346e891-b7f1-4ecd-a54a-2e669d61e539 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:04.180413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346e891-b7f1-4ecd-a54a-2e669d61e539 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:04.191939 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 389/780] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:05:03 2022] GET /identity/v3/services => generated 772 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:04.379817 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8ad85-c800-4231-bb5a-11e8b25e3773 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:04.380358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8ad85-c800-4231-bb5a-11e8b25e3773 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:04.380358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8ad85-c800-4231-bb5a-11e8b25e3773 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:04.382031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 392/781] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:04.387839 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa7771-6c85-45fb-b2e5-928f59cab3c3 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:04.388150 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa7771-6c85-45fb-b2e5-928f59cab3c3 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:04.388395 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa7771-6c85-45fb-b2e5-928f59cab3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:04.411441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-5baa7771-6c85-45fb-b2e5-928f59cab3c3 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:04.412659 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-5baa7771-6c85-45fb-b2e5-928f59cab3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:04.416098 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:04.507480 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 390/782] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:04.531205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8ad85-c800-4231-bb5a-11e8b25e3773 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:04.562366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:04.648177 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8ad85-c800-4231-bb5a-11e8b25e3773 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:04.686603 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a3771-f4ff-4d8e-a79d-2e695a668c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:04.687468 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a3771-f4ff-4d8e-a79d-2e695a668c2d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:04.687750 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a3771-f4ff-4d8e-a79d-2e695a668c2d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:04.688114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a3771-f4ff-4d8e-a79d-2e695a668c2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:04.701167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a3771-f4ff-4d8e-a79d-2e695a668c2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-790a3771-f4ff-4d8e-a79d-2e695a668c2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:05:04.709052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:04.710064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:04.710996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:04.710996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:04.710996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:04.710996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:04.710996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:05:04.710996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:05:04.710996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:05:04.710996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:05:04.710996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:05:04.710996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:05:04.710996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:05:04.710996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:04.710996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 393/783] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:05:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:04.716617 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa7771-6c85-45fb-b2e5-928f59cab3c3 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:04.856483 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa7771-6c85-45fb-b2e5-928f59cab3c3 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:04.906010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc32cdb-09bf-484c-b93c-cfbecd49421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:04.906521 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc32cdb-09bf-484c-b93c-cfbecd49421e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:04.906807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc32cdb-09bf-484c-b93c-cfbecd49421e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:04.907082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc32cdb-09bf-484c-b93c-cfbecd49421e None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:04.907837 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc32cdb-09bf-484c-b93c-cfbecd49421e None admin] RBAC: Adding query filter params (name=admin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:04.908145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc32cdb-09bf-484c-b93c-cfbecd49421e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:04.909545 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc32cdb-09bf-484c-b93c-cfbecd49421e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:04.920985 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 391/784] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:05:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:04.931583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a3771-f4ff-4d8e-a79d-2e695a668c2d None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:05.071278 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:05.085953 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a3771-f4ff-4d8e-a79d-2e695a668c2d None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:05.136779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842fde1-6eed-4de9-aa60-785ec6766f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:05.137601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842fde1-6eed-4de9-aa60-785ec6766f0a None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:05.137884 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842fde1-6eed-4de9-aa60-785ec6766f0a None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:05.138158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842fde1-6eed-4de9-aa60-785ec6766f0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:05.155613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842fde1-6eed-4de9-aa60-785ec6766f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:05.157152 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842fde1-6eed-4de9-aa60-785ec6766f0a None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:05.181516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-6842fde1-6eed-4de9-aa60-785ec6766f0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:05.181516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:05.181516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:05.181516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:05.181516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:05.181516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:05.181516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:05.181516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:05.181516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:05.181516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:05.181516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:05.181516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:05.181516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:05.181516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:05.181516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:05.182729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:05.183706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:05.183706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:05.183706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:05.183706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:05.184034 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 394/785] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:05.190103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc32cdb-09bf-484c-b93c-cfbecd49421e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:05.193138 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:05.326399 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc32cdb-09bf-484c-b93c-cfbecd49421e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:05.371631 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16821b0-ec67-4c38-9a39-4585b71493a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:05.372328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16821b0-ec67-4c38-9a39-4585b71493a3 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:05.372616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16821b0-ec67-4c38-9a39-4585b71493a3 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:05.372895 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16821b0-ec67-4c38-9a39-4585b71493a3 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:05.373640 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16821b0-ec67-4c38-9a39-4585b71493a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:05.373953 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16821b0-ec67-4c38-9a39-4585b71493a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:05.375392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16821b0-ec67-4c38-9a39-4585b71493a3 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:05.393154 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 392/786] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:05.398770 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842fde1-6eed-4de9-aa60-785ec6766f0a None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:05.518676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842fde1-6eed-4de9-aa60-785ec6766f0a None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:05.559004 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149845b-4d3b-41f7-84d9-b218b4ce205a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:05.559899 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149845b-4d3b-41f7-84d9-b218b4ce205a None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:05.560187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149845b-4d3b-41f7-84d9-b218b4ce205a None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:05.560469 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149845b-4d3b-41f7-84d9-b218b4ce205a None admin] PATH_INFO: `/v3/users/swift` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:05.579690 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149845b-4d3b-41f7-84d9-b218b4ce205a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:05.581256 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149845b-4d3b-41f7-84d9-b218b4ce205a None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:05.587612 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-0149845b-4d3b-41f7-84d9-b218b4ce205a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:05:05.606156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:05.607306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:05.608336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:05.608336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:05.608336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:05:05.608336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:05:05.608336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:05:05.608336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:05:05.608336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:05:05.608336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:05:05.608336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:05:05.608336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:05.611200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 395/787] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:05:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:05.617036 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16821b0-ec67-4c38-9a39-4585b71493a3 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:05.706427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:05.749042 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16821b0-ec67-4c38-9a39-4585b71493a3 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:05.765145 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba45bc-ca16-44dc-b997-5cbbdbad7655 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:05.765483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba45bc-ca16-44dc-b997-5cbbdbad7655 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:05.765708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba45bc-ca16-44dc-b997-5cbbdbad7655 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:05.767769 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 396/788] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:05:05.779650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a028d0-1ab2-4e8b-bdaf-625f66e79105 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:05.780006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a028d0-1ab2-4e8b-bdaf-625f66e79105 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:05.780384 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a028d0-1ab2-4e8b-bdaf-625f66e79105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:05.809246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66238e-fce6-4905-a26d-b41d95b8abfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:05.809938 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66238e-fce6-4905-a26d-b41d95b8abfc None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:05.810222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66238e-fce6-4905-a26d-b41d95b8abfc None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:05.810523 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66238e-fce6-4905-a26d-b41d95b8abfc None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:05.811382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66238e-fce6-4905-a26d-b41d95b8abfc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:05.811694 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66238e-fce6-4905-a26d-b41d95b8abfc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:05.813158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66238e-fce6-4905-a26d-b41d95b8abfc None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:05.815506 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-f6a028d0-1ab2-4e8b-bdaf-625f66e79105 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:05.816774 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-f6a028d0-1ab2-4e8b-bdaf-625f66e79105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:05.845759 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 393/789] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 20:05:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:05.855538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66238e-fce6-4905-a26d-b41d95b8abfc None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:05.908410 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 397/790] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:05.942764 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a028d0-1ab2-4e8b-bdaf-625f66e79105 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:06.062174 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a028d0-1ab2-4e8b-bdaf-625f66e79105 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:06.098033 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:06.101722 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548dc4b-1d5e-40c1-b05d-04f71c1aa9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:06.102583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548dc4b-1d5e-40c1-b05d-04f71c1aa9fb None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:06.102893 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548dc4b-1d5e-40c1-b05d-04f71c1aa9fb None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:06.103527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548dc4b-1d5e-40c1-b05d-04f71c1aa9fb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:06.104261 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548dc4b-1d5e-40c1-b05d-04f71c1aa9fb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:06.106169 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548dc4b-1d5e-40c1-b05d-04f71c1aa9fb None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-e548dc4b-1d5e-40c1-b05d-04f71c1aa9fb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:06.122084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:06.123150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:06.124075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 398/791] 10.222.0.84 () {56 vars in 1129 bytes} [Thu May 19 20:05:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:06.131693 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548dc4b-1d5e-40c1-b05d-04f71c1aa9fb None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:06.156006 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66238e-fce6-4905-a26d-b41d95b8abfc None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:06.195172 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:06.215509 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93632ad8-99a5-4fc7-b497-4b07ec9957e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:06.216872 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93632ad8-99a5-4fc7-b497-4b07ec9957e8 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:06.217257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93632ad8-99a5-4fc7-b497-4b07ec9957e8 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:06.217655 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93632ad8-99a5-4fc7-b497-4b07ec9957e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:06.232441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93632ad8-99a5-4fc7-b497-4b07ec9957e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:06.234085 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93632ad8-99a5-4fc7-b497-4b07ec9957e8 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:06.252430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-93632ad8-99a5-4fc7-b497-4b07ec9957e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:06.252430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:06.252430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:06.252430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:06.252430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:06.252430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:06.252430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:06.252430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:06.252430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:06.252430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:06.252430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:06.252430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:06.252430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:06.252430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:06.252430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:06.253444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:06.254997 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:06.254997 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:06.254997 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:06.254997 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:06.254997 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 394/792] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:06.255326 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff67f00-8a83-4d6c-99be-5153cec39215 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:06.255565 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff67f00-8a83-4d6c-99be-5153cec39215 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:06.255850 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff67f00-8a83-4d6c-99be-5153cec39215 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:06.257205 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 395/793] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:06.258097 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbaa19-a39a-4d78-b696-14407e86ad82 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:06.258409 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbaa19-a39a-4d78-b696-14407e86ad82 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:06.258687 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbaa19-a39a-4d78-b696-14407e86ad82 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:06.262296 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 396/794] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:06.262296 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbaa19-a39a-4d78-b696-14407e86ad82 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:06.278726 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548dc4b-1d5e-40c1-b05d-04f71c1aa9fb None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:06.314957 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15658cc9-6c75-46c6-a021-946e91dcdd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:06.315552 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15658cc9-6c75-46c6-a021-946e91dcdd41 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:06.315832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15658cc9-6c75-46c6-a021-946e91dcdd41 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:06.316117 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15658cc9-6c75-46c6-a021-946e91dcdd41 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:06.316824 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15658cc9-6c75-46c6-a021-946e91dcdd41 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:06.317124 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15658cc9-6c75-46c6-a021-946e91dcdd41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:06.318505 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15658cc9-6c75-46c6-a021-946e91dcdd41 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:06.328351 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 399/795] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 20:05:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:06.328351 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ee67b-0f86-4688-87f6-369f10b3bb3e None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:06.328351 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ee67b-0f86-4688-87f6-369f10b3bb3e None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:06.328351 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ee67b-0f86-4688-87f6-369f10b3bb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:06.344628 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-dc5ee67b-0f86-4688-87f6-369f10b3bb3e None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:06.345880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-dc5ee67b-0f86-4688-87f6-369f10b3bb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:06.370040 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbaa19-a39a-4d78-b696-14407e86ad82 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:06.401353 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:06.412060 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee1fbc-fcd9-4a50-9786-446c2ff724be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:06.412735 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee1fbc-fcd9-4a50-9786-446c2ff724be None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:06.413013 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee1fbc-fcd9-4a50-9786-446c2ff724be None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:06.413284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee1fbc-fcd9-4a50-9786-446c2ff724be None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:06.413999 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee1fbc-fcd9-4a50-9786-446c2ff724be None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:06.414322 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee1fbc-fcd9-4a50-9786-446c2ff724be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:06.415698 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee1fbc-fcd9-4a50-9786-446c2ff724be None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:06.416224 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 400/796] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:06.417791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29520e-33ed-41a3-b920-66cb136e5fb0 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:06.418081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29520e-33ed-41a3-b920-66cb136e5fb0 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:06.418385 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29520e-33ed-41a3-b920-66cb136e5fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:06.436779 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 397/797] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:06.437412 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-cc29520e-33ed-41a3-b920-66cb136e5fb0 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:06.438288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee1fbc-fcd9-4a50-9786-446c2ff724be None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:06.438682 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-cc29520e-33ed-41a3-b920-66cb136e5fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:06.476031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:06.515670 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 401/798] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:06.516900 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29520e-33ed-41a3-b920-66cb136e5fb0 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:06.570640 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee1fbc-fcd9-4a50-9786-446c2ff724be None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:06.608736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173f2ac-03ab-41bf-862f-e8bd3b8b4524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:06.609436 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173f2ac-03ab-41bf-862f-e8bd3b8b4524 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:06.609735 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173f2ac-03ab-41bf-862f-e8bd3b8b4524 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:06.610009 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173f2ac-03ab-41bf-862f-e8bd3b8b4524 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:06.610730 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173f2ac-03ab-41bf-862f-e8bd3b8b4524 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:06.611040 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173f2ac-03ab-41bf-862f-e8bd3b8b4524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:06.612685 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173f2ac-03ab-41bf-862f-e8bd3b8b4524 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:06.618431 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 398/799] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 20:05:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:06.619542 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173f2ac-03ab-41bf-862f-e8bd3b8b4524 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:06.633617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29520e-33ed-41a3-b920-66cb136e5fb0 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:06.655636 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:06.674393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef69eb0-a7c3-43b6-b01c-9de364970f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:06.675246 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef69eb0-a7c3-43b6-b01c-9de364970f51 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:06.675536 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef69eb0-a7c3-43b6-b01c-9de364970f51 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:06.675813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef69eb0-a7c3-43b6-b01c-9de364970f51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:06.684056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef69eb0-a7c3-43b6-b01c-9de364970f51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:06.685704 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef69eb0-a7c3-43b6-b01c-9de364970f51 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:06.700387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-cef69eb0-a7c3-43b6-b01c-9de364970f51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:06.700387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:06.700387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:06.700387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:06.700387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:06.700387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:06.700387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:06.700387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:06.700387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:06.700387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:06.700387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:06.700387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:06.700387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:05:06.700387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:05:06.700387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:06.701393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:06.703028 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:05:06.703028 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:05:06.703028 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:05:06.703028 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:05:06.703028 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:05:06.703028 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:05:06.703028 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:06.703028 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:06.703028 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 402/800] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:06.703028 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef69eb0-a7c3-43b6-b01c-9de364970f51 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:06.726201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:06.733836 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173f2ac-03ab-41bf-862f-e8bd3b8b4524 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:06.773141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6339c-68a1-46f8-9530-f89561b8eee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:06.774121 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6339c-68a1-46f8-9530-f89561b8eee7 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:06.774494 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6339c-68a1-46f8-9530-f89561b8eee7 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:06.774849 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6339c-68a1-46f8-9530-f89561b8eee7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:06.787643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6339c-68a1-46f8-9530-f89561b8eee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-d6c6339c-68a1-46f8-9530-f89561b8eee7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:05:06.795536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:06.796558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:06.797940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:06.797940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:06.797940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:06.797940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:06.797940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:05:06.797940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:05:06.797940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:05:06.797940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:05:06.797940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:05:06.797940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:05:06.797940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:05:06.797940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:06.797940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 399/801] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:05:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:06.797940 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6339c-68a1-46f8-9530-f89561b8eee7 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:06.815116 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef69eb0-a7c3-43b6-b01c-9de364970f51 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:06.858193 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06148e87-be53-4863-b653-4140862bc382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:06.859458 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06148e87-be53-4863-b653-4140862bc382 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:06.859742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06148e87-be53-4863-b653-4140862bc382 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:06.860023 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06148e87-be53-4863-b653-4140862bc382 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:06.872912 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06148e87-be53-4863-b653-4140862bc382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-06148e87-be53-4863-b653-4140862bc382 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:05:06.881275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:06.882295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:06.883210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:06.883210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:06.883210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:06.883210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:06.883210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:05:06.883210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:05:06.883210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:05:06.883210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:05:06.883210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:05:06.883210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:05:06.883210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:05:06.883210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:06.883210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 403/802] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:05:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:06.883975 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06148e87-be53-4863-b653-4140862bc382 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:06.917343 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6339c-68a1-46f8-9530-f89561b8eee7 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:06.931435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:06.954705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5a96d-8eed-45ce-8c3e-faf26379706e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:06.955370 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5a96d-8eed-45ce-8c3e-faf26379706e None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:06.955652 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5a96d-8eed-45ce-8c3e-faf26379706e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:06.955928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5a96d-8eed-45ce-8c3e-faf26379706e None admin] PATH_INFO: `/v3/endpoints` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:06.956675 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c5a96d-8eed-45ce-8c3e-faf26379706e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:06.958064 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c5a96d-8eed-45ce-8c3e-faf26379706e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:06.980116 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 400/803] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:05:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:05:06.980990 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5a96d-8eed-45ce-8c3e-faf26379706e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:07.005490 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:07.014721 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06148e87-be53-4863-b653-4140862bc382 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:07.068742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf0a3f-2384-4c03-ab05-04688a615505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:07.069434 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf0a3f-2384-4c03-ab05-04688a615505 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:07.069719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf0a3f-2384-4c03-ab05-04688a615505 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:07.069996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf0a3f-2384-4c03-ab05-04688a615505 None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:07.070846 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf0a3f-2384-4c03-ab05-04688a615505 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:07.071149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf0a3f-2384-4c03-ab05-04688a615505 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:07.072677 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf0a3f-2384-4c03-ab05-04688a615505 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:07.091447 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 404/804] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:05:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:07.091447 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf0a3f-2384-4c03-ab05-04688a615505 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:07.118658 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5a96d-8eed-45ce-8c3e-faf26379706e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:07.157750 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34aea90-f4cf-46c3-bcc1-82840ab11b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:07.158493 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34aea90-f4cf-46c3-bcc1-82840ab11b47 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:07.158778 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34aea90-f4cf-46c3-bcc1-82840ab11b47 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:07.159056 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34aea90-f4cf-46c3-bcc1-82840ab11b47 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:07.159828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34aea90-f4cf-46c3-bcc1-82840ab11b47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:07.160134 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34aea90-f4cf-46c3-bcc1-82840ab11b47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:07.161576 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34aea90-f4cf-46c3-bcc1-82840ab11b47 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:07.171631 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 401/805] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:05:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:07.173112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9c9e7-24b4-467c-84bc-6e70ac2a7788 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:07.173521 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9c9e7-24b4-467c-84bc-6e70ac2a7788 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:07.173937 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9c9e7-24b4-467c-84bc-6e70ac2a7788 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:07.178306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 402/806] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:07.178306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9c9e7-24b4-467c-84bc-6e70ac2a7788 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:07.213038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:07.224511 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf0a3f-2384-4c03-ab05-04688a615505 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:07.261035 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba4518-2ec8-4ef2-a69e-046da73cabaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:07.261719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba4518-2ec8-4ef2-a69e-046da73cabaf None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:07.262000 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba4518-2ec8-4ef2-a69e-046da73cabaf None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:07.262299 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba4518-2ec8-4ef2-a69e-046da73cabaf None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:07.263106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba4518-2ec8-4ef2-a69e-046da73cabaf None admin] RBAC: Adding query filter params (name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:07.263414 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba4518-2ec8-4ef2-a69e-046da73cabaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:07.264779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba4518-2ec8-4ef2-a69e-046da73cabaf None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:07.270586 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:07.277817 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 405/807] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:05:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:07.277817 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba4518-2ec8-4ef2-a69e-046da73cabaf None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:07.303413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9c9e7-24b4-467c-84bc-6e70ac2a7788 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:07.344074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8618393-8da6-4bc9-bb8d-e8c703f20f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:07.345605 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8618393-8da6-4bc9-bb8d-e8c703f20f2f None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:07.346126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8618393-8da6-4bc9-bb8d-e8c703f20f2f None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:07.346414 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8618393-8da6-4bc9-bb8d-e8c703f20f2f None admin] PATH_INFO: `/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/87b30300be1a41128ff88e12e260f63b/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:07.348422 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8618393-8da6-4bc9-bb8d-e8c703f20f2f None admin] RBAC: Authorizing `identity:create_grant(project_id=ba22fdc274b74521847e5fceb2d307b1, user_id=87b30300be1a41128ff88e12e260f63b, role_id=b34e4ad4fe7b4a60bd2fc669e2b31a66)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:07.349866 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8618393-8da6-4bc9-bb8d-e8c703f20f2f None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:07.365009 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 403/808] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:05:07 2022] PUT /identity/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/87b30300be1a41128ff88e12e260f63b/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:05:07.368577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12a8ee-0576-450b-b101-bec424e21fb5 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:07.369816 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12a8ee-0576-450b-b101-bec424e21fb5 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:07.370461 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12a8ee-0576-450b-b101-bec424e21fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:07.390058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba4518-2ec8-4ef2-a69e-046da73cabaf None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:07.397172 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-0c12a8ee-0576-450b-b101-bec424e21fb5 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:07.399681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-0c12a8ee-0576-450b-b101-bec424e21fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:07.431083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d15912-255e-49f7-bc35-278b055732a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:07.432477 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d15912-255e-49f7-bc35-278b055732a9 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:07.432477 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d15912-255e-49f7-bc35-278b055732a9 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:07.432689 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d15912-255e-49f7-bc35-278b055732a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:07.448571 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d15912-255e-49f7-bc35-278b055732a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:07.450055 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d15912-255e-49f7-bc35-278b055732a9 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:07.478418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-a9d15912-255e-49f7-bc35-278b055732a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:07.478418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:07.478418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:07.478418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:07.478418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:07.478418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:07.478418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:07.478418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:07.478418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:07.478418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:07.478418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:07.478418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:07.478418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:07.478418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:07.478418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:07.479839 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:07.480803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:07.480803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:07.480803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:07.480803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:07.483069 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 406/809] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:07.485087 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d15912-255e-49f7-bc35-278b055732a9 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:07.502821 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:07.507561 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 404/810] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:07.508691 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12a8ee-0576-450b-b101-bec424e21fb5 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:07.584332 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:07.629389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d15912-255e-49f7-bc35-278b055732a9 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:07.635980 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12a8ee-0576-450b-b101-bec424e21fb5 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:07.666920 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b13f96-6ab5-462a-b167-38eca1d1e7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:07.668049 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b13f96-6ab5-462a-b167-38eca1d1e7bd None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:07.668494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b13f96-6ab5-462a-b167-38eca1d1e7bd None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:07.668947 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b13f96-6ab5-462a-b167-38eca1d1e7bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:07.678916 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889c6a2-dcd7-4641-8b9e-7218a18667ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:07.680161 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889c6a2-dcd7-4641-8b9e-7218a18667ce None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:07.680510 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889c6a2-dcd7-4641-8b9e-7218a18667ce None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:07.681016 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889c6a2-dcd7-4641-8b9e-7218a18667ce None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:07.681965 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889c6a2-dcd7-4641-8b9e-7218a18667ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:07.682477 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b13f96-6ab5-462a-b167-38eca1d1e7bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:07.682605 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889c6a2-dcd7-4641-8b9e-7218a18667ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:07.684043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889c6a2-dcd7-4641-8b9e-7218a18667ce None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:07.684420 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b13f96-6ab5-462a-b167-38eca1d1e7bd None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:07.703888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-d3b13f96-6ab5-462a-b167-38eca1d1e7bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:07.703888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:07.703888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:07.703888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:07.703888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:07.703888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:07.703888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:07.703888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:07.703888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:07.703888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:07.703888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:07.703888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:07.703888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:07.703888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:07.703888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:07.705086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:07.706295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:07.706295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:07.706295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:07.706295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:07.708039 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 405/812] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:07.708207 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 407/812] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:07.708916 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889c6a2-dcd7-4641-8b9e-7218a18667ce None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:07.710593 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b13f96-6ab5-462a-b167-38eca1d1e7bd None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:07.779242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:07.830023 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b13f96-6ab5-462a-b167-38eca1d1e7bd None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:07.840078 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889c6a2-dcd7-4641-8b9e-7218a18667ce None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:07.869864 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770bb91-22f5-4fac-8b2e-fe8fe232c672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:07.871102 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:07.872053 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770bb91-22f5-4fac-8b2e-fe8fe232c672 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:07.872541 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770bb91-22f5-4fac-8b2e-fe8fe232c672 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:07.872993 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770bb91-22f5-4fac-8b2e-fe8fe232c672 None admin] PATH_INFO: `/v3/users/nova` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:07.883475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770bb91-22f5-4fac-8b2e-fe8fe232c672 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:07.883875 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b2ea3-986c-49f6-8d3a-457cb6bf69cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:07.884716 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b2ea3-986c-49f6-8d3a-457cb6bf69cb None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:07.885004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b2ea3-986c-49f6-8d3a-457cb6bf69cb None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:07.885284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b2ea3-986c-49f6-8d3a-457cb6bf69cb None admin] PATH_INFO: `/v3/services/network` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:07.885941 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b2ea3-986c-49f6-8d3a-457cb6bf69cb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:07.886496 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770bb91-22f5-4fac-8b2e-fe8fe232c672 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:07.887346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b2ea3-986c-49f6-8d3a-457cb6bf69cb None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-7770bb91-22f5-4fac-8b2e-fe8fe232c672 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:05:07.903934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:07.905226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:07.906405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:07.906405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:07.906405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:05:07.906405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:05:07.906405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:05:07.906405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:05:07.906405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:05:07.906405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:05:07.906405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:05:07.906405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-c02b2ea3-986c-49f6-8d3a-457cb6bf69cb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:07.906984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:07.908067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:07.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:07.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:07.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:07.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:07.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:07.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:07.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:07.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:07.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:07.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:07.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:07.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:07.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:05:07.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:07.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 406/813] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:07 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:07.909928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b2ea3-986c-49f6-8d3a-457cb6bf69cb None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:07.911633 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 408/814] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:05:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:07.913397 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770bb91-22f5-4fac-8b2e-fe8fe232c672 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:08.031209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b2ea3-986c-49f6-8d3a-457cb6bf69cb None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:08.050364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770bb91-22f5-4fac-8b2e-fe8fe232c672 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:08.076528 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e5227-e069-41c7-aff1-29a891d51b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:08.076528 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e5227-e069-41c7-aff1-29a891d51b8e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:08.076528 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e5227-e069-41c7-aff1-29a891d51b8e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:08.076528 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e5227-e069-41c7-aff1-29a891d51b8e None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:08.077062 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:08.078008 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e5227-e069-41c7-aff1-29a891d51b8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:08.078008 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e5227-e069-41c7-aff1-29a891d51b8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:08.080119 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e5227-e069-41c7-aff1-29a891d51b8e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:08.091190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f88ce5-e973-48fa-99b2-9343b8861790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:08.091898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f88ce5-e973-48fa-99b2-9343b8861790 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:08.092181 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f88ce5-e973-48fa-99b2-9343b8861790 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:08.092453 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f88ce5-e973-48fa-99b2-9343b8861790 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:08.093170 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f88ce5-e973-48fa-99b2-9343b8861790 None admin] RBAC: Adding query filter params (name=network) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:08.093475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f88ce5-e973-48fa-99b2-9343b8861790 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:08.095197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f88ce5-e973-48fa-99b2-9343b8861790 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:08.098532 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 407/815] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:08.099534 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e5227-e069-41c7-aff1-29a891d51b8e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:08.100998 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 409/816] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:05:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:08.102582 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f88ce5-e973-48fa-99b2-9343b8861790 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:08.154445 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:08.210870 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f88ce5-e973-48fa-99b2-9343b8861790 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:08.214128 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e5227-e069-41c7-aff1-29a891d51b8e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:08.246812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc068c7d-5fd6-4b7f-bf5e-c872fdd18a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:08.247792 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc068c7d-5fd6-4b7f-bf5e-c872fdd18a53 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:08.248171 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc068c7d-5fd6-4b7f-bf5e-c872fdd18a53 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:08.248654 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc068c7d-5fd6-4b7f-bf5e-c872fdd18a53 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:08.253721 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c4ab8-da9e-4b79-a5b4-904b146d8bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:08.254426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c4ab8-da9e-4b79-a5b4-904b146d8bbf None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:08.254709 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c4ab8-da9e-4b79-a5b4-904b146d8bbf None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:08.254983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c4ab8-da9e-4b79-a5b4-904b146d8bbf None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:08.255824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c4ab8-da9e-4b79-a5b4-904b146d8bbf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:08.256132 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c4ab8-da9e-4b79-a5b4-904b146d8bbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:08.257565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc068c7d-5fd6-4b7f-bf5e-c872fdd18a53 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:08.257707 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c4ab8-da9e-4b79-a5b4-904b146d8bbf None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:08.259230 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc068c7d-5fd6-4b7f-bf5e-c872fdd18a53 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-dc068c7d-5fd6-4b7f-bf5e-c872fdd18a53 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:05:08.275200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:08.276512 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:08.277760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:08.277760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:08.277760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:05:08.277760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:05:08.277760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:05:08.277760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:05:08.277760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:05:08.277760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:05:08.277760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:05:08.277760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:08.278484 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 408/817] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:05:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:08.278484 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c4ab8-da9e-4b79-a5b4-904b146d8bbf None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:08.281478 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 410/818] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:05:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:08.282756 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc068c7d-5fd6-4b7f-bf5e-c872fdd18a53 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:08.335346 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:08.389444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c4ab8-da9e-4b79-a5b4-904b146d8bbf None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:08.394962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc068c7d-5fd6-4b7f-bf5e-c872fdd18a53 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:08.398345 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:08.427482 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f8331-c6b7-4d69-a736-90d8f90880ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:08.428424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f8331-c6b7-4d69-a736-90d8f90880ef None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:08.428855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f8331-c6b7-4d69-a736-90d8f90880ef None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:08.429349 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f8331-c6b7-4d69-a736-90d8f90880ef None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:08.430427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f8331-c6b7-4d69-a736-90d8f90880ef None admin] RBAC: Adding query filter params (type=network) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:08.430947 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f8331-c6b7-4d69-a736-90d8f90880ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:08.432906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f8331-c6b7-4d69-a736-90d8f90880ef None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:08.433114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a977be-b12e-4a76-8b20-81bda52dcb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:08.433876 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a977be-b12e-4a76-8b20-81bda52dcb94 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:08.434253 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a977be-b12e-4a76-8b20-81bda52dcb94 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:08.434661 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a977be-b12e-4a76-8b20-81bda52dcb94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:08.439661 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 409/819] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:05:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:08.440607 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f8331-c6b7-4d69-a736-90d8f90880ef None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:08.509361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a977be-b12e-4a76-8b20-81bda52dcb94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:08.510958 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a977be-b12e-4a76-8b20-81bda52dcb94 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:08.544586 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f8331-c6b7-4d69-a736-90d8f90880ef None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:08.564870 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:08.584570 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102aee3-bf3c-477d-887a-e48a094951e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:08.585555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102aee3-bf3c-477d-887a-e48a094951e3 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:08.585986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102aee3-bf3c-477d-887a-e48a094951e3 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:08.586472 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102aee3-bf3c-477d-887a-e48a094951e3 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:08.587319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102aee3-bf3c-477d-887a-e48a094951e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:08.587733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102aee3-bf3c-477d-887a-e48a094951e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:08.589366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102aee3-bf3c-477d-887a-e48a094951e3 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:08.611854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 410/820] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 20:05:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:08.612733 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102aee3-bf3c-477d-887a-e48a094951e3 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:08.632266 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 411/821] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:05:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:05:08.633590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a977be-b12e-4a76-8b20-81bda52dcb94 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:08.641044 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:08.715559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102aee3-bf3c-477d-887a-e48a094951e3 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:08.743773 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a977be-b12e-4a76-8b20-81bda52dcb94 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:08.750870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04bea94-2f77-4131-97ec-b0029c965a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:08.751537 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bea94-2f77-4131-97ec-b0029c965a63 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:08.751816 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bea94-2f77-4131-97ec-b0029c965a63 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:08.752089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bea94-2f77-4131-97ec-b0029c965a63 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:08.752840 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04bea94-2f77-4131-97ec-b0029c965a63 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:08.754219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04bea94-2f77-4131-97ec-b0029c965a63 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:08.777175 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 411/822] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:05:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:05:08.778477 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04bea94-2f77-4131-97ec-b0029c965a63 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:08.802883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1832d-7300-4430-8a9b-87c02e0f0ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:08.804611 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1832d-7300-4430-8a9b-87c02e0f0ae9 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:08.804890 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1832d-7300-4430-8a9b-87c02e0f0ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:08.805162 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1832d-7300-4430-8a9b-87c02e0f0ae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:08.826987 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:08.828272 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd1832d-7300-4430-8a9b-87c02e0f0ae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:08.829834 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd1832d-7300-4430-8a9b-87c02e0f0ae9 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:08.850430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-7bd1832d-7300-4430-8a9b-87c02e0f0ae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:08.850430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:08.850430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:08.850430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:08.850430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:08.850430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:08.850430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:08.850430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:08.850430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:08.850430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:08.850430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:08.850430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:08.850430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:08.850430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:08.850430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:08.851650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:08.852970 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:08.852970 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:08.852970 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:08.852970 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:08.852970 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 412/823] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:08.857998 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1832d-7300-4430-8a9b-87c02e0f0ae9 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:08.951484 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:08.992740 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04bea94-2f77-4131-97ec-b0029c965a63 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:09.017952 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1832d-7300-4430-8a9b-87c02e0f0ae9 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:09.029623 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce999da-0c2b-4d97-83e3-26e71ee9a348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:09.030495 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce999da-0c2b-4d97-83e3-26e71ee9a348 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:09.030773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce999da-0c2b-4d97-83e3-26e71ee9a348 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:09.031046 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce999da-0c2b-4d97-83e3-26e71ee9a348 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:09.043479 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce999da-0c2b-4d97-83e3-26e71ee9a348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:09.045023 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce999da-0c2b-4d97-83e3-26e71ee9a348 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:09.055377 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67583976-16c3-43c5-9d36-e100297bb924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:09.056062 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67583976-16c3-43c5-9d36-e100297bb924 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:09.056411 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67583976-16c3-43c5-9d36-e100297bb924 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:09.056708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67583976-16c3-43c5-9d36-e100297bb924 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:09.057904 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67583976-16c3-43c5-9d36-e100297bb924 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:09.058221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67583976-16c3-43c5-9d36-e100297bb924 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:09.059658 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67583976-16c3-43c5-9d36-e100297bb924 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:09.065924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-7ce999da-0c2b-4d97-83e3-26e71ee9a348 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:09.065924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:09.065924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:09.065924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:09.065924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:09.065924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:09.065924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:09.065924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:09.065924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:09.065924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:09.065924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:09.065924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:09.065924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:09.065924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:09.065924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:09.067101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:09.068375 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:09.068375 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:09.068375 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:09.068375 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:09.068375 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 412/824] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:09.074793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce999da-0c2b-4d97-83e3-26e71ee9a348 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:09.077077 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 413/825] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:09.082958 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67583976-16c3-43c5-9d36-e100297bb924 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:09.151220 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:09.197696 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce999da-0c2b-4d97-83e3-26e71ee9a348 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:09.206082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67583976-16c3-43c5-9d36-e100297bb924 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:09.236362 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4701ce-39b2-4422-ad15-b26851906ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:09.237474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4701ce-39b2-4422-ad15-b26851906ffb None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:09.237983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4701ce-39b2-4422-ad15-b26851906ffb None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:09.238523 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4701ce-39b2-4422-ad15-b26851906ffb None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:09.239504 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4701ce-39b2-4422-ad15-b26851906ffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:09.240040 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4701ce-39b2-4422-ad15-b26851906ffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:09.241975 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4701ce-39b2-4422-ad15-b26851906ffb None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:09.245194 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb08c5-3543-4e43-a00d-0749677197b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:09.246129 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb08c5-3543-4e43-a00d-0749677197b1 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:09.246521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb08c5-3543-4e43-a00d-0749677197b1 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:09.246766 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:09.246889 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb08c5-3543-4e43-a00d-0749677197b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:09.256063 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb08c5-3543-4e43-a00d-0749677197b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:09.257740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb08c5-3543-4e43-a00d-0749677197b1 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:09.260424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 413/826] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:09.266582 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4701ce-39b2-4422-ad15-b26851906ffb None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:09.279259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-bdeb08c5-3543-4e43-a00d-0749677197b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:09.279259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:09.279259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:09.279259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:09.279259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:09.279259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:09.279259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:09.279259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:09.279259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:09.279259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:09.279259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:09.279259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:09.279259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:05:09.279259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:05:09.279259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:09.280364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:09.281401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:05:09.281401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:05:09.281401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:05:09.281401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:05:09.281401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:05:09.281401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:05:09.281401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:09.281401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:09.281401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 414/827] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:09.285398 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb08c5-3543-4e43-a00d-0749677197b1 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:09.382799 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4701ce-39b2-4422-ad15-b26851906ffb None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:09.419264 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe08ef-5552-4952-bcf1-d192363ded0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:09.420628 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe08ef-5552-4952-bcf1-d192363ded0a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:09.421124 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe08ef-5552-4952-bcf1-d192363ded0a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:09.421646 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe08ef-5552-4952-bcf1-d192363ded0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:09.427527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb08c5-3543-4e43-a00d-0749677197b1 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:09.429262 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe08ef-5552-4952-bcf1-d192363ded0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:09.430871 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe08ef-5552-4952-bcf1-d192363ded0a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:09.444370 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:09.445560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-7cfe08ef-5552-4952-bcf1-d192363ded0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:09.445560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:09.445560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:09.445560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:09.445560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:09.445560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:09.445560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:09.445560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:09.445560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:09.445560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:09.445560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:09.445560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:09.445560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:05:09.445560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:05:09.445560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:09.446753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:09.448480 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:05:09.448480 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:05:09.448480 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:05:09.448480 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:05:09.448480 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:05:09.448480 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:05:09.448480 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:09.448480 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:09.450029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 414/828] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:09.455594 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe08ef-5552-4952-bcf1-d192363ded0a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:09.482616 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd7a65-477c-4d16-a240-913ab0ddfab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:09.483685 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd7a65-477c-4d16-a240-913ab0ddfab4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:09.484115 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd7a65-477c-4d16-a240-913ab0ddfab4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:09.484582 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd7a65-477c-4d16-a240-913ab0ddfab4 None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:09.485623 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd7a65-477c-4d16-a240-913ab0ddfab4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:09.486162 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd7a65-477c-4d16-a240-913ab0ddfab4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:09.488240 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd7a65-477c-4d16-a240-913ab0ddfab4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:09.507266 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 415/829] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:05:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:09.507266 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d64ebe-ae06-49d7-aea2-543da5c74c9e None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:09.507266 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d64ebe-ae06-49d7-aea2-543da5c74c9e None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:09.507266 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d64ebe-ae06-49d7-aea2-543da5c74c9e None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:09.507266 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 416/830] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:09.512189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d64ebe-ae06-49d7-aea2-543da5c74c9e None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:09.522128 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:09.574525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe08ef-5552-4952-bcf1-d192363ded0a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:09.611137 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c366bc-9c21-4a91-b430-33fff658dcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:09.611813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c366bc-9c21-4a91-b430-33fff658dcf0 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:09.612089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c366bc-9c21-4a91-b430-33fff658dcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:09.612369 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c366bc-9c21-4a91-b430-33fff658dcf0 None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:09.613159 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c366bc-9c21-4a91-b430-33fff658dcf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:09.613472 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c366bc-9c21-4a91-b430-33fff658dcf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:09.614973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c366bc-9c21-4a91-b430-33fff658dcf0 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:09.627630 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 415/831] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:05:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:09.628938 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0546d3-76e1-4ef1-8eba-ebc281289994 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:09.629225 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0546d3-76e1-4ef1-8eba-ebc281289994 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:09.629496 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0546d3-76e1-4ef1-8eba-ebc281289994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:09.639059 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d64ebe-ae06-49d7-aea2-543da5c74c9e None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:09.648531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-2e0546d3-76e1-4ef1-8eba-ebc281289994 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:09.650244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-2e0546d3-76e1-4ef1-8eba-ebc281289994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:09.680097 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522ba39-d8f7-4931-b1a7-3bdad7e23c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:09.681170 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522ba39-d8f7-4931-b1a7-3bdad7e23c32 None admin] REQUEST_METHOD: `PUT` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:09.681777 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522ba39-d8f7-4931-b1a7-3bdad7e23c32 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:09.682707 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522ba39-d8f7-4931-b1a7-3bdad7e23c32 None admin] PATH_INFO: `/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/bdf915a5e0aa42a89e575ce1e8a65e2a/roles/bdaf8732b34c43a38465efde7e18a060` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:09.698644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522ba39-d8f7-4931-b1a7-3bdad7e23c32 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba22fdc274b74521847e5fceb2d307b1, user_id=bdf915a5e0aa42a89e575ce1e8a65e2a, role_id=bdaf8732b34c43a38465efde7e18a060)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:09.700102 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522ba39-d8f7-4931-b1a7-3bdad7e23c32 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:09.714695 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 417/832] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:05:09 2022] PUT /identity/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/bdf915a5e0aa42a89e575ce1e8a65e2a/roles/bdaf8732b34c43a38465efde7e18a060 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:05:09.715514 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522ba39-d8f7-4931-b1a7-3bdad7e23c32 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:09.737261 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 416/833] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:09.751060 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:09.764609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0546d3-76e1-4ef1-8eba-ebc281289994 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:09.824338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:09.856754 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522ba39-d8f7-4931-b1a7-3bdad7e23c32 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:09.895040 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f489eca-6943-4748-a68a-06bd7e92bdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:09.895338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0546d3-76e1-4ef1-8eba-ebc281289994 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:09.895729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f489eca-6943-4748-a68a-06bd7e92bdf4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:09.896016 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f489eca-6943-4748-a68a-06bd7e92bdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:09.896295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f489eca-6943-4748-a68a-06bd7e92bdf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:09.897162 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f489eca-6943-4748-a68a-06bd7e92bdf4 None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=378b046698c04b7b88194a9ffe5b045a) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:09.897483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f489eca-6943-4748-a68a-06bd7e92bdf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:09.899017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f489eca-6943-4748-a68a-06bd7e92bdf4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:09.908975 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 418/834] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:05:09 2022] GET /identity/v3/role_assignments?user.id=378b046698c04b7b88194a9ffe5b045a&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:09.908975 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c4178-b7ad-4ea1-9057-ee341189231e None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:09.908975 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c4178-b7ad-4ea1-9057-ee341189231e None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:09.908975 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c4178-b7ad-4ea1-9057-ee341189231e None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:09.909427 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 419/835] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:09.915916 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd9ab6-c7d3-4846-b095-4a3f3f69cd0f None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:09.916292 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd9ab6-c7d3-4846-b095-4a3f3f69cd0f None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:09.916647 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd9ab6-c7d3-4846-b095-4a3f3f69cd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:09.936627 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-7ecd9ab6-c7d3-4846-b095-4a3f3f69cd0f None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:09.942291 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-7ecd9ab6-c7d3-4846-b095-4a3f3f69cd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:09.956274 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f74791-1213-4a43-a67f-5ef17783eb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:09.957098 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f74791-1213-4a43-a67f-5ef17783eb45 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:09.957378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f74791-1213-4a43-a67f-5ef17783eb45 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:09.957649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f74791-1213-4a43-a67f-5ef17783eb45 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:09.960464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f74791-1213-4a43-a67f-5ef17783eb45 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:09.960464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f74791-1213-4a43-a67f-5ef17783eb45 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-d3f74791-1213-4a43-a67f-5ef17783eb45 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:09.976025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:09.977458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:09.978587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:09.978587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:09.978587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:09.978587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:09.978587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:09.978587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:09.978587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:09.978587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:09.978587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:09.978587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:09.978587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:09.978587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:09.978587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:05:09.978587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:09.980592 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 417/836] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:05:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:09.986879 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f74791-1213-4a43-a67f-5ef17783eb45 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:10.050350 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 420/837] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:10.073534 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd9ab6-c7d3-4846-b095-4a3f3f69cd0f None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:10.098745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:10.109463 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f74791-1213-4a43-a67f-5ef17783eb45 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:10.125893 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:10.151639 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a8cbe-c0a3-42de-b723-a1809e5acf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:10.152315 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a8cbe-c0a3-42de-b723-a1809e5acf37 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:10.152594 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a8cbe-c0a3-42de-b723-a1809e5acf37 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:10.152864 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a8cbe-c0a3-42de-b723-a1809e5acf37 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:10.153690 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a8cbe-c0a3-42de-b723-a1809e5acf37 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:10.154003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a8cbe-c0a3-42de-b723-a1809e5acf37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:10.155546 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a8cbe-c0a3-42de-b723-a1809e5acf37 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:10.161829 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 418/838] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:05:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:10.168783 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a8cbe-c0a3-42de-b723-a1809e5acf37 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:10.215729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd9ab6-c7d3-4846-b095-4a3f3f69cd0f None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:10.254145 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882f565-1b8c-441e-92f6-c47c5fb1874e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:10.254984 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882f565-1b8c-441e-92f6-c47c5fb1874e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:10.255265 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882f565-1b8c-441e-92f6-c47c5fb1874e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:10.255538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882f565-1b8c-441e-92f6-c47c5fb1874e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:10.266731 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882f565-1b8c-441e-92f6-c47c5fb1874e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-5882f565-1b8c-441e-92f6-c47c5fb1874e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:05:10.274680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:10.275798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:10.276716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:10.276716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:10.276716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:10.276716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:10.276716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:05:10.276716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:05:10.276716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:05:10.276716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:05:10.276716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:05:10.276716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:05:10.276716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:05:10.276716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:10.281163 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 421/839] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:05:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:10.282604 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882f565-1b8c-441e-92f6-c47c5fb1874e None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:10.304367 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a8cbe-c0a3-42de-b723-a1809e5acf37 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:10.352344 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdc7b3-c8e1-4d7a-9190-bf7dc63926a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:10.353121 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdc7b3-c8e1-4d7a-9190-bf7dc63926a9 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:10.353529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdc7b3-c8e1-4d7a-9190-bf7dc63926a9 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:10.353813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdc7b3-c8e1-4d7a-9190-bf7dc63926a9 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:10.354596 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdc7b3-c8e1-4d7a-9190-bf7dc63926a9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:10.354914 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdc7b3-c8e1-4d7a-9190-bf7dc63926a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:10.356594 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdc7b3-c8e1-4d7a-9190-bf7dc63926a9 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:10.362039 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 419/840] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:05:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:10.381882 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:10.414817 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882f565-1b8c-441e-92f6-c47c5fb1874e None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:10.452293 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f4d0e-5132-4991-a79c-483203948f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:10.453089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f4d0e-5132-4991-a79c-483203948f49 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:10.453319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f4d0e-5132-4991-a79c-483203948f49 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:10.453603 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f4d0e-5132-4991-a79c-483203948f49 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:10.454407 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f4d0e-5132-4991-a79c-483203948f49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:10.454724 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f4d0e-5132-4991-a79c-483203948f49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:10.456200 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f4d0e-5132-4991-a79c-483203948f49 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:10.466510 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 422/841] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:05:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:10.475506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdc7b3-c8e1-4d7a-9190-bf7dc63926a9 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:10.521123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:10.604972 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdc7b3-c8e1-4d7a-9190-bf7dc63926a9 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:10.639752 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68031b-67f2-4669-a95d-6dd77c1d0cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:10.640602 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68031b-67f2-4669-a95d-6dd77c1d0cd6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:10.640882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68031b-67f2-4669-a95d-6dd77c1d0cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:10.641152 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68031b-67f2-4669-a95d-6dd77c1d0cd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:10.652223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68031b-67f2-4669-a95d-6dd77c1d0cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:10.653685 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68031b-67f2-4669-a95d-6dd77c1d0cd6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:10.671903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-3b68031b-67f2-4669-a95d-6dd77c1d0cd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:10.671903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:10.671903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:10.671903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:10.671903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:10.671903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:10.671903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:10.671903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:10.671903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:10.671903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:10.671903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:10.671903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:10.671903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:10.671903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:10.671903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:10.674112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:10.676368 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:10.676368 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:10.676368 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:10.676368 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:10.676938 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 420/842] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:10.681944 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f4d0e-5132-4991-a79c-483203948f49 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:10.796927 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f4d0e-5132-4991-a79c-483203948f49 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:10.833972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6335369-d7a2-49bd-b3f4-a6ef173378fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:10.835030 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6335369-d7a2-49bd-b3f4-a6ef173378fa None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:10.835519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6335369-d7a2-49bd-b3f4-a6ef173378fa None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:10.835994 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6335369-d7a2-49bd-b3f4-a6ef173378fa None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:10.836921 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6335369-d7a2-49bd-b3f4-a6ef173378fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:10.837426 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6335369-d7a2-49bd-b3f4-a6ef173378fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:10.839771 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6335369-d7a2-49bd-b3f4-a6ef173378fa None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:10.852991 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:10.858014 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 423/843] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:10.859530 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68031b-67f2-4669-a95d-6dd77c1d0cd6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:10.959176 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:10.969821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68031b-67f2-4669-a95d-6dd77c1d0cd6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:11.007931 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa99d05-b1b5-478e-8829-13597e067ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:11.008802 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa99d05-b1b5-478e-8829-13597e067ea3 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:11.009117 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa99d05-b1b5-478e-8829-13597e067ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:11.009344 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa99d05-b1b5-478e-8829-13597e067ea3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:11.019324 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa99d05-b1b5-478e-8829-13597e067ea3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:11.020804 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa99d05-b1b5-478e-8829-13597e067ea3 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-4fa99d05-b1b5-478e-8829-13597e067ea3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:05:11.036047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:11.037258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:11.038337 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:11.038337 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:11.038337 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:05:11.038337 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:05:11.038337 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:05:11.038337 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:05:11.038337 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:05:11.038337 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:05:11.038337 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:05:11.038337 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:11.042499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 421/844] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:05:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:11.043884 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6335369-d7a2-49bd-b3f4-a6ef173378fa None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:11.153605 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6335369-d7a2-49bd-b3f4-a6ef173378fa None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:11.191365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecffe0-a56a-43f7-ab7b-da59fa257a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:11.192130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecffe0-a56a-43f7-ab7b-da59fa257a9d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:11.192415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecffe0-a56a-43f7-ab7b-da59fa257a9d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:11.192741 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecffe0-a56a-43f7-ab7b-da59fa257a9d None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:11.193612 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcecffe0-a56a-43f7-ab7b-da59fa257a9d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:11.193923 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcecffe0-a56a-43f7-ab7b-da59fa257a9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:11.195405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcecffe0-a56a-43f7-ab7b-da59fa257a9d None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:11.220092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 424/845] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 20:05:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:11.221490 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa99d05-b1b5-478e-8829-13597e067ea3 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:11.328814 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa99d05-b1b5-478e-8829-13597e067ea3 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:11.364449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b510499-3c9e-4415-a01b-a3f05e89e492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:11.365113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b510499-3c9e-4415-a01b-a3f05e89e492 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:11.365397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b510499-3c9e-4415-a01b-a3f05e89e492 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:11.365666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b510499-3c9e-4415-a01b-a3f05e89e492 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:11.398655 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:11.439110 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b510499-3c9e-4415-a01b-a3f05e89e492 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:11.440511 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b510499-3c9e-4415-a01b-a3f05e89e492 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:11.555856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 422/846] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:05:11 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:05:11.558458 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecffe0-a56a-43f7-ab7b-da59fa257a9d None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:11.630870 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:11.674727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecffe0-a56a-43f7-ab7b-da59fa257a9d None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:11.712235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbf7d0-3bad-4859-bbcb-4c14bfa2670d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:11.713722 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbf7d0-3bad-4859-bbcb-4c14bfa2670d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:11.714012 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbf7d0-3bad-4859-bbcb-4c14bfa2670d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:11.714316 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbf7d0-3bad-4859-bbcb-4c14bfa2670d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:11.725862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbf7d0-3bad-4859-bbcb-4c14bfa2670d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:11.727362 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbf7d0-3bad-4859-bbcb-4c14bfa2670d None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:11.747587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-32bbf7d0-3bad-4859-bbcb-4c14bfa2670d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:11.747587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:11.747587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:11.747587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:11.747587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:11.747587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:11.747587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:11.747587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:11.747587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:11.747587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:11.747587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:11.747587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:11.747587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:11.747587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:11.747587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:11.748583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:11.749542 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:11.749542 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:11.749542 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:11.749542 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:11.753499 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 425/847] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:11.754941 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b510499-3c9e-4415-a01b-a3f05e89e492 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:11.820877 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:11.862528 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b510499-3c9e-4415-a01b-a3f05e89e492 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:11.901947 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9fa209-ae58-4040-a13e-0392cdf29754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:11.902678 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fa209-ae58-4040-a13e-0392cdf29754 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:11.902959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fa209-ae58-4040-a13e-0392cdf29754 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:11.903235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fa209-ae58-4040-a13e-0392cdf29754 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:11.903979 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9fa209-ae58-4040-a13e-0392cdf29754 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:11.904291 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9fa209-ae58-4040-a13e-0392cdf29754 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:11.905827 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9fa209-ae58-4040-a13e-0392cdf29754 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:11.924269 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 423/848] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:11.925650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbf7d0-3bad-4859-bbcb-4c14bfa2670d None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:12.036360 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbf7d0-3bad-4859-bbcb-4c14bfa2670d None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:12.047308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:12.081190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef35bc-6d15-428f-b7fe-9c4107f9fd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:12.082046 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef35bc-6d15-428f-b7fe-9c4107f9fd4b None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:12.082359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef35bc-6d15-428f-b7fe-9c4107f9fd4b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:12.082640 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef35bc-6d15-428f-b7fe-9c4107f9fd4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:12.091202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef35bc-6d15-428f-b7fe-9c4107f9fd4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:12.092694 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef35bc-6d15-428f-b7fe-9c4107f9fd4b None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:12.095181 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acf43f-dda0-4aaf-a811-08ef569f3e29 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:12.095473 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acf43f-dda0-4aaf-a811-08ef569f3e29 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:12.095753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acf43f-dda0-4aaf-a811-08ef569f3e29 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:12.101636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 424/849] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:12.104538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d2724-c2a0-41a9-9208-119313727292 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:12.104837 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d2724-c2a0-41a9-9208-119313727292 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:12.105120 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d2724-c2a0-41a9-9208-119313727292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:12.109415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-13ef35bc-6d15-428f-b7fe-9c4107f9fd4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:12.109415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:12.109415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:12.109415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:12.109415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:12.109415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:12.109415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:12.109415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:12.109415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:12.109415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:12.109415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:12.109415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:12.109415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:05:12.109415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:05:12.109415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:12.110391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:12.111332 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:05:12.111332 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:05:12.111332 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:05:12.111332 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:05:12.111332 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:05:12.111332 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:05:12.111332 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:12.111332 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:12.111332 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 426/850] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:12.117434 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef35bc-6d15-428f-b7fe-9c4107f9fd4b None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:12.126965 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-b52d2724-c2a0-41a9-9208-119313727292 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:12.128184 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-b52d2724-c2a0-41a9-9208-119313727292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:12.230287 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 425/851] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:12.240943 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef35bc-6d15-428f-b7fe-9c4107f9fd4b None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:12.248676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d2724-c2a0-41a9-9208-119313727292 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:12.281949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a1333-9279-42f6-b3c4-a9848e250d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:12.282751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a1333-9279-42f6-b3c4-a9848e250d96 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:12.283038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a1333-9279-42f6-b3c4-a9848e250d96 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:12.283323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a1333-9279-42f6-b3c4-a9848e250d96 None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:12.284111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a1333-9279-42f6-b3c4-a9848e250d96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:12.284423 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a1333-9279-42f6-b3c4-a9848e250d96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:12.286088 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a1333-9279-42f6-b3c4-a9848e250d96 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:12.296899 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:12.300817 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 427/852] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:05:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:12.302242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e786999-ed93-436f-af78-1531912a0509 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:12.302558 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e786999-ed93-436f-af78-1531912a0509 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:12.302835 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e786999-ed93-436f-af78-1531912a0509 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:12.308308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 428/853] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:12.308469 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e786999-ed93-436f-af78-1531912a0509 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:12.339840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:12.374552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d2724-c2a0-41a9-9208-119313727292 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:12.409931 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b89335-86d3-4fb8-b627-5decb0bbc2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:12.410773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b89335-86d3-4fb8-b627-5decb0bbc2ce None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:12.411048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b89335-86d3-4fb8-b627-5decb0bbc2ce None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:12.411332 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b89335-86d3-4fb8-b627-5decb0bbc2ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:12.422240 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b89335-86d3-4fb8-b627-5decb0bbc2ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:05:12.425236 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e786999-ed93-436f-af78-1531912a0509 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-41b89335-86d3-4fb8-b627-5decb0bbc2ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:05:12.430049 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:12.431107 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:12.432277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:12.432277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:12.432277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:12.432277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:12.432277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:05:12.432277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:05:12.432277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:05:12.432277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:05:12.432277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:05:12.432277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:05:12.432277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:05:12.432277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:12.432277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 426/854] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:05:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:12.433380 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b99c0-df82-40ef-9b1b-3c9dc27136a8 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:12.433604 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b99c0-df82-40ef-9b1b-3c9dc27136a8 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:12.433888 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b99c0-df82-40ef-9b1b-3c9dc27136a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:12.453133 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-3b5b99c0-df82-40ef-9b1b-3c9dc27136a8 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:12.454361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-3b5b99c0-df82-40ef-9b1b-3c9dc27136a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:12.468267 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52783cc-fc56-437a-8cb9-49acd4083def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:12.469678 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52783cc-fc56-437a-8cb9-49acd4083def None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:12.470249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52783cc-fc56-437a-8cb9-49acd4083def None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:12.470731 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52783cc-fc56-437a-8cb9-49acd4083def None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:12.471803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52783cc-fc56-437a-8cb9-49acd4083def None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=87b30300be1a41128ff88e12e260f63b) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:12.472425 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52783cc-fc56-437a-8cb9-49acd4083def None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:12.474280 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52783cc-fc56-437a-8cb9-49acd4083def None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:12.481024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 429/855] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:05:12 2022] GET /identity/v3/role_assignments?user.id=87b30300be1a41128ff88e12e260f63b&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:12.481806 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52783cc-fc56-437a-8cb9-49acd4083def None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:12.571067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 427/856] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:12.589659 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b99c0-df82-40ef-9b1b-3c9dc27136a8 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:12.604521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:12.615317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52783cc-fc56-437a-8cb9-49acd4083def None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:12.623616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:12.657868 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04da41c-ec01-4908-be91-616a2fa32112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:12.659086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04da41c-ec01-4908-be91-616a2fa32112 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:12.659639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04da41c-ec01-4908-be91-616a2fa32112 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:12.660170 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04da41c-ec01-4908-be91-616a2fa32112 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:12.661149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04da41c-ec01-4908-be91-616a2fa32112 None admin] RBAC: Adding query filter params (name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:12.661678 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04da41c-ec01-4908-be91-616a2fa32112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:12.663434 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04da41c-ec01-4908-be91-616a2fa32112 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:12.673544 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 430/857] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:05:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:12.678887 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04da41c-ec01-4908-be91-616a2fa32112 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:12.703621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b99c0-df82-40ef-9b1b-3c9dc27136a8 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:12.746458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098b63f-c796-4430-b130-68826e08c245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:12.747298 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098b63f-c796-4430-b130-68826e08c245 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:12.747575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098b63f-c796-4430-b130-68826e08c245 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:12.747846 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098b63f-c796-4430-b130-68826e08c245 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:12.748512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098b63f-c796-4430-b130-68826e08c245 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:12.750003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098b63f-c796-4430-b130-68826e08c245 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-4098b63f-c796-4430-b130-68826e08c245 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:12.763359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:12.764432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:12.765353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 428/858] 10.222.0.84 () {56 vars in 1132 bytes} [Thu May 19 20:05:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:12.766253 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29c358-21de-4d16-a3e6-667d712ee2da None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:12.770520 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29c358-21de-4d16-a3e6-667d712ee2da None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:12.771011 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29c358-21de-4d16-a3e6-667d712ee2da None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:12.772647 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 429/859] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:12.773525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29c358-21de-4d16-a3e6-667d712ee2da None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:12.798856 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04da41c-ec01-4908-be91-616a2fa32112 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:12.838976 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed6d95-786d-4167-83f2-afc8c4f1ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:12.839868 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed6d95-786d-4167-83f2-afc8c4f1ba19 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:12.840254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed6d95-786d-4167-83f2-afc8c4f1ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:12.840536 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed6d95-786d-4167-83f2-afc8c4f1ba19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:12.853493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed6d95-786d-4167-83f2-afc8c4f1ba19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:12.855011 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed6d95-786d-4167-83f2-afc8c4f1ba19 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:12.873136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-68ed6d95-786d-4167-83f2-afc8c4f1ba19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:12.873136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:12.873136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:12.873136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:12.873136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:12.873136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:12.873136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:12.873136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:12.873136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:12.873136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:12.873136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:12.873136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:12.873136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:12.873136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:12.873136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:12.874314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:12.875484 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:12.875484 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:12.875484 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:12.875484 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:12.875484 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 431/860] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:12.875911 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168d51e-a277-47de-bc06-ea33c4a97ef6 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:12.876188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168d51e-a277-47de-bc06-ea33c4a97ef6 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:12.876463 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168d51e-a277-47de-bc06-ea33c4a97ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:12.885583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:12.889222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:12.894218 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29c358-21de-4d16-a3e6-667d712ee2da None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:12.895472 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-1168d51e-a277-47de-bc06-ea33c4a97ef6 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:12.897402 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-1168d51e-a277-47de-bc06-ea33c4a97ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:12.930949 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93309551-8464-4bef-bd21-1d5bd461cbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:12.931709 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93309551-8464-4bef-bd21-1d5bd461cbe7 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:12.932019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93309551-8464-4bef-bd21-1d5bd461cbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:12.932421 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93309551-8464-4bef-bd21-1d5bd461cbe7 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:12.933062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93309551-8464-4bef-bd21-1d5bd461cbe7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:12.933353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93309551-8464-4bef-bd21-1d5bd461cbe7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:12.934795 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93309551-8464-4bef-bd21-1d5bd461cbe7 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:12.944609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 430/861] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:05:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:12.944609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93309551-8464-4bef-bd21-1d5bd461cbe7 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:12.974310 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 432/862] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:12.974310 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168d51e-a277-47de-bc06-ea33c4a97ef6 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:13.060476 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93309551-8464-4bef-bd21-1d5bd461cbe7 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:13.082110 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168d51e-a277-47de-bc06-ea33c4a97ef6 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:13.096396 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f7d1a-be80-44a4-9cae-291e73368aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:13.097001 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f7d1a-be80-44a4-9cae-291e73368aa3 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:13.097278 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f7d1a-be80-44a4-9cae-291e73368aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:13.097561 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f7d1a-be80-44a4-9cae-291e73368aa3 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:13.098288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f7d1a-be80-44a4-9cae-291e73368aa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:13.098605 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f7d1a-be80-44a4-9cae-291e73368aa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:13.099993 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f7d1a-be80-44a4-9cae-291e73368aa3 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:13.116798 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 431/863] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:13.116798 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f7d1a-be80-44a4-9cae-291e73368aa3 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:13.124595 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b7691-ac17-4259-b7e2-d3e9f908c0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:13.125260 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b7691-ac17-4259-b7e2-d3e9f908c0b5 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:13.125541 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b7691-ac17-4259-b7e2-d3e9f908c0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:13.125817 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b7691-ac17-4259-b7e2-d3e9f908c0b5 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:13.126590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b7691-ac17-4259-b7e2-d3e9f908c0b5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:13.126900 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b7691-ac17-4259-b7e2-d3e9f908c0b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:13.128284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b7691-ac17-4259-b7e2-d3e9f908c0b5 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:13.131440 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:13.135363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:13.135513 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 433/864] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:05:12 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:13.136352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b7691-ac17-4259-b7e2-d3e9f908c0b5 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:13.244533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f7d1a-be80-44a4-9cae-291e73368aa3 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:13.259830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b7691-ac17-4259-b7e2-d3e9f908c0b5 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:13.285588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc0bc4-e50e-4b73-97de-2d41859173ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:13.286757 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc0bc4-e50e-4b73-97de-2d41859173ff None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:13.287322 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc0bc4-e50e-4b73-97de-2d41859173ff None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:13.287757 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc0bc4-e50e-4b73-97de-2d41859173ff None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:13.288674 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc0bc4-e50e-4b73-97de-2d41859173ff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:13.290625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc0bc4-e50e-4b73-97de-2d41859173ff None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:13.300018 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b5ce0-dcf0-4217-9699-07bf83cbbb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:13.300867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b5ce0-dcf0-4217-9699-07bf83cbbb22 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:13.301170 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b5ce0-dcf0-4217-9699-07bf83cbbb22 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:13.301423 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b5ce0-dcf0-4217-9699-07bf83cbbb22 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:13.318356 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b5ce0-dcf0-4217-9699-07bf83cbbb22 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:13.318356 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b5ce0-dcf0-4217-9699-07bf83cbbb22 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:13.318654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 432/865] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:05:13 2022] POST /identity/v3/services => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-151b5ce0-dcf0-4217-9699-07bf83cbbb22 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:05:13.335464 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:13.336602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:13.338103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:13.338103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:13.338103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:05:13.338103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:05:13.338103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:05:13.338103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:05:13.338103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:05:13.338103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:05:13.338103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:05:13.338103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:13.338103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 434/866] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:05:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:13.346319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc0bc4-e50e-4b73-97de-2d41859173ff None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:13.448990 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:13.482530 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc0bc4-e50e-4b73-97de-2d41859173ff None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:13.518776 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf398af-e8ee-45ea-a7be-9338b8bf2819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:13.519458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf398af-e8ee-45ea-a7be-9338b8bf2819 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:13.519740 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf398af-e8ee-45ea-a7be-9338b8bf2819 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:13.520009 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf398af-e8ee-45ea-a7be-9338b8bf2819 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:13.520851 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf398af-e8ee-45ea-a7be-9338b8bf2819 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:13.521156 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf398af-e8ee-45ea-a7be-9338b8bf2819 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:13.522701 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf398af-e8ee-45ea-a7be-9338b8bf2819 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:13.540425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 433/867] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 20:05:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:13.547747 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b5ce0-dcf0-4217-9699-07bf83cbbb22 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:13.617033 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:13.658729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b5ce0-dcf0-4217-9699-07bf83cbbb22 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:13.694850 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff408e-2c94-4da9-98ff-169557fa2738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:13.695552 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff408e-2c94-4da9-98ff-169557fa2738 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:13.695820 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff408e-2c94-4da9-98ff-169557fa2738 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:13.696096 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff408e-2c94-4da9-98ff-169557fa2738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:13.768482 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff408e-2c94-4da9-98ff-169557fa2738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:13.769852 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff408e-2c94-4da9-98ff-169557fa2738 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:13.828892 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:13.876860 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 435/868] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:05:13 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:05:13.878324 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf398af-e8ee-45ea-a7be-9338b8bf2819 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:13.990226 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf398af-e8ee-45ea-a7be-9338b8bf2819 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:14.024881 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:14.032350 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc6a42-5914-4ed0-a1f5-a41f1fb098ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:14.033451 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc6a42-5914-4ed0-a1f5-a41f1fb098ee None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:14.033944 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc6a42-5914-4ed0-a1f5-a41f1fb098ee None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:14.034470 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc6a42-5914-4ed0-a1f5-a41f1fb098ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:14.046315 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc6a42-5914-4ed0-a1f5-a41f1fb098ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:14.048069 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc6a42-5914-4ed0-a1f5-a41f1fb098ee None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:14.067153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-79bc6a42-5914-4ed0-a1f5-a41f1fb098ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:14.067153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:14.067153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:14.067153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:14.067153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:14.067153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:14.067153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:14.067153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:14.067153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:14.067153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:14.067153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:14.067153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:14.067153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:14.067153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:14.067153 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:14.068158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:14.069133 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:14.069133 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:14.069133 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:14.069133 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:14.071413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 434/869] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:14.078797 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff408e-2c94-4da9-98ff-169557fa2738 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:14.179487 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff408e-2c94-4da9-98ff-169557fa2738 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:14.215110 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6ad41-b27b-4264-90b4-9df167a86548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:14.215786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6ad41-b27b-4264-90b4-9df167a86548 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:14.216064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6ad41-b27b-4264-90b4-9df167a86548 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:14.216334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6ad41-b27b-4264-90b4-9df167a86548 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:14.217058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6ad41-b27b-4264-90b4-9df167a86548 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:14.217358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6ad41-b27b-4264-90b4-9df167a86548 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:14.218749 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6ad41-b27b-4264-90b4-9df167a86548 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:14.235628 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 436/870] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:14.236991 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc6a42-5914-4ed0-a1f5-a41f1fb098ee None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:14.342322 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc6a42-5914-4ed0-a1f5-a41f1fb098ee None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:14.380261 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a3ec9-8625-4e3b-86ab-84333435335e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:14.381518 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a3ec9-8625-4e3b-86ab-84333435335e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:14.382105 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a3ec9-8625-4e3b-86ab-84333435335e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:14.382629 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a3ec9-8625-4e3b-86ab-84333435335e None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:14.390797 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a3ec9-8625-4e3b-86ab-84333435335e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:14.392528 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a3ec9-8625-4e3b-86ab-84333435335e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:14.407406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-3f5a3ec9-8625-4e3b-86ab-84333435335e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:14.407406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:14.407406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:14.407406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:14.407406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:14.407406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:14.407406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:14.407406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:14.407406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:14.407406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:14.407406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:14.407406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:14.407406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:05:14.407406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:05:14.407406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:14.408497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:14.409474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:05:14.409474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:05:14.409474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:05:14.409474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:05:14.409474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:05:14.409474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:05:14.409474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:14.409474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:14.412250 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 435/871] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:14.420892 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6ad41-b27b-4264-90b4-9df167a86548 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:14.441803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:14.522724 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6ad41-b27b-4264-90b4-9df167a86548 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:14.560781 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e6c61-36fb-4d7c-9d83-2756d0a9f3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:14.561460 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6c61-36fb-4d7c-9d83-2756d0a9f3b4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:14.561738 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6c61-36fb-4d7c-9d83-2756d0a9f3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:14.562007 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6c61-36fb-4d7c-9d83-2756d0a9f3b4 None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:14.562845 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6c61-36fb-4d7c-9d83-2756d0a9f3b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:14.563151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6c61-36fb-4d7c-9d83-2756d0a9f3b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:14.564623 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6c61-36fb-4d7c-9d83-2756d0a9f3b4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:14.582474 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 437/872] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:05:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:14.583822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a3ec9-8625-4e3b-86ab-84333435335e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:14.636265 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:14.699583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a3ec9-8625-4e3b-86ab-84333435335e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:14.737509 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3295982-697f-445e-a016-ea463b4605f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:14.738612 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3295982-697f-445e-a016-ea463b4605f0 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:14.739129 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3295982-697f-445e-a016-ea463b4605f0 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:14.739625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3295982-697f-445e-a016-ea463b4605f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:14.740730 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3295982-697f-445e-a016-ea463b4605f0 None admin] RBAC: Adding query filter params (role.id=bdaf8732b34c43a38465efde7e18a060, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=bdf915a5e0aa42a89e575ce1e8a65e2a) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:14.741258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3295982-697f-445e-a016-ea463b4605f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:14.742996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3295982-697f-445e-a016-ea463b4605f0 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:14.750310 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 436/873] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:05:14 2022] GET /identity/v3/role_assignments?user.id=bdf915a5e0aa42a89e575ce1e8a65e2a&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=bdaf8732b34c43a38465efde7e18a060 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:14.816283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcac0d-0ae9-46d1-812a-1ed41b8112f0 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:14.816845 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcac0d-0ae9-46d1-812a-1ed41b8112f0 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:14.816845 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcac0d-0ae9-46d1-812a-1ed41b8112f0 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:14.818331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 438/874] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:14.824407 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2459612-5435-4888-bf57-0191abb97ef3 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:14.825035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2459612-5435-4888-bf57-0191abb97ef3 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:14.825538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2459612-5435-4888-bf57-0191abb97ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:14.845234 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-c2459612-5435-4888-bf57-0191abb97ef3 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:14.846938 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-c2459612-5435-4888-bf57-0191abb97ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:14.925812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 437/875] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:14 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:14.949702 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcac0d-0ae9-46d1-812a-1ed41b8112f0 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:15.041009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:15.063813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcac0d-0ae9-46d1-812a-1ed41b8112f0 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:15.102834 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c17afc-a222-4ff7-8ba4-00ed3b01378c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:15.103775 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c17afc-a222-4ff7-8ba4-00ed3b01378c None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:15.104097 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c17afc-a222-4ff7-8ba4-00ed3b01378c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:15.104467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c17afc-a222-4ff7-8ba4-00ed3b01378c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:15.105309 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c17afc-a222-4ff7-8ba4-00ed3b01378c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:15.106829 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c17afc-a222-4ff7-8ba4-00ed3b01378c None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-76c17afc-a222-4ff7-8ba4-00ed3b01378c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:15.121242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:15.122698 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:15.123894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:15.123894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:15.123894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:15.123894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:15.123894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:15.123894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:15.123894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:15.123894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:15.123894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:15.123894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:15.123894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:15.123894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:15.123894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:05:15.123894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:15.123894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 439/876] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:05:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:15.128306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2459612-5435-4888-bf57-0191abb97ef3 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:15.178738 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:15.250808 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2459612-5435-4888-bf57-0191abb97ef3 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:15.310084 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d31cd-ca12-4e5f-8902-e39a197cf1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:15.310822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d31cd-ca12-4e5f-8902-e39a197cf1f9 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:15.313744 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d31cd-ca12-4e5f-8902-e39a197cf1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:15.314024 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d31cd-ca12-4e5f-8902-e39a197cf1f9 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:15.314846 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d31cd-ca12-4e5f-8902-e39a197cf1f9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:15.315163 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d31cd-ca12-4e5f-8902-e39a197cf1f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:15.316594 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d31cd-ca12-4e5f-8902-e39a197cf1f9 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:15.324976 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 438/877] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:05:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:15.331547 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c17afc-a222-4ff7-8ba4-00ed3b01378c None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:15.450310 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c17afc-a222-4ff7-8ba4-00ed3b01378c None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:15.492356 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590954fa-e59c-4427-b608-70bc4025b001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:15.493500 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590954fa-e59c-4427-b608-70bc4025b001 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:15.494097 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590954fa-e59c-4427-b608-70bc4025b001 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:15.494639 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590954fa-e59c-4427-b608-70bc4025b001 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:15.495736 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590954fa-e59c-4427-b608-70bc4025b001 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:15.496266 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590954fa-e59c-4427-b608-70bc4025b001 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:15.497913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590954fa-e59c-4427-b608-70bc4025b001 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:15.504658 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 440/878] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:05:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:15.741322 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c0a2d-ec4a-40b1-a5e4-deea56503378 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:15.741525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c0a2d-ec4a-40b1-a5e4-deea56503378 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:15.741805 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c0a2d-ec4a-40b1-a5e4-deea56503378 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:15.743511 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 439/879] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:15.750450 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dd2e3-91dc-44a4-8bb5-43794f2e79ad None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:15.750952 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dd2e3-91dc-44a4-8bb5-43794f2e79ad None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:15.751468 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dd2e3-91dc-44a4-8bb5-43794f2e79ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:15.770478 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-5a0dd2e3-91dc-44a4-8bb5-43794f2e79ad None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:15.771699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-5a0dd2e3-91dc-44a4-8bb5-43794f2e79ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:15.774082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:15.805565 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc74ef-781f-48b8-874b-99de16db1539 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:15.805902 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc74ef-781f-48b8-874b-99de16db1539 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:15.806143 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc74ef-781f-48b8-874b-99de16db1539 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:15.808312 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 440/880] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:05:15.814329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a02ce-d0dc-4cb2-97d8-ca5c3ee507e6 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:15.814643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a02ce-d0dc-4cb2-97d8-ca5c3ee507e6 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:15.814891 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a02ce-d0dc-4cb2-97d8-ca5c3ee507e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:15.835715 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-e65a02ce-d0dc-4cb2-97d8-ca5c3ee507e6 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:15.836927 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-e65a02ce-d0dc-4cb2-97d8-ca5c3ee507e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:15.861899 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 441/881] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:15 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:15.889942 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0dd2e3-91dc-44a4-8bb5-43794f2e79ad None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:15.923400 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 441/882] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:15 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:05:15.950708 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a02ce-d0dc-4cb2-97d8-ca5c3ee507e6 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:15.989886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:16.000219 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0dd2e3-91dc-44a4-8bb5-43794f2e79ad None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:16.049993 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec8377-4615-4cd5-84fb-c8bf518ae950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:16.050730 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec8377-4615-4cd5-84fb-c8bf518ae950 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:16.051011 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec8377-4615-4cd5-84fb-c8bf518ae950 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:16.051285 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec8377-4615-4cd5-84fb-c8bf518ae950 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:16.052369 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec8377-4615-4cd5-84fb-c8bf518ae950 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:16.054293 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec8377-4615-4cd5-84fb-c8bf518ae950 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:16.068719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:16.076590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 442/883] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:05:15 2022] POST /identity/v3/services => generated 269 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:05:16.090589 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a02ce-d0dc-4cb2-97d8-ca5c3ee507e6 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:16.142489 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51414c3e-de36-4b4f-9d6f-4639910d157a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:16.143322 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51414c3e-de36-4b4f-9d6f-4639910d157a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:16.143605 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51414c3e-de36-4b4f-9d6f-4639910d157a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:16.143881 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51414c3e-de36-4b4f-9d6f-4639910d157a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:16.144572 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51414c3e-de36-4b4f-9d6f-4639910d157a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:16.146024 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51414c3e-de36-4b4f-9d6f-4639910d157a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-51414c3e-de36-4b4f-9d6f-4639910d157a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:16.160219 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:16.161378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:16.162538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:16.162538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:16.162538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:16.162538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:16.162538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:16.162538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:16.162538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:16.162538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:16.162538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:16.162538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:16.162538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:16.162538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:16.162538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:05:16.162538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:16.162538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 442/884] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:05:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:16.170615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec8377-4615-4cd5-84fb-c8bf518ae950 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:16.286294 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec8377-4615-4cd5-84fb-c8bf518ae950 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:16.321996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723f4db-92dd-43f3-b836-8882ac4673a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:16.322707 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723f4db-92dd-43f3-b836-8882ac4673a9 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:16.322991 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723f4db-92dd-43f3-b836-8882ac4673a9 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:16.323272 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723f4db-92dd-43f3-b836-8882ac4673a9 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:16.323995 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723f4db-92dd-43f3-b836-8882ac4673a9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:16.324336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723f4db-92dd-43f3-b836-8882ac4673a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:16.325727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723f4db-92dd-43f3-b836-8882ac4673a9 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:16.331362 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 443/885] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:05:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:16.336995 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51414c3e-de36-4b4f-9d6f-4639910d157a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:16.453214 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:16.463140 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51414c3e-de36-4b4f-9d6f-4639910d157a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:16.508932 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4858e-85ef-4f2c-b078-f50653cadda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:16.510286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4858e-85ef-4f2c-b078-f50653cadda0 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:16.510736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4858e-85ef-4f2c-b078-f50653cadda0 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:16.511020 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4858e-85ef-4f2c-b078-f50653cadda0 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:16.511753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4858e-85ef-4f2c-b078-f50653cadda0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:16.512067 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4858e-85ef-4f2c-b078-f50653cadda0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:16.513467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4858e-85ef-4f2c-b078-f50653cadda0 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:16.519100 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 443/886] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:05:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:16.524653 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723f4db-92dd-43f3-b836-8882ac4673a9 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:16.597219 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:16.637448 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723f4db-92dd-43f3-b836-8882ac4673a9 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:16.672355 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606e542-4940-4735-b1d6-2af7212cc18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:16.673032 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606e542-4940-4735-b1d6-2af7212cc18d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:16.673309 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606e542-4940-4735-b1d6-2af7212cc18d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:16.673583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606e542-4940-4735-b1d6-2af7212cc18d None admin] PATH_INFO: `/v3/endpoints` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:16.674409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606e542-4940-4735-b1d6-2af7212cc18d None admin] RBAC: Adding query filter params (interface=public, service_id=131338d362334538bf055bc7974cd7cc, region_id=RegionOne) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:16.674722 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606e542-4940-4735-b1d6-2af7212cc18d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:16.676094 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606e542-4940-4735-b1d6-2af7212cc18d None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:16.681710 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 444/887] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:05:16 2022] GET /identity/v3/endpoints?service_id=131338d362334538bf055bc7974cd7cc&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:16.687365 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4858e-85ef-4f2c-b078-f50653cadda0 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:16.791576 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4858e-85ef-4f2c-b078-f50653cadda0 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:16.831608 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a77ab2-89bc-42e4-b6e2-da9ce98c84eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:16.832202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a77ab2-89bc-42e4-b6e2-da9ce98c84eb None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:16.832488 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a77ab2-89bc-42e4-b6e2-da9ce98c84eb None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:16.832766 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a77ab2-89bc-42e4-b6e2-da9ce98c84eb None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:16.833455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a77ab2-89bc-42e4-b6e2-da9ce98c84eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:16.834922 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a77ab2-89bc-42e4-b6e2-da9ce98c84eb None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:16.842138 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 444/888] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:05:16 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:05:17.471399 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcce61c-ac2e-4034-b711-c778775e6d44 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:17.471942 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcce61c-ac2e-4034-b711-c778775e6d44 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:17.471942 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcce61c-ac2e-4034-b711-c778775e6d44 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:17.473558 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 445/889] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:17.479519 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd80c01-e960-49e4-912c-871b52138fcf None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:17.479941 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd80c01-e960-49e4-912c-871b52138fcf None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:17.480117 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd80c01-e960-49e4-912c-871b52138fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:17.497469 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-dbd80c01-e960-49e4-912c-871b52138fcf None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:17.498710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-dbd80c01-e960-49e4-912c-871b52138fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:17.514112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:17.571070 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 445/890] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:17.596333 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcce61c-ac2e-4034-b711-c778775e6d44 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:17.707995 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcce61c-ac2e-4034-b711-c778775e6d44 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:17.739906 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:17.743666 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a095a85-792a-4c7d-87cf-b12b6ae33c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:17.744480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a095a85-792a-4c7d-87cf-b12b6ae33c7e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:17.744772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a095a85-792a-4c7d-87cf-b12b6ae33c7e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:17.745045 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a095a85-792a-4c7d-87cf-b12b6ae33c7e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:17.756256 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a095a85-792a-4c7d-87cf-b12b6ae33c7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-8a095a85-792a-4c7d-87cf-b12b6ae33c7e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:05:17.764494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:17.765521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:17.766475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:17.766475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:17.766475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:17.766475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:17.766475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:05:17.766475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:05:17.766475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:05:17.766475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:05:17.766475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:05:17.766475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:05:17.766475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:05:17.766475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:17.766475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 446/891] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 20:05:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:17.771860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd80c01-e960-49e4-912c-871b52138fcf None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:17.812673 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6d11b-2f37-43ee-ab1e-9050539bacd0 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:17.813050 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6d11b-2f37-43ee-ab1e-9050539bacd0 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:17.813268 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6d11b-2f37-43ee-ab1e-9050539bacd0 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:17.818369 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 447/892] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:05:17.819912 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dfef2-792b-4638-8897-0b2fa01be5c7 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:17.820195 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dfef2-792b-4638-8897-0b2fa01be5c7 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:17.820470 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dfef2-792b-4638-8897-0b2fa01be5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:17.837402 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-972dfef2-792b-4638-8897-0b2fa01be5c7 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:17.838619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-972dfef2-792b-4638-8897-0b2fa01be5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:17.879838 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd80c01-e960-49e4-912c-871b52138fcf None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:17.916440 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 448/893] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:17.919501 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfeefa7-94a8-4bd2-bf8b-4a158dd2ea5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:17.920268 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeefa7-94a8-4bd2-bf8b-4a158dd2ea5c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:17.920622 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeefa7-94a8-4bd2-bf8b-4a158dd2ea5c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:17.920968 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeefa7-94a8-4bd2-bf8b-4a158dd2ea5c None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:17.921794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfeefa7-94a8-4bd2-bf8b-4a158dd2ea5c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:17.922170 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfeefa7-94a8-4bd2-bf8b-4a158dd2ea5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:17.923795 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfeefa7-94a8-4bd2-bf8b-4a158dd2ea5c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:17.933332 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 446/894] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 20:05:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:17.939015 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972dfef2-792b-4638-8897-0b2fa01be5c7 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:17.939237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfeefa7-94a8-4bd2-bf8b-4a158dd2ea5c None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:17.959115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:18.055821 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972dfef2-792b-4638-8897-0b2fa01be5c7 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:18.060958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfeefa7-94a8-4bd2-bf8b-4a158dd2ea5c None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:18.075750 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:18.097194 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6f51f-df61-4f71-a213-4e5e6e5a9c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:18.097895 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6f51f-df61-4f71-a213-4e5e6e5a9c30 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:18.098192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6f51f-df61-4f71-a213-4e5e6e5a9c30 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:18.098510 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6f51f-df61-4f71-a213-4e5e6e5a9c30 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:18.099247 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6f51f-df61-4f71-a213-4e5e6e5a9c30 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:18.100643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6f51f-df61-4f71-a213-4e5e6e5a9c30 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:18.107637 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebae5e-0ba4-4141-bea7-752a67a41927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:18.109114 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebae5e-0ba4-4141-bea7-752a67a41927 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:18.109716 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebae5e-0ba4-4141-bea7-752a67a41927 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:18.110345 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebae5e-0ba4-4141-bea7-752a67a41927 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:18.118096 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 449/895] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:05:17 2022] POST /identity/v3/services => generated 241 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:05:18.126076 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebae5e-0ba4-4141-bea7-752a67a41927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:18.128080 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebae5e-0ba4-4141-bea7-752a67a41927 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:18.149478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-a3ebae5e-0ba4-4141-bea7-752a67a41927 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:18.149478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:18.149478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:18.149478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:18.149478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:18.149478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:18.149478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:18.149478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:18.149478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:18.149478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:18.149478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:18.149478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:18.149478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:18.149478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:18.149478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:18.151531 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:18.153042 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:18.153042 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:18.153042 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:18.153042 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:18.155403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 447/896] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:18.161706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6f51f-df61-4f71-a213-4e5e6e5a9c30 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:18.305537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6f51f-df61-4f71-a213-4e5e6e5a9c30 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:18.344961 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26536149-708e-4a96-8139-d70a1bf1a580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:18.345558 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26536149-708e-4a96-8139-d70a1bf1a580 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:18.345842 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26536149-708e-4a96-8139-d70a1bf1a580 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:18.346118 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26536149-708e-4a96-8139-d70a1bf1a580 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:18.346885 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26536149-708e-4a96-8139-d70a1bf1a580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:18.347193 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26536149-708e-4a96-8139-d70a1bf1a580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:18.348601 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26536149-708e-4a96-8139-d70a1bf1a580 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:18.361533 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 450/897] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:18.367369 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebae5e-0ba4-4141-bea7-752a67a41927 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:18.440567 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:18.462519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389319e0-88bf-42c7-bf81-0e3566c26df0 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:18.462902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389319e0-88bf-42c7-bf81-0e3566c26df0 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:18.463414 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389319e0-88bf-42c7-bf81-0e3566c26df0 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:18.465148 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 451/898] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:05:18.470794 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d372a05-b3d1-4efb-bd3d-3bbdd3a49dfe None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:18.471084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d372a05-b3d1-4efb-bd3d-3bbdd3a49dfe None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:18.471371 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d372a05-b3d1-4efb-bd3d-3bbdd3a49dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:18.485811 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebae5e-0ba4-4141-bea7-752a67a41927 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:18.489388 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-1d372a05-b3d1-4efb-bd3d-3bbdd3a49dfe None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:18.490624 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-1d372a05-b3d1-4efb-bd3d-3bbdd3a49dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:18.522726 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:18.533324 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56122b-16bf-43b7-9758-43095ff13d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:18.534615 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56122b-16bf-43b7-9758-43095ff13d62 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:18.535121 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56122b-16bf-43b7-9758-43095ff13d62 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:18.535593 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56122b-16bf-43b7-9758-43095ff13d62 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:18.544025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56122b-16bf-43b7-9758-43095ff13d62 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:18.545805 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56122b-16bf-43b7-9758-43095ff13d62 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-3d56122b-16bf-43b7-9758-43095ff13d62 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:05:18.561711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:18.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:18.563855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:18.563855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:18.563855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:05:18.563855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:05:18.563855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:05:18.563855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:05:18.563855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:05:18.563855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:05:18.563855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:05:18.563855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:18.571068 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 448/899] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:05:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:18.572974 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56122b-16bf-43b7-9758-43095ff13d62 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:18.574092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 452/900] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:05:18.597923 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d372a05-b3d1-4efb-bd3d-3bbdd3a49dfe None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:18.680952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56122b-16bf-43b7-9758-43095ff13d62 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:18.706225 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:18.706622 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d372a05-b3d1-4efb-bd3d-3bbdd3a49dfe None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:18.719650 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dff4a4-6616-49ce-9b5f-18f80e355c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:18.720758 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dff4a4-6616-49ce-9b5f-18f80e355c5e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:18.721243 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dff4a4-6616-49ce-9b5f-18f80e355c5e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:18.721752 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dff4a4-6616-49ce-9b5f-18f80e355c5e None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:18.722825 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dff4a4-6616-49ce-9b5f-18f80e355c5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:18.723350 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dff4a4-6616-49ce-9b5f-18f80e355c5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:18.725016 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dff4a4-6616-49ce-9b5f-18f80e355c5e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:18.750101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 449/901] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 20:05:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:18.750487 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d5ad6-23d7-46ac-9cf9-370dd64d4095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:18.751424 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d5ad6-23d7-46ac-9cf9-370dd64d4095 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:18.751708 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d5ad6-23d7-46ac-9cf9-370dd64d4095 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:18.752038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d5ad6-23d7-46ac-9cf9-370dd64d4095 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:18.752710 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d5ad6-23d7-46ac-9cf9-370dd64d4095 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:18.752958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dff4a4-6616-49ce-9b5f-18f80e355c5e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:18.754204 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d5ad6-23d7-46ac-9cf9-370dd64d4095 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-6f1d5ad6-23d7-46ac-9cf9-370dd64d4095 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:18.768274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:18.769349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:18.770686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:18.770686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:18.770686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:18.770686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:18.770686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:18.770686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:18.770686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:18.770686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:18.770686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:18.770686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:18.770686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:18.770686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:18.770686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:18.770686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:05:18.770686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:18.774389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 453/902] 10.222.0.84 () {56 vars in 1132 bytes} [Thu May 19 20:05:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:18.776169 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d5ad6-23d7-46ac-9cf9-370dd64d4095 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:18.815744 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:18.877111 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dff4a4-6616-49ce-9b5f-18f80e355c5e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:18.904925 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d5ad6-23d7-46ac-9cf9-370dd64d4095 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:18.927890 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eedc43-1940-444a-94a7-8a051ca95671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:18.929224 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eedc43-1940-444a-94a7-8a051ca95671 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:18.929734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eedc43-1940-444a-94a7-8a051ca95671 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:18.930461 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eedc43-1940-444a-94a7-8a051ca95671 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:18.947205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88cfdb-ff49-4be9-915e-dd06d9428b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:18.947919 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88cfdb-ff49-4be9-915e-dd06d9428b92 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:18.948202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88cfdb-ff49-4be9-915e-dd06d9428b92 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:18.948478 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88cfdb-ff49-4be9-915e-dd06d9428b92 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:18.949201 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88cfdb-ff49-4be9-915e-dd06d9428b92 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:18.949509 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88cfdb-ff49-4be9-915e-dd06d9428b92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:18.950920 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88cfdb-ff49-4be9-915e-dd06d9428b92 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:18.961077 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 454/903] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:05:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:18.962012 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88cfdb-ff49-4be9-915e-dd06d9428b92 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:19.002826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:19.027048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eedc43-1940-444a-94a7-8a051ca95671 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:19.028577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eedc43-1940-444a-94a7-8a051ca95671 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:19.090727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:19.105054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88cfdb-ff49-4be9-915e-dd06d9428b92 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:19.141897 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 450/904] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:05:18 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:05:19.143335 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eedc43-1940-444a-94a7-8a051ca95671 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:19.147752 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65240092-cfd7-4c4f-99d1-c568b68201bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:19.148665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65240092-cfd7-4c4f-99d1-c568b68201bf None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:19.149116 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65240092-cfd7-4c4f-99d1-c568b68201bf None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:19.149560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65240092-cfd7-4c4f-99d1-c568b68201bf None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:19.151275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65240092-cfd7-4c4f-99d1-c568b68201bf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:19.151772 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65240092-cfd7-4c4f-99d1-c568b68201bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:19.153828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65240092-cfd7-4c4f-99d1-c568b68201bf None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:19.165854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 455/905] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:05:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:19.166967 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65240092-cfd7-4c4f-99d1-c568b68201bf None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:19.241423 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:19.263856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eedc43-1940-444a-94a7-8a051ca95671 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:19.285331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65240092-cfd7-4c4f-99d1-c568b68201bf None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:19.301073 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04c889-d0a1-4ea3-945c-43a595513309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:19.302143 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04c889-d0a1-4ea3-945c-43a595513309 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:19.302628 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04c889-d0a1-4ea3-945c-43a595513309 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:19.303064 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04c889-d0a1-4ea3-945c-43a595513309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:19.313549 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04c889-d0a1-4ea3-945c-43a595513309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:19.315046 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04c889-d0a1-4ea3-945c-43a595513309 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:19.321106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65e7c4-49a0-48d9-9d24-7b61b0f27a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:19.322911 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65e7c4-49a0-48d9-9d24-7b61b0f27a9c None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:19.323215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65e7c4-49a0-48d9-9d24-7b61b0f27a9c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:19.323492 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65e7c4-49a0-48d9-9d24-7b61b0f27a9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:19.324303 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65e7c4-49a0-48d9-9d24-7b61b0f27a9c None admin] RBAC: Adding query filter params (interface=public, service_id=f6b9839073de4dcb9e8726d1642cef9b, region_id=RegionOne) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:19.324612 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65e7c4-49a0-48d9-9d24-7b61b0f27a9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:19.326000 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65e7c4-49a0-48d9-9d24-7b61b0f27a9c None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:19.331380 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 456/906] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:05:19 2022] GET /identity/v3/endpoints?service_id=f6b9839073de4dcb9e8726d1642cef9b&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:19.332631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f6dfd-5865-410a-a4c4-6d1c43787780 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:19.332920 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f6dfd-5865-410a-a4c4-6d1c43787780 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:19.333198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f6dfd-5865-410a-a4c4-6d1c43787780 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:19.334734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-3e04c889-d0a1-4ea3-945c-43a595513309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:19.334734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:19.334734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:19.334734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:19.334734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:19.334734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:19.334734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:19.334734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:19.334734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:19.334734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:19.334734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:19.334734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:19.334734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:19.334734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:19.334734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:19.336108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:19.337363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:19.337363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:19.337363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:19.337363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:19.338714 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 457/907] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:19.343644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f6dfd-5865-410a-a4c4-6d1c43787780 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:19.343779 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 451/908] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:19.343779 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb679765-8579-4037-b36f-807d3cddf560 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:19.343959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb679765-8579-4037-b36f-807d3cddf560 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:19.344814 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb679765-8579-4037-b36f-807d3cddf560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:19.364678 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-bb679765-8579-4037-b36f-807d3cddf560 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:19.365941 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-bb679765-8579-4037-b36f-807d3cddf560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:19.366637 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:19.442409 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 452/909] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:19.443065 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb679765-8579-4037-b36f-807d3cddf560 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:19.467923 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f6dfd-5865-410a-a4c4-6d1c43787780 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:19.506641 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d144394-6120-4950-a937-dd5c48f8e14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:19.507317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d144394-6120-4950-a937-dd5c48f8e14c None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:19.507679 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d144394-6120-4950-a937-dd5c48f8e14c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:19.507870 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d144394-6120-4950-a937-dd5c48f8e14c None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:19.508562 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d144394-6120-4950-a937-dd5c48f8e14c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:19.510614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d144394-6120-4950-a937-dd5c48f8e14c None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:19.513408 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:19.522298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 458/910] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:05:19 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:05:19.522298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d144394-6120-4950-a937-dd5c48f8e14c None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:19.564966 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb679765-8579-4037-b36f-807d3cddf560 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:19.610699 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2274e8-3b39-4f15-97de-d9bc3b26c9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:19.611657 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2274e8-3b39-4f15-97de-d9bc3b26c9f5 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:19.612147 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2274e8-3b39-4f15-97de-d9bc3b26c9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:19.612421 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2274e8-3b39-4f15-97de-d9bc3b26c9f5 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:19.613195 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2274e8-3b39-4f15-97de-d9bc3b26c9f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:19.613483 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2274e8-3b39-4f15-97de-d9bc3b26c9f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:19.614977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2274e8-3b39-4f15-97de-d9bc3b26c9f5 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:19.640635 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 453/911] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:19.646605 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:19.647425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2274e8-3b39-4f15-97de-d9bc3b26c9f5 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:19.674136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d144394-6120-4950-a937-dd5c48f8e14c None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:19.714680 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a3e7e-c2cc-4126-b21b-56cf6c718d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:19.715426 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a3e7e-c2cc-4126-b21b-56cf6c718d3b None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:19.715707 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a3e7e-c2cc-4126-b21b-56cf6c718d3b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:19.715980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a3e7e-c2cc-4126-b21b-56cf6c718d3b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:19.716655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a3e7e-c2cc-4126-b21b-56cf6c718d3b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:19.718060 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a3e7e-c2cc-4126-b21b-56cf6c718d3b None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-366a3e7e-c2cc-4126-b21b-56cf6c718d3b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:19.735214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:19.736358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:19.737286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:19.737286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:19.737286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:19.737286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:19.737286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:19.737286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:19.737286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:19.737286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:19.737286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:19.737286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:19.737286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:19.737286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:19.737286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:05:19.737286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:19.737286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 459/912] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:05:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:19.743022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a3e7e-c2cc-4126-b21b-56cf6c718d3b None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:19.797737 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2274e8-3b39-4f15-97de-d9bc3b26c9f5 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:19.835793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:19.845205 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdefe66-92b6-437f-90ea-d444792cb580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:19.846299 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdefe66-92b6-437f-90ea-d444792cb580 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:19.846583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdefe66-92b6-437f-90ea-d444792cb580 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:19.846859 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdefe66-92b6-437f-90ea-d444792cb580 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:19.856966 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdefe66-92b6-437f-90ea-d444792cb580 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:19.859148 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a3e7e-c2cc-4126-b21b-56cf6c718d3b None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:19.859818 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdefe66-92b6-437f-90ea-d444792cb580 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:19.879426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-abdefe66-92b6-437f-90ea-d444792cb580 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:19.879426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:19.879426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:19.879426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:19.879426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:19.879426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:19.879426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:19.879426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:19.879426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:19.879426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:19.879426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:19.879426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:19.879426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:05:19.879426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:05:19.879426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:19.880491 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:19.881452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:05:19.881452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:05:19.881452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:05:19.881452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:05:19.881452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:05:19.881452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:05:19.881452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:19.881452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:19.885810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 454/913] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:19.888009 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdefe66-92b6-437f-90ea-d444792cb580 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:19.902147 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0daf5-cdfd-4879-9108-5ed0861afe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:19.903277 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0daf5-cdfd-4879-9108-5ed0861afe74 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:19.903775 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0daf5-cdfd-4879-9108-5ed0861afe74 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:19.904281 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0daf5-cdfd-4879-9108-5ed0861afe74 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:19.905210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0daf5-cdfd-4879-9108-5ed0861afe74 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:19.905759 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0daf5-cdfd-4879-9108-5ed0861afe74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:19.907399 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0daf5-cdfd-4879-9108-5ed0861afe74 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:19.913262 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 460/914] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:05:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:19.922283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0daf5-cdfd-4879-9108-5ed0861afe74 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:19.932308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:20.024157 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdefe66-92b6-437f-90ea-d444792cb580 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:20.036597 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0daf5-cdfd-4879-9108-5ed0861afe74 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:20.070078 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57db137-d836-47b9-a43f-8198d60f8c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:20.071123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57db137-d836-47b9-a43f-8198d60f8c3d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:20.071560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57db137-d836-47b9-a43f-8198d60f8c3d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:20.072138 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57db137-d836-47b9-a43f-8198d60f8c3d None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:20.073644 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57db137-d836-47b9-a43f-8198d60f8c3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:20.074008 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bee891-585c-4d07-932c-34c3fb06349f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:20.074359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57db137-d836-47b9-a43f-8198d60f8c3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:20.074721 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bee891-585c-4d07-932c-34c3fb06349f None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:20.075002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bee891-585c-4d07-932c-34c3fb06349f None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:20.075275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bee891-585c-4d07-932c-34c3fb06349f None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:20.075990 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bee891-585c-4d07-932c-34c3fb06349f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:20.076292 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bee891-585c-4d07-932c-34c3fb06349f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:20.076817 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57db137-d836-47b9-a43f-8198d60f8c3d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:20.077645 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bee891-585c-4d07-932c-34c3fb06349f None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:20.089046 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 461/915] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:05:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:20.089636 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bee891-585c-4d07-932c-34c3fb06349f None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:20.098632 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 455/916] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:05:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:20.105416 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57db137-d836-47b9-a43f-8198d60f8c3d None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:20.174902 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:20.179752 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:20.210625 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bee891-585c-4d07-932c-34c3fb06349f None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:20.247976 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ad19a-a733-43a0-a3fb-f6867893f9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:20.248563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ad19a-a733-43a0-a3fb-f6867893f9a4 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:20.248844 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ad19a-a733-43a0-a3fb-f6867893f9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:20.249120 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ad19a-a733-43a0-a3fb-f6867893f9a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:20.249917 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ad19a-a733-43a0-a3fb-f6867893f9a4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:20.251427 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ad19a-a733-43a0-a3fb-f6867893f9a4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:20.252205 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57db137-d836-47b9-a43f-8198d60f8c3d None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:20.279684 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 462/917] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:05:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:05:20.298010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1cb31-1169-4b55-9b9f-81f72a4de118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:20.299155 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1cb31-1169-4b55-9b9f-81f72a4de118 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:20.299642 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1cb31-1169-4b55-9b9f-81f72a4de118 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:20.300151 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1cb31-1169-4b55-9b9f-81f72a4de118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:20.301358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1cb31-1169-4b55-9b9f-81f72a4de118 None admin] RBAC: Adding query filter params (role.id=326ab279cac14b3fbafe4155443ed9b7, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=bdf915a5e0aa42a89e575ce1e8a65e2a) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:20.301915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1cb31-1169-4b55-9b9f-81f72a4de118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:20.303715 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1cb31-1169-4b55-9b9f-81f72a4de118 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:20.318444 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 456/918] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:05:20 2022] GET /identity/v3/role_assignments?user.id=bdf915a5e0aa42a89e575ce1e8a65e2a&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=326ab279cac14b3fbafe4155443ed9b7 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:20.537979 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991447d-b101-49db-a6f7-74e53bceb3e4 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:20.538592 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991447d-b101-49db-a6f7-74e53bceb3e4 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:20.538592 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991447d-b101-49db-a6f7-74e53bceb3e4 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:20.540125 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 463/919] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:05:20.545791 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13964160-cf86-4c19-b990-9ff2eeea3b0b None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:20.546695 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13964160-cf86-4c19-b990-9ff2eeea3b0b None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:20.547190 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13964160-cf86-4c19-b990-9ff2eeea3b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:20.564888 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-13964160-cf86-4c19-b990-9ff2eeea3b0b None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:20.566460 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-13964160-cf86-4c19-b990-9ff2eeea3b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:20.637710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 457/920] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:20.659808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991447d-b101-49db-a6f7-74e53bceb3e4 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:20.769328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991447d-b101-49db-a6f7-74e53bceb3e4 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:20.798644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:20.804545 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdabf5-ddb0-42b7-8ea9-a9ace58239d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:20.805671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdabf5-ddb0-42b7-8ea9-a9ace58239d9 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:20.806388 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdabf5-ddb0-42b7-8ea9-a9ace58239d9 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:20.806707 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdabf5-ddb0-42b7-8ea9-a9ace58239d9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:20.807493 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efdabf5-ddb0-42b7-8ea9-a9ace58239d9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:20.809032 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efdabf5-ddb0-42b7-8ea9-a9ace58239d9 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-6efdabf5-ddb0-42b7-8ea9-a9ace58239d9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:20.821719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:20.823056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:20.823972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:20.823972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:20.823972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:20.823972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:20.823972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:20.823972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:20.823972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:20.823972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:20.823972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:20.823972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:20.823972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:20.823972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:20.823972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:05:20.823972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:20.823972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 464/921] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:05:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:20.829488 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13964160-cf86-4c19-b990-9ff2eeea3b0b None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:20.864571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:20.940214 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13964160-cf86-4c19-b990-9ff2eeea3b0b None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:20.975690 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76766bfc-c947-48a9-9b4b-32e098bcf072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:20.976369 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76766bfc-c947-48a9-9b4b-32e098bcf072 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:20.976649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76766bfc-c947-48a9-9b4b-32e098bcf072 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:20.976926 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76766bfc-c947-48a9-9b4b-32e098bcf072 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:20.977640 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76766bfc-c947-48a9-9b4b-32e098bcf072 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:20.978054 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76766bfc-c947-48a9-9b4b-32e098bcf072 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:20.979586 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76766bfc-c947-48a9-9b4b-32e098bcf072 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:20.984927 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 458/922] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:05:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:20.990527 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdabf5-ddb0-42b7-8ea9-a9ace58239d9 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:21.094525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdabf5-ddb0-42b7-8ea9-a9ace58239d9 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:21.129287 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21343842-68c0-4a99-92b8-205253116517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:21.129968 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21343842-68c0-4a99-92b8-205253116517 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:21.130244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21343842-68c0-4a99-92b8-205253116517 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:21.130546 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21343842-68c0-4a99-92b8-205253116517 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:21.131256 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21343842-68c0-4a99-92b8-205253116517 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:21.131558 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21343842-68c0-4a99-92b8-205253116517 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:21.132923 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21343842-68c0-4a99-92b8-205253116517 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:21.138789 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 465/923] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:05:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:21.144781 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76766bfc-c947-48a9-9b4b-32e098bcf072 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:21.249261 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:21.253266 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76766bfc-c947-48a9-9b4b-32e098bcf072 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:21.289482 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d126b0-fb3b-4dcd-b6a8-b6fe2f831bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:21.290164 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d126b0-fb3b-4dcd-b6a8-b6fe2f831bc5 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:21.290640 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d126b0-fb3b-4dcd-b6a8-b6fe2f831bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:21.290988 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d126b0-fb3b-4dcd-b6a8-b6fe2f831bc5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:21.291793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d126b0-fb3b-4dcd-b6a8-b6fe2f831bc5 None admin] RBAC: Adding query filter params (interface=public, service_id=01a9df4a977c4d9cbd04b57a7ef86286, region_id=RegionOne) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:21.292077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d126b0-fb3b-4dcd-b6a8-b6fe2f831bc5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:21.293474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d126b0-fb3b-4dcd-b6a8-b6fe2f831bc5 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:21.301412 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 459/924] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:05:21 2022] GET /identity/v3/endpoints?service_id=01a9df4a977c4d9cbd04b57a7ef86286&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:21.307232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21343842-68c0-4a99-92b8-205253116517 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:21.367971 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:21.411638 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21343842-68c0-4a99-92b8-205253116517 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:21.446950 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c536e0a-6fd8-418a-aea4-f8ab64ba9f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:21.447648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c536e0a-6fd8-418a-aea4-f8ab64ba9f47 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:21.447928 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c536e0a-6fd8-418a-aea4-f8ab64ba9f47 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:21.448202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c536e0a-6fd8-418a-aea4-f8ab64ba9f47 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:21.448912 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c536e0a-6fd8-418a-aea4-f8ab64ba9f47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:21.450339 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c536e0a-6fd8-418a-aea4-f8ab64ba9f47 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:21.457558 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 466/925] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:05:21 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:05:21.927634 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bca6f8-ae86-4a01-85cb-082dbd8aecca None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:21.928379 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bca6f8-ae86-4a01-85cb-082dbd8aecca None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:21.928379 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bca6f8-ae86-4a01-85cb-082dbd8aecca None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:21.929710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 460/926] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:21.936235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b9de6-4a25-4e16-a0cf-b22f143be1a1 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:21.936235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b9de6-4a25-4e16-a0cf-b22f143be1a1 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:21.936235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b9de6-4a25-4e16-a0cf-b22f143be1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:21.954177 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-c91b9de6-4a25-4e16-a0cf-b22f143be1a1 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:21.955714 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-c91b9de6-4a25-4e16-a0cf-b22f143be1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:22.024449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 467/927] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:05:22.048022 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bca6f8-ae86-4a01-85cb-082dbd8aecca None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:22.155927 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bca6f8-ae86-4a01-85cb-082dbd8aecca None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:22.192276 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b715e7d-cab7-4d81-a65b-e062bd993c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:22.193475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b715e7d-cab7-4d81-a65b-e062bd993c26 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:22.193973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b715e7d-cab7-4d81-a65b-e062bd993c26 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:22.194493 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b715e7d-cab7-4d81-a65b-e062bd993c26 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:22.195366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b715e7d-cab7-4d81-a65b-e062bd993c26 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:22.197091 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b715e7d-cab7-4d81-a65b-e062bd993c26 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-0b715e7d-cab7-4d81-a65b-e062bd993c26 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:22.210354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:22.211387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:22.212405 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:22.212405 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:22.212405 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:22.212405 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:22.212405 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:22.212405 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:22.212405 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:22.212405 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:22.212405 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:22.212405 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:22.212405 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:22.212405 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:22.212405 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:22.212405 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:05:22.212405 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:22.215511 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 461/928] 10.222.0.84 () {56 vars in 1132 bytes} [Thu May 19 20:05:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:22.222380 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b9de6-4a25-4e16-a0cf-b22f143be1a1 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:22.278697 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:22.329520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b9de6-4a25-4e16-a0cf-b22f143be1a1 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:22.365684 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c22c12-a638-454a-aeb0-5fc6906fe06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:22.366815 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c22c12-a638-454a-aeb0-5fc6906fe06a None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:22.367333 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c22c12-a638-454a-aeb0-5fc6906fe06a None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:22.367838 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c22c12-a638-454a-aeb0-5fc6906fe06a None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:22.368973 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c22c12-a638-454a-aeb0-5fc6906fe06a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:22.369554 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c22c12-a638-454a-aeb0-5fc6906fe06a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:22.371275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c22c12-a638-454a-aeb0-5fc6906fe06a None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:22.377801 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 468/929] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:05:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:22.384241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b715e7d-cab7-4d81-a65b-e062bd993c26 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:22.415817 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:22.490234 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b715e7d-cab7-4d81-a65b-e062bd993c26 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:22.524687 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e613e0-d43a-4dc4-a7f4-ac56a5eb82dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:22.525445 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e613e0-d43a-4dc4-a7f4-ac56a5eb82dd None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:22.525728 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e613e0-d43a-4dc4-a7f4-ac56a5eb82dd None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:22.526008 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e613e0-d43a-4dc4-a7f4-ac56a5eb82dd None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:22.526755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e613e0-d43a-4dc4-a7f4-ac56a5eb82dd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:22.527065 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e613e0-d43a-4dc4-a7f4-ac56a5eb82dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:22.528471 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e613e0-d43a-4dc4-a7f4-ac56a5eb82dd None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:22.533938 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 462/930] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:05:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:22.539365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c22c12-a638-454a-aeb0-5fc6906fe06a None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:22.576927 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb8982-1b7e-4344-801a-22ec2c0a2471 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:22.577291 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb8982-1b7e-4344-801a-22ec2c0a2471 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:22.577477 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb8982-1b7e-4344-801a-22ec2c0a2471 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:22.579247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 463/931] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:05:22.585139 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c5df1-2d1f-4a2b-b4cf-baef7967c1ca None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:22.585416 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c5df1-2d1f-4a2b-b4cf-baef7967c1ca None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:22.585694 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c5df1-2d1f-4a2b-b4cf-baef7967c1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:22.605038 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-b23c5df1-2d1f-4a2b-b4cf-baef7967c1ca None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:22.606246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-b23c5df1-2d1f-4a2b-b4cf-baef7967c1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:22.657278 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c22c12-a638-454a-aeb0-5fc6906fe06a None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:22.678321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:22.691410 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 464/932] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:05:22.699949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c99d24-b661-410e-aaa5-8c644b407694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:22.700722 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c99d24-b661-410e-aaa5-8c644b407694 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:22.701020 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c99d24-b661-410e-aaa5-8c644b407694 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:22.701299 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c99d24-b661-410e-aaa5-8c644b407694 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:22.702060 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c99d24-b661-410e-aaa5-8c644b407694 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:22.703488 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c99d24-b661-410e-aaa5-8c644b407694 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:22.719016 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c5df1-2d1f-4a2b-b4cf-baef7967c1ca None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:22.724975 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 469/933] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:05:22 2022] POST /identity/v3/endpoints => generated 347 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:05:22.747704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:22.794797 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068f5ca-630c-437f-bcd8-d8a348502109 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:22.795134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068f5ca-630c-437f-bcd8-d8a348502109 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:22.795363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068f5ca-630c-437f-bcd8-d8a348502109 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:22.801187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 470/934] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:05:22.803376 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875e68b-9bb6-4b92-a9e6-5b6ff64f4ba4 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:22.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875e68b-9bb6-4b92-a9e6-5b6ff64f4ba4 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:22.804157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875e68b-9bb6-4b92-a9e6-5b6ff64f4ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:22.824380 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-6875e68b-9bb6-4b92-a9e6-5b6ff64f4ba4 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:22.826087 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-6875e68b-9bb6-4b92-a9e6-5b6ff64f4ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:22.836365 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c5df1-2d1f-4a2b-b4cf-baef7967c1ca None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:22.890468 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eade510-e0fc-4181-af18-62693db78374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:22.891146 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eade510-e0fc-4181-af18-62693db78374 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:22.891426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eade510-e0fc-4181-af18-62693db78374 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:22.891697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eade510-e0fc-4181-af18-62693db78374 None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:22.892929 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eade510-e0fc-4181-af18-62693db78374 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:22.894714 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eade510-e0fc-4181-af18-62693db78374 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:22.896767 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: WARNING keystone.server.flask.common [None req-8eade510-e0fc-4181-af18-62693db78374 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 20:05:22.907672 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 471/935] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:22.927038 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.sql.core [None req-8eade510-e0fc-4181-af18-62693db78374 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:05:22.927038 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 20:05:22.927038 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [parameters: {'id': 'aec0e1fbc5c14c29bdd3a68ef30ec3a9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 20:05:22.927038 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=214314) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:05:22.932596 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875e68b-9bb6-4b92-a9e6-5b6ff64f4ba4 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:22.950141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-8eade510-e0fc-4181-af18-62693db78374 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id aec0e1fbc5c14c29bdd3a68ef30ec3a9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id aec0e1fbc5c14c29bdd3a68ef30ec3a9. May 19 20:05:22.950141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:22.950141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:05:22.950141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:05:22.950141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:05:22.950141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:05:22.950141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:05:22.950141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:05:22.950141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:05:22.950141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application conn.query(q) May 19 20:05:22.950141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:05:22.950141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:05:22.950141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application result.read() May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 20:05:22.951538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application next(self.gen) May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application yield resource May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application next(self.gen) May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise self.value May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application session.commit() May 19 20:05:22.952564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self.session.flush() May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application compat.raise_( May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:05:22.953583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:05:22.954587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:05:22.954587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:05:22.954587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:05:22.954587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:05:22.954587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 20:05:22.954587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 20:05:22.954587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:05:22.954587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:05:22.954587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:05:22.954587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:05:22.954587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:05:22.954587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:05:22.954587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:05:22.954587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:05:22.954587 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application conn.query(q) May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application result.read() May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:05:22.955474 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:05:22.956404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:05:22.956404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:05:22.956404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:05:22.956404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:05:22.956404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:05:22.956404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:05:22.956404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:05:22.956404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 20:05:22.956404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [parameters: {'id': 'aec0e1fbc5c14c29bdd3a68ef30ec3a9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 20:05:22.956404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:05:22.956404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:22.956404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:22.956404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:22.956404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:22.956404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:22.957257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:22.958244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:22.958244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 20:05:22.958244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 20:05:22.958244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:22.958244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:22.958244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 20:05:22.958244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 20:05:22.958244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id aec0e1fbc5c14c29bdd3a68ef30ec3a9. May 19 20:05:22.958244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:22.958244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 465/936] 10.222.0.84 () {60 vars in 1140 bytes} [Thu May 19 20:05:22 2022] POST /identity/v3/projects => generated 262 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:22.958244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eade510-e0fc-4181-af18-62693db78374 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:23.053330 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:23.063565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875e68b-9bb6-4b92-a9e6-5b6ff64f4ba4 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:23.065388 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:23.101617 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eade510-e0fc-4181-af18-62693db78374 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:23.106058 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261af7e-54d1-435c-a82d-b0196966fdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:23.107039 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261af7e-54d1-435c-a82d-b0196966fdbe None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:23.107418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261af7e-54d1-435c-a82d-b0196966fdbe None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:23.107789 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261af7e-54d1-435c-a82d-b0196966fdbe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:23.120082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261af7e-54d1-435c-a82d-b0196966fdbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-6261af7e-54d1-435c-a82d-b0196966fdbe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:05:23.128983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:23.130197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:23.132228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:23.132228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:23.132228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:23.132228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:23.132228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:05:23.132228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:05:23.132228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:05:23.132228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:05:23.132228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:05:23.132228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:05:23.132228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:05:23.132228 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:23.137064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 472/937] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 20:05:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:23.138225 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261af7e-54d1-435c-a82d-b0196966fdbe None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:23.145125 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273823b-427a-4e24-ab25-6159d69cac66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:23.146919 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273823b-427a-4e24-ab25-6159d69cac66 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:23.147212 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273823b-427a-4e24-ab25-6159d69cac66 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:23.147501 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273823b-427a-4e24-ab25-6159d69cac66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:23.156434 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273823b-427a-4e24-ab25-6159d69cac66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:23.158744 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273823b-427a-4e24-ab25-6159d69cac66 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:23.176544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-a273823b-427a-4e24-ab25-6159d69cac66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:23.176544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:23.176544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:23.176544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:23.176544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:23.176544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:23.176544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:23.176544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:23.176544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:23.176544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:23.176544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:23.176544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:23.176544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:05:23.176544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:05:23.176544 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:23.178676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:23.180284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:05:23.180284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:05:23.180284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:05:23.180284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:05:23.180284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:05:23.180284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:05:23.180284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:23.180284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:23.180284 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 466/938] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:23.185262 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273823b-427a-4e24-ab25-6159d69cac66 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:23.260713 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261af7e-54d1-435c-a82d-b0196966fdbe None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:23.299043 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42033c5-0088-4b26-bc42-a44d29daf7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:23.299979 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42033c5-0088-4b26-bc42-a44d29daf7f8 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:23.300359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42033c5-0088-4b26-bc42-a44d29daf7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:23.300727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42033c5-0088-4b26-bc42-a44d29daf7f8 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:23.301585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42033c5-0088-4b26-bc42-a44d29daf7f8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:23.302172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42033c5-0088-4b26-bc42-a44d29daf7f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:23.303795 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42033c5-0088-4b26-bc42-a44d29daf7f8 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:23.316756 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273823b-427a-4e24-ab25-6159d69cac66 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:23.318971 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 473/939] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 20:05:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:23.320494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42033c5-0088-4b26-bc42-a44d29daf7f8 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:23.350175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:23.358542 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:23.366294 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2af4a-9cb0-4465-86df-49f42aa8f090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:23.367022 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2af4a-9cb0-4465-86df-49f42aa8f090 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:23.367307 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2af4a-9cb0-4465-86df-49f42aa8f090 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:23.367589 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2af4a-9cb0-4465-86df-49f42aa8f090 None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:23.368357 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2af4a-9cb0-4465-86df-49f42aa8f090 None admin] RBAC: Adding query filter params (name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:23.368705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2af4a-9cb0-4465-86df-49f42aa8f090 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:23.370171 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2af4a-9cb0-4465-86df-49f42aa8f090 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:23.402462 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 467/940] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:05:23 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:23.446614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42033c5-0088-4b26-bc42-a44d29daf7f8 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:23.485203 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea5257-edc7-49e8-bf32-037b1b7f9ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:23.486478 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea5257-edc7-49e8-bf32-037b1b7f9ec4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:23.486998 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea5257-edc7-49e8-bf32-037b1b7f9ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:23.487494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea5257-edc7-49e8-bf32-037b1b7f9ec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:23.500471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea5257-edc7-49e8-bf32-037b1b7f9ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:23.502445 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea5257-edc7-49e8-bf32-037b1b7f9ec4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:23.522275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-cfea5257-edc7-49e8-bf32-037b1b7f9ec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:23.522275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:23.522275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:23.522275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:23.522275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:23.522275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:23.522275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:23.522275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:23.522275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:23.522275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:23.522275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:23.522275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:23.522275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:23.522275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:23.522275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:23.523318 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:23.524317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:23.524317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:23.524317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:23.524317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:23.531731 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 474/941] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:23.533549 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2af4a-9cb0-4465-86df-49f42aa8f090 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:23.654426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2af4a-9cb0-4465-86df-49f42aa8f090 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:23.694620 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e6b0c-2e27-4e79-95d1-3b153f003960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:23.695301 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e6b0c-2e27-4e79-95d1-3b153f003960 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:23.695631 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e6b0c-2e27-4e79-95d1-3b153f003960 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:23.695959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e6b0c-2e27-4e79-95d1-3b153f003960 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:23.696673 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e6b0c-2e27-4e79-95d1-3b153f003960 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:23.696982 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e6b0c-2e27-4e79-95d1-3b153f003960 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:23.698480 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e6b0c-2e27-4e79-95d1-3b153f003960 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:23.711135 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 468/942] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:23.717239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea5257-edc7-49e8-bf32-037b1b7f9ec4 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:23.799932 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:23.818817 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ee823-155e-4e9e-b556-c0a2ccbdab8e None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:23.819204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ee823-155e-4e9e-b556-c0a2ccbdab8e None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:23.819391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ee823-155e-4e9e-b556-c0a2ccbdab8e None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:23.821025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 469/943] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:05:23.826477 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaab0cf-471c-490d-a715-f157e34a328b None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:23.826758 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaab0cf-471c-490d-a715-f157e34a328b None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:23.827030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaab0cf-471c-490d-a715-f157e34a328b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:23.828670 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea5257-edc7-49e8-bf32-037b1b7f9ec4 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:23.846648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-cfaab0cf-471c-490d-a715-f157e34a328b None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:23.848217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-cfaab0cf-471c-490d-a715-f157e34a328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:23.875544 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd468d9-d305-4db8-ae0a-a9f137f6e5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:23.876826 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd468d9-d305-4db8-ae0a-a9f137f6e5a1 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:23.877534 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd468d9-d305-4db8-ae0a-a9f137f6e5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:23.878445 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd468d9-d305-4db8-ae0a-a9f137f6e5a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:23.889688 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd468d9-d305-4db8-ae0a-a9f137f6e5a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:23.891487 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd468d9-d305-4db8-ae0a-a9f137f6e5a1 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:23.901699 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-3dd468d9-d305-4db8-ae0a-a9f137f6e5a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:05:23.913969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:23.915092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:23.916331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:23.916331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:23.916331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:05:23.916331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:05:23.916331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:05:23.916331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:05:23.916331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:05:23.916331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:05:23.916331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:05:23.916331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:23.923667 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 475/944] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:05:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:23.925345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd468d9-d305-4db8-ae0a-a9f137f6e5a1 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:23.926704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 470/945] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:23.950164 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaab0cf-471c-490d-a715-f157e34a328b None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:24.038932 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd468d9-d305-4db8-ae0a-a9f137f6e5a1 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:24.054431 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaab0cf-471c-490d-a715-f157e34a328b None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:24.076764 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e71f9-9dea-4941-a7da-7f20431bcffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:24.078173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e71f9-9dea-4941-a7da-7f20431bcffa None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:24.078677 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e71f9-9dea-4941-a7da-7f20431bcffa None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:24.079254 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e71f9-9dea-4941-a7da-7f20431bcffa None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:24.080290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e71f9-9dea-4941-a7da-7f20431bcffa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:24.080796 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e71f9-9dea-4941-a7da-7f20431bcffa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:24.082670 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e71f9-9dea-4941-a7da-7f20431bcffa None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:24.085814 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:24.091335 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f171e-db47-45c0-b9eb-b3aac650f7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:24.092243 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f171e-db47-45c0-b9eb-b3aac650f7e9 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:24.092586 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f171e-db47-45c0-b9eb-b3aac650f7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:24.093465 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f171e-db47-45c0-b9eb-b3aac650f7e9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:24.094683 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f171e-db47-45c0-b9eb-b3aac650f7e9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:24.096350 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f171e-db47-45c0-b9eb-b3aac650f7e9 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-4e4f171e-db47-45c0-b9eb-b3aac650f7e9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:24.111216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:24.112548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:24.113664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:24.113664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:24.113664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:24.113664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:24.113664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:24.113664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:24.113664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:24.113664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:24.113664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:24.113664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:24.113664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:24.113664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:24.113664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:05:24.113664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:24.114738 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 476/947] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 20:05:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:24.116251 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e71f9-9dea-4941-a7da-7f20431bcffa None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:24.116737 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 471/947] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:05:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:24.117686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f171e-db47-45c0-b9eb-b3aac650f7e9 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:24.167580 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:24.223030 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e71f9-9dea-4941-a7da-7f20431bcffa None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:24.229157 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f171e-db47-45c0-b9eb-b3aac650f7e9 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:24.259638 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5441f-65c9-45b4-ab10-ee294cce182b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:24.260824 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5441f-65c9-45b4-ab10-ee294cce182b None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:24.261292 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5441f-65c9-45b4-ab10-ee294cce182b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:24.261770 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5441f-65c9-45b4-ab10-ee294cce182b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:24.267006 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0065cd-14e9-4460-a19e-7d2cfa9314a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:24.267752 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0065cd-14e9-4460-a19e-7d2cfa9314a1 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:24.268099 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0065cd-14e9-4460-a19e-7d2cfa9314a1 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:24.268457 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0065cd-14e9-4460-a19e-7d2cfa9314a1 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:24.269237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0065cd-14e9-4460-a19e-7d2cfa9314a1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:24.269607 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0065cd-14e9-4460-a19e-7d2cfa9314a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:24.271216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0065cd-14e9-4460-a19e-7d2cfa9314a1 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:24.275006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5441f-65c9-45b4-ab10-ee294cce182b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:24.276646 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5441f-65c9-45b4-ab10-ee294cce182b None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:24.281453 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 472/948] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:05:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:24.282401 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0065cd-14e9-4460-a19e-7d2cfa9314a1 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:24.296183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-c0a5441f-65c9-45b4-ab10-ee294cce182b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:24.296183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:24.296183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:24.296183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:24.296183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:24.296183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:24.296183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:24.296183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:24.296183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:24.296183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:24.296183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:24.296183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:24.296183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:24.296183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:24.296183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:24.297172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:24.298133 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:24.298133 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:24.298133 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:24.298133 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:24.304940 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 477/949] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:24.307089 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5441f-65c9-45b4-ab10-ee294cce182b None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:24.356243 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:24.394785 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0065cd-14e9-4460-a19e-7d2cfa9314a1 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:24.406205 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:24.412794 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5441f-65c9-45b4-ab10-ee294cce182b None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:24.433158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cced3-340a-4e5f-973b-e2cb0485900c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:24.433831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cced3-340a-4e5f-973b-e2cb0485900c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:24.434111 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cced3-340a-4e5f-973b-e2cb0485900c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:24.434428 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cced3-340a-4e5f-973b-e2cb0485900c None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:24.435139 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81cced3-340a-4e5f-973b-e2cb0485900c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:24.435769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81cced3-340a-4e5f-973b-e2cb0485900c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:24.436917 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81cced3-340a-4e5f-973b-e2cb0485900c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:24.442692 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 473/950] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:05:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:24.448225 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cced3-340a-4e5f-973b-e2cb0485900c None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:24.451453 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74032a-7abc-4774-93b4-11a27052e6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:24.452332 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74032a-7abc-4774-93b4-11a27052e6eb None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:24.452770 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74032a-7abc-4774-93b4-11a27052e6eb None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:24.453215 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74032a-7abc-4774-93b4-11a27052e6eb None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:24.454102 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74032a-7abc-4774-93b4-11a27052e6eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:24.454710 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74032a-7abc-4774-93b4-11a27052e6eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:24.456258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74032a-7abc-4774-93b4-11a27052e6eb None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:24.473457 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 478/951] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:24.474972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74032a-7abc-4774-93b4-11a27052e6eb None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:24.557626 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cced3-340a-4e5f-973b-e2cb0485900c None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:24.578490 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74032a-7abc-4774-93b4-11a27052e6eb None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:24.586178 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:24.597258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75a7ff-5708-484c-a439-2da60dcbb9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:24.598077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75a7ff-5708-484c-a439-2da60dcbb9c3 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:24.598368 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75a7ff-5708-484c-a439-2da60dcbb9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:24.598692 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75a7ff-5708-484c-a439-2da60dcbb9c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:24.599456 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75a7ff-5708-484c-a439-2da60dcbb9c3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:24.600834 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75a7ff-5708-484c-a439-2da60dcbb9c3 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:24.617390 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2359f-7ae6-439f-806e-1ade7cbe2d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:24.618731 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2359f-7ae6-439f-806e-1ade7cbe2d7b None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:24.619172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2359f-7ae6-439f-806e-1ade7cbe2d7b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:24.619626 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2359f-7ae6-439f-806e-1ade7cbe2d7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:24.626963 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 474/952] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:05:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:05:24.628175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2359f-7ae6-439f-806e-1ade7cbe2d7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:24.629909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2359f-7ae6-439f-806e-1ade7cbe2d7b None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:24.646910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-2ce2359f-7ae6-439f-806e-1ade7cbe2d7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:24.646910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:24.646910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:24.646910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:24.646910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:24.646910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:24.646910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:24.646910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:24.646910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:24.646910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:24.646910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:24.646910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:24.646910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:05:24.646910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:05:24.646910 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:24.648185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:24.649413 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:05:24.649413 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:05:24.649413 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:05:24.649413 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:05:24.649413 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:05:24.649413 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:05:24.649413 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:24.649413 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:24.656954 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 479/953] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:24.658824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75a7ff-5708-484c-a439-2da60dcbb9c3 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:24.706642 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:24.782339 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75a7ff-5708-484c-a439-2da60dcbb9c3 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:24.825386 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26485022-8f4d-4157-9d27-ce5e4f41667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:24.826125 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26485022-8f4d-4157-9d27-ce5e4f41667f None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:24.826437 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26485022-8f4d-4157-9d27-ce5e4f41667f None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:24.826711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26485022-8f4d-4157-9d27-ce5e4f41667f None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:24.827513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26485022-8f4d-4157-9d27-ce5e4f41667f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:24.827821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26485022-8f4d-4157-9d27-ce5e4f41667f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:24.829296 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26485022-8f4d-4157-9d27-ce5e4f41667f None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:24.844485 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 475/954] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:05:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:24.850613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2359f-7ae6-439f-806e-1ade7cbe2d7b None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:24.965915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2359f-7ae6-439f-806e-1ade7cbe2d7b None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:25.002543 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22e75a-ad21-44e7-881f-1d61ee96c048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:25.003851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22e75a-ad21-44e7-881f-1d61ee96c048 None admin] REQUEST_METHOD: `PUT` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:25.004375 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22e75a-ad21-44e7-881f-1d61ee96c048 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:25.004859 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22e75a-ad21-44e7-881f-1d61ee96c048 None admin] PATH_INFO: `/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/bdf915a5e0aa42a89e575ce1e8a65e2a/roles/326ab279cac14b3fbafe4155443ed9b7` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:25.007707 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22e75a-ad21-44e7-881f-1d61ee96c048 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba22fdc274b74521847e5fceb2d307b1, user_id=bdf915a5e0aa42a89e575ce1e8a65e2a, role_id=326ab279cac14b3fbafe4155443ed9b7)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:25.009502 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22e75a-ad21-44e7-881f-1d61ee96c048 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:25.034418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 480/955] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:05:24 2022] PUT /identity/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/bdf915a5e0aa42a89e575ce1e8a65e2a/roles/326ab279cac14b3fbafe4155443ed9b7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:05:25.098791 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4235842-595b-4a8c-b08b-09c7a42ffad5 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:25.099354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4235842-595b-4a8c-b08b-09c7a42ffad5 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:25.099354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4235842-595b-4a8c-b08b-09c7a42ffad5 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:25.100988 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 476/956] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:25.106947 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81425c56-2798-4700-b4cd-bff422f3849c None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:25.107471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81425c56-2798-4700-b4cd-bff422f3849c None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:25.107471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81425c56-2798-4700-b4cd-bff422f3849c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:25.130586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-81425c56-2798-4700-b4cd-bff422f3849c None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:25.132255 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-81425c56-2798-4700-b4cd-bff422f3849c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:25.156747 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:25.212588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 481/957] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:25.236773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4235842-595b-4a8c-b08b-09c7a42ffad5 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:25.332093 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:25.350685 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4235842-595b-4a8c-b08b-09c7a42ffad5 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:25.389759 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a99338-d7c7-432e-a742-7bedd60a3f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:25.390981 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a99338-d7c7-432e-a742-7bedd60a3f55 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:25.391514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a99338-d7c7-432e-a742-7bedd60a3f55 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:25.392017 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a99338-d7c7-432e-a742-7bedd60a3f55 None admin] PATH_INFO: `/v3/services/compute` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:25.392892 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a99338-d7c7-432e-a742-7bedd60a3f55 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:25.394624 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a99338-d7c7-432e-a742-7bedd60a3f55 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-f4a99338-d7c7-432e-a742-7bedd60a3f55 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:25.407681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:25.408854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:25.409775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:25.409775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:25.409775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:25.409775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:25.409775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:25.409775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:25.409775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:25.409775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:25.409775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:25.409775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:25.409775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:25.409775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:25.409775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:05:25.409775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:25.412784 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 477/958] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:25.418891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81425c56-2798-4700-b4cd-bff422f3849c None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:25.521652 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81425c56-2798-4700-b4cd-bff422f3849c None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:25.556713 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d905d-a147-4fe7-a90d-a0b4f0ce10f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:25.557393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d905d-a147-4fe7-a90d-a0b4f0ce10f4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:25.557673 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d905d-a147-4fe7-a90d-a0b4f0ce10f4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:25.557947 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d905d-a147-4fe7-a90d-a0b4f0ce10f4 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:25.558706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d905d-a147-4fe7-a90d-a0b4f0ce10f4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:25.559015 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d905d-a147-4fe7-a90d-a0b4f0ce10f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:25.560473 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d905d-a147-4fe7-a90d-a0b4f0ce10f4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:25.570358 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 482/959] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:05:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:25.572436 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a99338-d7c7-432e-a742-7bedd60a3f55 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:25.685121 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a99338-d7c7-432e-a742-7bedd60a3f55 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:25.720585 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66cbe9-0d16-4a92-9cca-e7d869b57ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:25.721292 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66cbe9-0d16-4a92-9cca-e7d869b57ae0 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:25.721573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66cbe9-0d16-4a92-9cca-e7d869b57ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:25.721847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66cbe9-0d16-4a92-9cca-e7d869b57ae0 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:25.722598 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66cbe9-0d16-4a92-9cca-e7d869b57ae0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:25.722906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66cbe9-0d16-4a92-9cca-e7d869b57ae0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:25.724307 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66cbe9-0d16-4a92-9cca-e7d869b57ae0 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:25.729823 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 478/960] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:05:25 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:25.820570 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be4421-33fa-4b76-9c6c-bc99a0b13a0f None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:25.820957 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be4421-33fa-4b76-9c6c-bc99a0b13a0f None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:25.821164 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be4421-33fa-4b76-9c6c-bc99a0b13a0f None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:25.822914 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 483/961] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:25.828634 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c30c55-62ff-4548-a9b8-43e709ecabf0 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:25.828969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c30c55-62ff-4548-a9b8-43e709ecabf0 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:25.829207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c30c55-62ff-4548-a9b8-43e709ecabf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:25.848211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-b0c30c55-62ff-4548-a9b8-43e709ecabf0 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:25.849453 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-b0c30c55-62ff-4548-a9b8-43e709ecabf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:25.862374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:25.932635 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 479/962] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:25.960399 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be4421-33fa-4b76-9c6c-bc99a0b13a0f None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:25.983344 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:26.068746 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be4421-33fa-4b76-9c6c-bc99a0b13a0f None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:26.104278 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e5257-27dd-459a-82f9-7ed1b24efc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:26.104889 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e5257-27dd-459a-82f9-7ed1b24efc57 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:26.105172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e5257-27dd-459a-82f9-7ed1b24efc57 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:26.105560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e5257-27dd-459a-82f9-7ed1b24efc57 None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:26.106290 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e5257-27dd-459a-82f9-7ed1b24efc57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:26.107743 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e5257-27dd-459a-82f9-7ed1b24efc57 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:26.109057 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: WARNING keystone.server.flask.common [None req-cc8e5257-27dd-459a-82f9-7ed1b24efc57 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 20:05:26.123192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.sql.core [None req-cc8e5257-27dd-459a-82f9-7ed1b24efc57 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:05:26.123192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 20:05:26.123192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [parameters: {'user_id': 'c616d6ffb28945079365f71237794277', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 20:05:26.123192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=214315) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:05:26.139319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-cc8e5257-27dd-459a-82f9-7ed1b24efc57 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 20:05:26.139319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:26.139319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:05:26.139319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:05:26.139319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:05:26.139319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:05:26.139319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:05:26.139319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:05:26.139319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:05:26.139319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application conn.query(q) May 19 20:05:26.139319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:05:26.139319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:05:26.139319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:05:26.139319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application result.read() May 19 20:05:26.139319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:05:26.140284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application next(self.gen) May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application yield resource May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application next(self.gen) May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise self.value May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application session.commit() May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:05:26.141305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self.session.flush() May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application compat.raise_( May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:05:26.142244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:05:26.143239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application conn.query(q) May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application result.read() May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:05:26.144140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:05:26.145002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:05:26.145002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:05:26.145002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:05:26.145002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:05:26.145002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:05:26.145002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 20:05:26.145002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c616d6ffb28945079365f71237794277', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 20:05:26.145002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:05:26.145002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:26.145002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:26.145002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:26.145002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:26.145002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:26.145002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:26.145002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 20:05:26.145862 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 20:05:26.152846 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 20:05:26.152846 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 20:05:26.152846 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:05:26.152846 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:05:26.152846 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 20:05:26.152846 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:26.152846 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 484/963] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 20:05:25 2022] POST /identity/v3/users => generated 160 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:26.153825 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c30c55-62ff-4548-a9b8-43e709ecabf0 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:26.251415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c30c55-62ff-4548-a9b8-43e709ecabf0 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:26.289251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662fef7-d143-48fd-9ecf-68e3aaac2f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:26.290072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662fef7-d143-48fd-9ecf-68e3aaac2f0f None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:26.290434 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662fef7-d143-48fd-9ecf-68e3aaac2f0f None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:26.290717 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662fef7-d143-48fd-9ecf-68e3aaac2f0f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:26.298546 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662fef7-d143-48fd-9ecf-68e3aaac2f0f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:26.300078 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662fef7-d143-48fd-9ecf-68e3aaac2f0f None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-5662fef7-d143-48fd-9ecf-68e3aaac2f0f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:05:26.315179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:26.317454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:26.319816 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:26.319816 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:26.319816 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:05:26.319816 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:05:26.319816 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:05:26.319816 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:05:26.319816 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:05:26.319816 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:05:26.319816 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:05:26.319816 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:26.319816 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 480/964] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:05:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:26.325592 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e5257-27dd-459a-82f9-7ed1b24efc57 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:26.427996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:26.448672 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e5257-27dd-459a-82f9-7ed1b24efc57 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:26.487866 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91802ba5-b150-4358-8932-14f0ef3ce27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:26.488551 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91802ba5-b150-4358-8932-14f0ef3ce27b None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:26.488831 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91802ba5-b150-4358-8932-14f0ef3ce27b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:26.489105 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91802ba5-b150-4358-8932-14f0ef3ce27b None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:26.489923 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91802ba5-b150-4358-8932-14f0ef3ce27b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:26.490230 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91802ba5-b150-4358-8932-14f0ef3ce27b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:26.491916 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91802ba5-b150-4358-8932-14f0ef3ce27b None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:26.531678 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 485/965] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:05:26 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:26.995611 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950f5dd-263d-4ab1-b64e-15c11461d64e None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:26.995888 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950f5dd-263d-4ab1-b64e-15c11461d64e None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:26.996133 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950f5dd-263d-4ab1-b64e-15c11461d64e None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:26.997694 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 481/966] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:27.003353 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc85f2f-86de-4ebe-a1c4-56106977587c None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:27.003621 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc85f2f-86de-4ebe-a1c4-56106977587c None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:27.003921 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc85f2f-86de-4ebe-a1c4-56106977587c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:27.021642 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-cbc85f2f-86de-4ebe-a1c4-56106977587c None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:27.023190 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-cbc85f2f-86de-4ebe-a1c4-56106977587c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:27.095916 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 486/967] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:27.118408 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950f5dd-263d-4ab1-b64e-15c11461d64e None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:27.123770 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:27.232165 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950f5dd-263d-4ab1-b64e-15c11461d64e None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:27.269341 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e82860-7ef7-4366-a89a-2d16eef32026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:27.270066 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e82860-7ef7-4366-a89a-2d16eef32026 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:27.270374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e82860-7ef7-4366-a89a-2d16eef32026 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:27.270651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e82860-7ef7-4366-a89a-2d16eef32026 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:27.271307 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e82860-7ef7-4366-a89a-2d16eef32026 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:27.272703 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e82860-7ef7-4366-a89a-2d16eef32026 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-69e82860-7ef7-4366-a89a-2d16eef32026 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:27.285209 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:27.286415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:27.287355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:27.287355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:27.287355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:27.287355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:27.287355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:27.287355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:27.287355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:27.287355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:27.287355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:27.287355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:27.287355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:27.287355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:27.287355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:05:27.287355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:27.290425 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 482/968] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:05:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:27.293101 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc85f2f-86de-4ebe-a1c4-56106977587c None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:27.369305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:27.392551 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ee5e8-d688-4efe-bdf8-360f214ea715 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:27.392809 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ee5e8-d688-4efe-bdf8-360f214ea715 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:27.393085 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ee5e8-d688-4efe-bdf8-360f214ea715 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:27.394492 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 483/969] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:05:27.400204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c3a4b-67fd-49af-8801-e327c571620f None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:27.400867 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c3a4b-67fd-49af-8801-e327c571620f None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:27.401207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c3a4b-67fd-49af-8801-e327c571620f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:27.406793 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc85f2f-86de-4ebe-a1c4-56106977587c None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:27.420711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-8d7c3a4b-67fd-49af-8801-e327c571620f None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:27.421930 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-8d7c3a4b-67fd-49af-8801-e327c571620f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:27.444392 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428e4b5-6929-4e08-8ba4-133b73efed2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:27.445074 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428e4b5-6929-4e08-8ba4-133b73efed2c None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:27.445352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428e4b5-6929-4e08-8ba4-133b73efed2c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:27.445618 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428e4b5-6929-4e08-8ba4-133b73efed2c None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:27.446360 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428e4b5-6929-4e08-8ba4-133b73efed2c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:27.447305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428e4b5-6929-4e08-8ba4-133b73efed2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:27.448441 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428e4b5-6929-4e08-8ba4-133b73efed2c None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:27.453942 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 487/970] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:05:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:27.459843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428e4b5-6929-4e08-8ba4-133b73efed2c None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:27.498606 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:27.514414 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 484/971] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:27.529902 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c3a4b-67fd-49af-8801-e327c571620f None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:27.569469 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428e4b5-6929-4e08-8ba4-133b73efed2c None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:27.606728 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cde22-e794-4241-9e1a-aa9ae528d0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:27.607404 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cde22-e794-4241-9e1a-aa9ae528d0c1 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:27.607676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cde22-e794-4241-9e1a-aa9ae528d0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:27.607947 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cde22-e794-4241-9e1a-aa9ae528d0c1 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:27.608659 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cde22-e794-4241-9e1a-aa9ae528d0c1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:27.608967 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cde22-e794-4241-9e1a-aa9ae528d0c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:27.610356 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cde22-e794-4241-9e1a-aa9ae528d0c1 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:27.611768 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:27.616619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 488/972] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:05:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:27.622626 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cde22-e794-4241-9e1a-aa9ae528d0c1 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:27.649158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c3a4b-67fd-49af-8801-e327c571620f None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:27.683753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb53239-c2dd-48b3-8ac7-b77733248ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:27.684573 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb53239-c2dd-48b3-8ac7-b77733248ab0 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:27.684848 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb53239-c2dd-48b3-8ac7-b77733248ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:27.685117 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb53239-c2dd-48b3-8ac7-b77733248ab0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:27.699009 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb53239-c2dd-48b3-8ac7-b77733248ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-afb53239-c2dd-48b3-8ac7-b77733248ab0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:05:27.707506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:27.708525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:27.709435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:27.709435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:27.709435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:27.709435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:27.709435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:05:27.709435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:05:27.709435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:05:27.709435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:05:27.709435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:05:27.709435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:05:27.709435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:05:27.709435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:27.709435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 485/973] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 20:05:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:27.715703 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb53239-c2dd-48b3-8ac7-b77733248ab0 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:27.730843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cde22-e794-4241-9e1a-aa9ae528d0c1 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:27.766046 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa144a-ee68-483b-8a27-a729bee00e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:27.766739 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa144a-ee68-483b-8a27-a729bee00e1f None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:27.767020 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa144a-ee68-483b-8a27-a729bee00e1f None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:27.767293 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa144a-ee68-483b-8a27-a729bee00e1f None admin] PATH_INFO: `/v3/endpoints` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:27.768085 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa144a-ee68-483b-8a27-a729bee00e1f None admin] RBAC: Adding query filter params (interface=admin, service_id=01a9df4a977c4d9cbd04b57a7ef86286, region_id=RegionOne) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:27.768394 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa144a-ee68-483b-8a27-a729bee00e1f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:27.769763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa144a-ee68-483b-8a27-a729bee00e1f None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:27.774888 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 489/974] 10.222.0.84 () {56 vars in 1249 bytes} [Thu May 19 20:05:27 2022] GET /identity/v3/endpoints?service_id=01a9df4a977c4d9cbd04b57a7ef86286&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:27.781055 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa144a-ee68-483b-8a27-a729bee00e1f None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:27.787403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:27.831476 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb53239-c2dd-48b3-8ac7-b77733248ab0 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:27.847970 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:27.873610 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbef02e-6fbf-4668-a409-42900beb5a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:27.874485 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbef02e-6fbf-4668-a409-42900beb5a48 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:27.874770 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbef02e-6fbf-4668-a409-42900beb5a48 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:27.875051 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbef02e-6fbf-4668-a409-42900beb5a48 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:27.875814 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbef02e-6fbf-4668-a409-42900beb5a48 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:27.876126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbef02e-6fbf-4668-a409-42900beb5a48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:27.877555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbef02e-6fbf-4668-a409-42900beb5a48 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:27.887169 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 486/975] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 20:05:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:27.892711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbef02e-6fbf-4668-a409-42900beb5a48 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:27.898173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa144a-ee68-483b-8a27-a729bee00e1f None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:27.934025 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a50f72-c9f8-46ea-9aab-843e1130ebdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:27.934814 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a50f72-c9f8-46ea-9aab-843e1130ebdd None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:27.935095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a50f72-c9f8-46ea-9aab-843e1130ebdd None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:27.935368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a50f72-c9f8-46ea-9aab-843e1130ebdd None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:27.936061 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a50f72-c9f8-46ea-9aab-843e1130ebdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:27.937467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a50f72-c9f8-46ea-9aab-843e1130ebdd None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:27.946301 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 490/976] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:05:27 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 20:05:28.009706 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbef02e-6fbf-4668-a409-42900beb5a48 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:28.033064 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:28.038336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b0583-d469-4f6d-89c2-138bf8c0a122 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:28.038646 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b0583-d469-4f6d-89c2-138bf8c0a122 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:28.038903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b0583-d469-4f6d-89c2-138bf8c0a122 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:28.044530 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 491/977] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:28.046657 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdc3d8-6d26-4a27-a5fb-70ece3011686 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:28.046939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdc3d8-6d26-4a27-a5fb-70ece3011686 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:28.047207 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdc3d8-6d26-4a27-a5fb-70ece3011686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:28.050605 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9810f2-ecb2-43df-ad2c-d05946d382bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:28.051403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9810f2-ecb2-43df-ad2c-d05946d382bb None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:28.051680 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9810f2-ecb2-43df-ad2c-d05946d382bb None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:28.051948 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9810f2-ecb2-43df-ad2c-d05946d382bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:28.064991 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-40fdc3d8-6d26-4a27-a5fb-70ece3011686 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:28.065655 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9810f2-ecb2-43df-ad2c-d05946d382bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:28.066426 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-40fdc3d8-6d26-4a27-a5fb-70ece3011686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:28.067173 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9810f2-ecb2-43df-ad2c-d05946d382bb None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:28.088755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-8e9810f2-ecb2-43df-ad2c-d05946d382bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:28.088755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:28.088755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:28.088755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:28.088755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:28.088755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:28.088755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:28.088755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:28.088755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:28.088755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:28.088755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:28.088755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:28.088755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:28.088755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:28.088755 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:28.089813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:28.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:28.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:28.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:28.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:28.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 487/978] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:28.096302 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9810f2-ecb2-43df-ad2c-d05946d382bb None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:28.150540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 492/979] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:28.173465 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdc3d8-6d26-4a27-a5fb-70ece3011686 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:28.218585 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9810f2-ecb2-43df-ad2c-d05946d382bb None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:28.234165 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:28.254806 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92dd54-95ab-4984-8ec6-1ca1c431eef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:28.255548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92dd54-95ab-4984-8ec6-1ca1c431eef6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:28.255744 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92dd54-95ab-4984-8ec6-1ca1c431eef6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:28.256017 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92dd54-95ab-4984-8ec6-1ca1c431eef6 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:28.256747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92dd54-95ab-4984-8ec6-1ca1c431eef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:28.257048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92dd54-95ab-4984-8ec6-1ca1c431eef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:28.258450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92dd54-95ab-4984-8ec6-1ca1c431eef6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:28.271456 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 488/980] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:28.277648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92dd54-95ab-4984-8ec6-1ca1c431eef6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:28.286152 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdc3d8-6d26-4a27-a5fb-70ece3011686 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:28.311312 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:28.327907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703288b-20cb-4b18-aeaa-c14a74e2ba6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:28.328752 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703288b-20cb-4b18-aeaa-c14a74e2ba6c None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:28.329421 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703288b-20cb-4b18-aeaa-c14a74e2ba6c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:28.329707 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703288b-20cb-4b18-aeaa-c14a74e2ba6c None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:28.330644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a703288b-20cb-4b18-aeaa-c14a74e2ba6c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:28.332038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a703288b-20cb-4b18-aeaa-c14a74e2ba6c None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:28.347182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 493/981] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:05:28 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:05:28.390692 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92dd54-95ab-4984-8ec6-1ca1c431eef6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:28.436856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b7f28-b284-462d-908e-d2e40629aacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:28.437701 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b7f28-b284-462d-908e-d2e40629aacf None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:28.437980 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b7f28-b284-462d-908e-d2e40629aacf None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:28.438251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b7f28-b284-462d-908e-d2e40629aacf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:28.446676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b7f28-b284-462d-908e-d2e40629aacf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:28.448395 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b7f28-b284-462d-908e-d2e40629aacf None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-791b7f28-b284-462d-908e-d2e40629aacf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:05:28.465113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:28.466238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:28.471514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:28.471514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:28.471514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:05:28.471514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:05:28.471514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:05:28.471514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:05:28.471514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:05:28.471514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:05:28.471514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:05:28.471514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:28.471514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 489/982] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:05:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:28.475342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703288b-20cb-4b18-aeaa-c14a74e2ba6c None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:28.594615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703288b-20cb-4b18-aeaa-c14a74e2ba6c None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:28.611157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:28.629166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a237f11-8e23-44da-a361-0bd8465c0ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:28.630022 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a237f11-8e23-44da-a361-0bd8465c0ec5 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:28.630395 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a237f11-8e23-44da-a361-0bd8465c0ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:28.630823 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a237f11-8e23-44da-a361-0bd8465c0ec5 None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:28.631753 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a237f11-8e23-44da-a361-0bd8465c0ec5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:28.632298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a237f11-8e23-44da-a361-0bd8465c0ec5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:28.633738 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a237f11-8e23-44da-a361-0bd8465c0ec5 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:28.652361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 494/983] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 20:05:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:28.658081 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b7f28-b284-462d-908e-d2e40629aacf None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:28.749392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:28.763051 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b7f28-b284-462d-908e-d2e40629aacf None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:28.797595 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc8940-a2b3-4803-9fdf-afe9b3c4eb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:28.798255 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc8940-a2b3-4803-9fdf-afe9b3c4eb6c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:28.798565 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc8940-a2b3-4803-9fdf-afe9b3c4eb6c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:28.798843 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc8940-a2b3-4803-9fdf-afe9b3c4eb6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:28.868731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc8940-a2b3-4803-9fdf-afe9b3c4eb6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:28.870182 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc8940-a2b3-4803-9fdf-afe9b3c4eb6c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:28.957695 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:28.974552 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 490/984] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:05:28 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:05:28.980616 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a237f11-8e23-44da-a361-0bd8465c0ec5 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:29.096046 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a237f11-8e23-44da-a361-0bd8465c0ec5 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:29.134884 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12cb90-6a2b-4b62-8a88-b5ee280bfdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:29.136098 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12cb90-6a2b-4b62-8a88-b5ee280bfdb2 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:29.136571 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12cb90-6a2b-4b62-8a88-b5ee280bfdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:29.137030 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12cb90-6a2b-4b62-8a88-b5ee280bfdb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:29.147989 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12cb90-6a2b-4b62-8a88-b5ee280bfdb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:29.149720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12cb90-6a2b-4b62-8a88-b5ee280bfdb2 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:29.168182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-1b12cb90-6a2b-4b62-8a88-b5ee280bfdb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:29.168182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:29.168182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:29.168182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:29.168182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:29.168182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:29.168182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:29.168182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:29.168182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:29.168182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:29.168182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:29.168182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:29.168182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:29.168182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:29.168182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:29.169205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:29.170171 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:29.170171 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:29.170171 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:29.170171 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:29.177077 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 495/985] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:29.178748 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc8940-a2b3-4803-9fdf-afe9b3c4eb6c None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:29.289092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc8940-a2b3-4803-9fdf-afe9b3c4eb6c None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:29.325027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad4824-e148-456f-a008-6c6348b3545b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:29.334018 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad4824-e148-456f-a008-6c6348b3545b None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:29.334443 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad4824-e148-456f-a008-6c6348b3545b None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:29.334772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad4824-e148-456f-a008-6c6348b3545b None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:29.335539 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad4824-e148-456f-a008-6c6348b3545b None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:29.335851 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad4824-e148-456f-a008-6c6348b3545b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:29.337375 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad4824-e148-456f-a008-6c6348b3545b None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:29.351323 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 491/986] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:29.359292 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12cb90-6a2b-4b62-8a88-b5ee280bfdb2 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:29.402934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:29.478816 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12cb90-6a2b-4b62-8a88-b5ee280bfdb2 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:29.516354 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd3187-8cf2-431e-9c29-da5f8a94c5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:29.517159 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd3187-8cf2-431e-9c29-da5f8a94c5d8 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:29.517434 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd3187-8cf2-431e-9c29-da5f8a94c5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:29.517702 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd3187-8cf2-431e-9c29-da5f8a94c5d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:29.524641 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd3187-8cf2-431e-9c29-da5f8a94c5d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:29.526238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd3187-8cf2-431e-9c29-da5f8a94c5d8 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:29.541818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-b0cd3187-8cf2-431e-9c29-da5f8a94c5d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:29.541818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:29.541818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:29.541818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:29.541818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:29.541818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:29.541818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:29.541818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:29.541818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:29.541818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:29.541818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:29.541818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:29.541818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:05:29.541818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:05:29.541818 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:29.542930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:29.543886 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:05:29.543886 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:05:29.543886 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:05:29.543886 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:05:29.543886 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:05:29.543886 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:05:29.543886 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:29.543886 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:29.543886 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 496/987] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:29.547390 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad4824-e148-456f-a008-6c6348b3545b None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:29.604648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:29.661592 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad4824-e148-456f-a008-6c6348b3545b None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:29.695562 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981295b-4929-495f-bb29-1c40e79f07cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:29.696217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981295b-4929-495f-bb29-1c40e79f07cb None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:29.696553 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981295b-4929-495f-bb29-1c40e79f07cb None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:29.696928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981295b-4929-495f-bb29-1c40e79f07cb None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:29.697712 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981295b-4929-495f-bb29-1c40e79f07cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:29.698015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981295b-4929-495f-bb29-1c40e79f07cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:29.699475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981295b-4929-495f-bb29-1c40e79f07cb None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:29.712694 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 492/988] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:05:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:29.718475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd3187-8cf2-431e-9c29-da5f8a94c5d8 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:29.804875 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:29.821683 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd3187-8cf2-431e-9c29-da5f8a94c5d8 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:29.855954 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9823ae-1830-4340-8b24-77ee197f821b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:29.856611 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9823ae-1830-4340-8b24-77ee197f821b None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:29.856887 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9823ae-1830-4340-8b24-77ee197f821b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:29.857157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9823ae-1830-4340-8b24-77ee197f821b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:29.857985 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9823ae-1830-4340-8b24-77ee197f821b None admin] RBAC: Adding query filter params (role.id=326ab279cac14b3fbafe4155443ed9b7, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=bdf915a5e0aa42a89e575ce1e8a65e2a) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:29.858301 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9823ae-1830-4340-8b24-77ee197f821b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:29.859742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9823ae-1830-4340-8b24-77ee197f821b None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:29.865977 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 497/989] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:05:29 2022] GET /identity/v3/role_assignments?user.id=bdf915a5e0aa42a89e575ce1e8a65e2a&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=326ab279cac14b3fbafe4155443ed9b7 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:30.267114 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0408c-63b8-453f-bbe3-494bb94cdef0 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:30.267376 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0408c-63b8-453f-bbe3-494bb94cdef0 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:30.267647 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0408c-63b8-453f-bbe3-494bb94cdef0 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:30.269016 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 493/990] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:30.275095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930dcb89-2771-4cd5-a172-b8e758613f0b None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:30.275825 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930dcb89-2771-4cd5-a172-b8e758613f0b None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:30.276113 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930dcb89-2771-4cd5-a172-b8e758613f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:30.295835 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-930dcb89-2771-4cd5-a172-b8e758613f0b None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:30.297056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-930dcb89-2771-4cd5-a172-b8e758613f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:30.374307 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 498/991] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:30.394833 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0408c-63b8-453f-bbe3-494bb94cdef0 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:30.508650 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0408c-63b8-453f-bbe3-494bb94cdef0 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:30.518745 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:30.551487 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec3da1-1d70-4085-8ea2-f381475e209a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:30.552315 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec3da1-1d70-4085-8ea2-f381475e209a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:30.552597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec3da1-1d70-4085-8ea2-f381475e209a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:30.552872 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec3da1-1d70-4085-8ea2-f381475e209a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:30.553538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec3da1-1d70-4085-8ea2-f381475e209a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:30.554962 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec3da1-1d70-4085-8ea2-f381475e209a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-bcec3da1-1d70-4085-8ea2-f381475e209a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:30.567763 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:30.568909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:30.569831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:30.569831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:30.569831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:30.569831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:30.569831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:30.569831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:30.569831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:30.569831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:30.569831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:30.569831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:30.569831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:30.569831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:30.569831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:05:30.569831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:30.574334 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 494/992] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:05:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:30.575379 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930dcb89-2771-4cd5-a172-b8e758613f0b None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:30.596270 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628c665-0abd-4ed1-ad1b-a7f69751334a None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:30.596582 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628c665-0abd-4ed1-ad1b-a7f69751334a None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:30.596829 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628c665-0abd-4ed1-ad1b-a7f69751334a None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:30.598837 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 495/993] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:30.604560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14557490-67ff-49bd-898f-887e3f725b16 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:30.604863 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14557490-67ff-49bd-898f-887e3f725b16 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:30.605118 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14557490-67ff-49bd-898f-887e3f725b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:30.623046 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-14557490-67ff-49bd-898f-887e3f725b16 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:30.624238 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-14557490-67ff-49bd-898f-887e3f725b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:30.670757 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:30.693201 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930dcb89-2771-4cd5-a172-b8e758613f0b None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:30.706062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 496/994] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:30.729496 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14557490-67ff-49bd-898f-887e3f725b16 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:30.730622 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75807d70-0093-4ce4-9707-b3e654c84cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:30.731340 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75807d70-0093-4ce4-9707-b3e654c84cfe None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:30.731838 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75807d70-0093-4ce4-9707-b3e654c84cfe None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:30.732120 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75807d70-0093-4ce4-9707-b3e654c84cfe None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:30.733735 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75807d70-0093-4ce4-9707-b3e654c84cfe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:30.734048 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75807d70-0093-4ce4-9707-b3e654c84cfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:30.735465 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75807d70-0093-4ce4-9707-b3e654c84cfe None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:30.741939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 499/995] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:05:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:30.747787 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75807d70-0093-4ce4-9707-b3e654c84cfe None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:30.835186 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14557490-67ff-49bd-898f-887e3f725b16 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:30.836904 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:30.855419 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75807d70-0093-4ce4-9707-b3e654c84cfe None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:30.874437 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52599a86-e12e-4d8b-885c-3c1d3174183f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:30.875250 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52599a86-e12e-4d8b-885c-3c1d3174183f None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:30.875535 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52599a86-e12e-4d8b-885c-3c1d3174183f None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:30.875808 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52599a86-e12e-4d8b-885c-3c1d3174183f None admin] PATH_INFO: `/v3/services/compute` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:30.876877 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52599a86-e12e-4d8b-885c-3c1d3174183f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:30.878301 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52599a86-e12e-4d8b-885c-3c1d3174183f None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:30.890643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6a95a-3da8-474d-b1ef-fd9e3239ff0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-52599a86-e12e-4d8b-885c-3c1d3174183f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:30.890996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:30.892062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:30.898452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:30.898452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:30.898452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:30.898452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:30.898452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:30.898452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:30.898452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:30.898452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:30.898452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:30.898452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:30.898452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:30.898452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:30.898452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:05:30.898452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:30.899280 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6a95a-3da8-474d-b1ef-fd9e3239ff0d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:30.899280 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6a95a-3da8-474d-b1ef-fd9e3239ff0d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:30.899280 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6a95a-3da8-474d-b1ef-fd9e3239ff0d None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:30.899478 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 497/996] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:30.899579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6a95a-3da8-474d-b1ef-fd9e3239ff0d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:30.899579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6a95a-3da8-474d-b1ef-fd9e3239ff0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:30.899579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6a95a-3da8-474d-b1ef-fd9e3239ff0d None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:30.900270 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52599a86-e12e-4d8b-885c-3c1d3174183f None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:30.908959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 500/997] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:05:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:30.909914 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6a95a-3da8-474d-b1ef-fd9e3239ff0d None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:30.921585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:31.020851 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52599a86-e12e-4d8b-885c-3c1d3174183f None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:31.024097 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6a95a-3da8-474d-b1ef-fd9e3239ff0d None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:31.055407 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7532b-2677-4fed-aafa-3d0eacdcdb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:31.056133 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7532b-2677-4fed-aafa-3d0eacdcdb47 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:31.056413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7532b-2677-4fed-aafa-3d0eacdcdb47 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:31.056687 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7532b-2677-4fed-aafa-3d0eacdcdb47 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:31.057392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7532b-2677-4fed-aafa-3d0eacdcdb47 None admin] RBAC: Adding query filter params (name=compute) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:31.057698 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7532b-2677-4fed-aafa-3d0eacdcdb47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:31.059123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7532b-2677-4fed-aafa-3d0eacdcdb47 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:31.062907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbf297-4efc-4388-b5f2-c1fb2a5ac4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:31.063582 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbf297-4efc-4388-b5f2-c1fb2a5ac4f4 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:31.063938 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbf297-4efc-4388-b5f2-c1fb2a5ac4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:31.064496 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbf297-4efc-4388-b5f2-c1fb2a5ac4f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:31.064597 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 498/998] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:05:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:31.064879 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fbf297-4efc-4388-b5f2-c1fb2a5ac4f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:31.066243 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fbf297-4efc-4388-b5f2-c1fb2a5ac4f4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:31.070058 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7532b-2677-4fed-aafa-3d0eacdcdb47 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:31.086167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 501/999] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:05:30 2022] POST /identity/v3/endpoints => generated 325 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:05:31.106823 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:31.188686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7532b-2677-4fed-aafa-3d0eacdcdb47 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:31.233148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de245d05-afa0-4ef5-9e46-2a5282cf59e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:31.233830 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de245d05-afa0-4ef5-9e46-2a5282cf59e6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:31.234109 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de245d05-afa0-4ef5-9e46-2a5282cf59e6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:31.234404 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de245d05-afa0-4ef5-9e46-2a5282cf59e6 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:31.235113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de245d05-afa0-4ef5-9e46-2a5282cf59e6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:31.235415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de245d05-afa0-4ef5-9e46-2a5282cf59e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:31.236805 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de245d05-afa0-4ef5-9e46-2a5282cf59e6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:31.242650 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 499/1000] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:05:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:31.248491 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbf297-4efc-4388-b5f2-c1fb2a5ac4f4 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:31.322697 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:31.360475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbf297-4efc-4388-b5f2-c1fb2a5ac4f4 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:31.395317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e35c5-e8fc-4552-a735-c042b2c6476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:31.395984 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e35c5-e8fc-4552-a735-c042b2c6476c None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:31.396251 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e35c5-e8fc-4552-a735-c042b2c6476c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:31.396530 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e35c5-e8fc-4552-a735-c042b2c6476c None admin] PATH_INFO: `/v3/endpoints` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:31.397330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e35c5-e8fc-4552-a735-c042b2c6476c None admin] RBAC: Adding query filter params (interface=public, service_id=9fd05610f98b4145bb675eb2f25fa70b, region_id=RegionOne) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:31.397638 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e35c5-e8fc-4552-a735-c042b2c6476c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:31.399048 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e35c5-e8fc-4552-a735-c042b2c6476c None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:31.404531 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 502/1001] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:05:31 2022] GET /identity/v3/endpoints?service_id=9fd05610f98b4145bb675eb2f25fa70b&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:31.409906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de245d05-afa0-4ef5-9e46-2a5282cf59e6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:31.504012 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:31.515764 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de245d05-afa0-4ef5-9e46-2a5282cf59e6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:31.554353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7d958-9e29-476d-978a-4d7643cc535d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:31.555087 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7d958-9e29-476d-978a-4d7643cc535d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:31.555362 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7d958-9e29-476d-978a-4d7643cc535d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:31.555604 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7d958-9e29-476d-978a-4d7643cc535d None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:31.556397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7d958-9e29-476d-978a-4d7643cc535d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:31.557767 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7d958-9e29-476d-978a-4d7643cc535d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:31.570375 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 500/1002] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:05:31 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:05:32.168295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff318f9e-e3b2-4c47-a102-1a5a437a977f None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:32.168586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff318f9e-e3b2-4c47-a102-1a5a437a977f None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:32.168897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff318f9e-e3b2-4c47-a102-1a5a437a977f None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:32.170429 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 503/1003] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:32.175696 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8549e-c487-4778-b0ac-17f48af55acb None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:32.175998 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8549e-c487-4778-b0ac-17f48af55acb None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:32.176261 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8549e-c487-4778-b0ac-17f48af55acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:32.194190 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-f0c8549e-c487-4778-b0ac-17f48af55acb None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:32.195417 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-f0c8549e-c487-4778-b0ac-17f48af55acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:32.280329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 501/1004] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:32 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:32.303355 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff318f9e-e3b2-4c47-a102-1a5a437a977f None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:32.422721 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff318f9e-e3b2-4c47-a102-1a5a437a977f None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:32.452775 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:32.458960 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c208f-9dd7-42e6-80d4-e87b6a1d6f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:32.459766 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c208f-9dd7-42e6-80d4-e87b6a1d6f8a None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:32.460076 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c208f-9dd7-42e6-80d4-e87b6a1d6f8a None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:32.460325 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c208f-9dd7-42e6-80d4-e87b6a1d6f8a None admin] PATH_INFO: `/v3/services/image` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:32.460988 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c208f-9dd7-42e6-80d4-e87b6a1d6f8a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:32.462499 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c208f-9dd7-42e6-80d4-e87b6a1d6f8a None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-d74c208f-9dd7-42e6-80d4-e87b6a1d6f8a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:32.474615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:32.475686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:32.481858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:32.481858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:32.481858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:32.481858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:32.481858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:32.481858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:32.481858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:32.481858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:32.481858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:32.481858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:32.481858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:32.481858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:32.481858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:05:32.481858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:32.481858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 504/1005] 10.222.0.84 () {56 vars in 1105 bytes} [Thu May 19 20:05:32 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:32.482843 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8549e-c487-4778-b0ac-17f48af55acb None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:32.576908 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:32.594437 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8549e-c487-4778-b0ac-17f48af55acb None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:32.630961 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92552e6-a552-4857-bb5e-9e7b633421c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:32.631673 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92552e6-a552-4857-bb5e-9e7b633421c6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:32.631955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92552e6-a552-4857-bb5e-9e7b633421c6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:32.632204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92552e6-a552-4857-bb5e-9e7b633421c6 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:32.632914 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92552e6-a552-4857-bb5e-9e7b633421c6 None admin] RBAC: Adding query filter params (name=image) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:32.633216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92552e6-a552-4857-bb5e-9e7b633421c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:32.634668 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92552e6-a552-4857-bb5e-9e7b633421c6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:32.640177 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 502/1006] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:32.645712 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c208f-9dd7-42e6-80d4-e87b6a1d6f8a None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:32.757833 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c208f-9dd7-42e6-80d4-e87b6a1d6f8a None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:32.792141 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78673068-3cc1-4040-b11b-83791ec6bad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:32.792829 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78673068-3cc1-4040-b11b-83791ec6bad1 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:32.793115 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78673068-3cc1-4040-b11b-83791ec6bad1 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:32.793399 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78673068-3cc1-4040-b11b-83791ec6bad1 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:32.794128 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78673068-3cc1-4040-b11b-83791ec6bad1 None admin] RBAC: Adding query filter params (type=image) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:32.794468 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78673068-3cc1-4040-b11b-83791ec6bad1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:32.795866 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78673068-3cc1-4040-b11b-83791ec6bad1 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:32.800964 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 505/1007] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:32 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:33.344939 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc6aed-671f-4238-b68c-a69322de9ef6 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:33.345316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc6aed-671f-4238-b68c-a69322de9ef6 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:33.345506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc6aed-671f-4238-b68c-a69322de9ef6 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:33.346952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 503/1008] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:33.352426 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a612c-a894-409a-8464-bc63c8ca7c69 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:33.352764 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a612c-a894-409a-8464-bc63c8ca7c69 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:33.353016 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a612c-a894-409a-8464-bc63c8ca7c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:33.371236 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-1f4a612c-a894-409a-8464-bc63c8ca7c69 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:33.372428 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-1f4a612c-a894-409a-8464-bc63c8ca7c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:33.431030 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:33.445091 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 506/1009] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:33 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:33.468542 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc6aed-671f-4238-b68c-a69322de9ef6 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:33.586297 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc6aed-671f-4238-b68c-a69322de9ef6 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:33.625477 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f11ee-e240-4d38-9ba0-fe54e8873c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:33.626217 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f11ee-e240-4d38-9ba0-fe54e8873c6d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:33.626529 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f11ee-e240-4d38-9ba0-fe54e8873c6d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:33.626804 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f11ee-e240-4d38-9ba0-fe54e8873c6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:33.628078 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f11ee-e240-4d38-9ba0-fe54e8873c6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:33.629568 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f11ee-e240-4d38-9ba0-fe54e8873c6d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:33.631500 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 504/1010] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:33.637526 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a612c-a894-409a-8464-bc63c8ca7c69 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:33.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a612c-a894-409a-8464-bc63c8ca7c69 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:33.778121 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506907c-0fb6-426f-8d88-398f0857172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:33.778828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506907c-0fb6-426f-8d88-398f0857172f None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:33.779106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506907c-0fb6-426f-8d88-398f0857172f None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:33.779391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506907c-0fb6-426f-8d88-398f0857172f None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:33.780078 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506907c-0fb6-426f-8d88-398f0857172f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:33.781516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506907c-0fb6-426f-8d88-398f0857172f None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:33.801860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ea670-bb38-4902-9c88-d0f703ecc5cf None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:33.802162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ea670-bb38-4902-9c88-d0f703ecc5cf None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:33.802613 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ea670-bb38-4902-9c88-d0f703ecc5cf None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:33.804138 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 505/1011] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:05:33.809627 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fdf39-473f-455b-816a-7e2296d7f16b None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:33.809759 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 507/1012] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:05:33 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:05:33.809887 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fdf39-473f-455b-816a-7e2296d7f16b None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:33.810080 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fdf39-473f-455b-816a-7e2296d7f16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:33.828488 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-077fdf39-473f-455b-816a-7e2296d7f16b None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:33.829726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-077fdf39-473f-455b-816a-7e2296d7f16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:33.842889 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:33.909607 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 506/1013] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:33 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:33.933902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506907c-0fb6-426f-8d88-398f0857172f None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:33.983544 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:34.049266 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506907c-0fb6-426f-8d88-398f0857172f None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:34.083109 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c91e0-ddc5-4a28-ad9e-c9d3d70cb9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:34.083923 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c91e0-ddc5-4a28-ad9e-c9d3d70cb9bd None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:34.084202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c91e0-ddc5-4a28-ad9e-c9d3d70cb9bd None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:34.084467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c91e0-ddc5-4a28-ad9e-c9d3d70cb9bd None admin] PATH_INFO: `/v3/services/compute` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:34.085115 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c91e0-ddc5-4a28-ad9e-c9d3d70cb9bd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:34.086578 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c91e0-ddc5-4a28-ad9e-c9d3d70cb9bd None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-1d7c91e0-ddc5-4a28-ad9e-c9d3d70cb9bd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:34.098915 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:34.100005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:34.100924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:34.100924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:34.100924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:34.100924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:34.100924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:34.100924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:34.100924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:34.100924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:34.100924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:34.100924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:34.100924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:34.100924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:34.100924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:05:34.100924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:34.100924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 508/1014] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:34.106356 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fdf39-473f-455b-816a-7e2296d7f16b None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:34.208325 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fdf39-473f-455b-816a-7e2296d7f16b None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:34.244636 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c391dd-c36e-4508-999d-20611950cf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:34.245301 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c391dd-c36e-4508-999d-20611950cf3e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:34.245577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c391dd-c36e-4508-999d-20611950cf3e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:34.245849 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c391dd-c36e-4508-999d-20611950cf3e None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:34.246580 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c391dd-c36e-4508-999d-20611950cf3e None admin] RBAC: Adding query filter params (name=compute) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:34.246886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c391dd-c36e-4508-999d-20611950cf3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:34.248262 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c391dd-c36e-4508-999d-20611950cf3e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:34.253570 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 507/1015] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:05:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:34.259235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c91e0-ddc5-4a28-ad9e-c9d3d70cb9bd None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:34.365163 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c91e0-ddc5-4a28-ad9e-c9d3d70cb9bd None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:34.374303 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:34.402700 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3e3de-b349-4cd6-a9d5-f33377e2d018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:34.403666 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3e3de-b349-4cd6-a9d5-f33377e2d018 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:34.404159 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3e3de-b349-4cd6-a9d5-f33377e2d018 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:34.404612 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3e3de-b349-4cd6-a9d5-f33377e2d018 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:34.405704 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee3e3de-b349-4cd6-a9d5-f33377e2d018 None admin] RBAC: Adding query filter params (type=compute) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:34.406193 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee3e3de-b349-4cd6-a9d5-f33377e2d018 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:34.407756 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee3e3de-b349-4cd6-a9d5-f33377e2d018 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:34.413071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 509/1016] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:05:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:34.419043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c391dd-c36e-4508-999d-20611950cf3e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:34.432158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:34.527968 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c391dd-c36e-4508-999d-20611950cf3e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:34.569643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b140a-0602-4e37-a5ea-a875d6553c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:34.570014 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b140a-0602-4e37-a5ea-a875d6553c08 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:34.570340 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b140a-0602-4e37-a5ea-a875d6553c08 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:34.570613 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b140a-0602-4e37-a5ea-a875d6553c08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:34.571361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b140a-0602-4e37-a5ea-a875d6553c08 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:34.572836 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b140a-0602-4e37-a5ea-a875d6553c08 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:34.598534 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 508/1017] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:05:34 2022] POST /identity/v3/endpoints => generated 334 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:05:35.098986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3f294-4515-4a67-8de1-fcde60198b76 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:35.099541 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3f294-4515-4a67-8de1-fcde60198b76 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:35.099623 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3f294-4515-4a67-8de1-fcde60198b76 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:35.101597 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 510/1018] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:35.108957 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a32ee-80c5-4d27-a0f7-27aab3f72adf None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:35.109311 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a32ee-80c5-4d27-a0f7-27aab3f72adf None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:35.109723 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a32ee-80c5-4d27-a0f7-27aab3f72adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:35.130752 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-ad6a32ee-80c5-4d27-a0f7-27aab3f72adf None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:35.131952 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-ad6a32ee-80c5-4d27-a0f7-27aab3f72adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:35.195268 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:35.211092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 509/1019] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:35 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:35.234055 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3f294-4515-4a67-8de1-fcde60198b76 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:35.336570 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3f294-4515-4a67-8de1-fcde60198b76 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:35.369489 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f7bcb-3b5d-4bf0-a91d-0560dba37b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:35.370081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f7bcb-3b5d-4bf0-a91d-0560dba37b26 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:35.370391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f7bcb-3b5d-4bf0-a91d-0560dba37b26 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:35.370662 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f7bcb-3b5d-4bf0-a91d-0560dba37b26 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:35.371405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f7bcb-3b5d-4bf0-a91d-0560dba37b26 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:35.372751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f7bcb-3b5d-4bf0-a91d-0560dba37b26 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:35.388990 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 511/1020] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:05:35 2022] POST /identity/v3/services => generated 242 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:05:36.089117 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bd91c-c995-48b4-8e32-e3e412533196 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:36.089681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bd91c-c995-48b4-8e32-e3e412533196 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:36.089681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bd91c-c995-48b4-8e32-e3e412533196 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:36.091044 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 510/1021] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:36.097253 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd2f8b-d1e4-49ff-bb19-bdf5f7417737 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:36.097553 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd2f8b-d1e4-49ff-bb19-bdf5f7417737 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:36.097835 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd2f8b-d1e4-49ff-bb19-bdf5f7417737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:36.115307 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-0fbd2f8b-d1e4-49ff-bb19-bdf5f7417737 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:36.116480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-0fbd2f8b-d1e4-49ff-bb19-bdf5f7417737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:36.141239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:36.193530 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 512/1022] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:36.218142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50bd91c-c995-48b4-8e32-e3e412533196 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:36.323684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50bd91c-c995-48b4-8e32-e3e412533196 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:36.359668 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1cc72-b276-4df7-b563-f9c33b3d6e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:36.360371 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1cc72-b276-4df7-b563-f9c33b3d6e25 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:36.360650 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1cc72-b276-4df7-b563-f9c33b3d6e25 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:36.360916 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1cc72-b276-4df7-b563-f9c33b3d6e25 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:36.362123 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1cc72-b276-4df7-b563-f9c33b3d6e25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:36.363616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1cc72-b276-4df7-b563-f9c33b3d6e25 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:36.365447 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 511/1023] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:36.371314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd2f8b-d1e4-49ff-bb19-bdf5f7417737 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:36.475826 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd2f8b-d1e4-49ff-bb19-bdf5f7417737 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:36.511741 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9172aed-4ba9-4a19-9c97-91b62b3a1a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:36.512417 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9172aed-4ba9-4a19-9c97-91b62b3a1a72 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:36.512702 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9172aed-4ba9-4a19-9c97-91b62b3a1a72 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:36.512963 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9172aed-4ba9-4a19-9c97-91b62b3a1a72 None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:36.513694 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9172aed-4ba9-4a19-9c97-91b62b3a1a72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:36.515156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9172aed-4ba9-4a19-9c97-91b62b3a1a72 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:36.542316 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 513/1024] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:05:36 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:05:36.930596 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3dce4-9f0b-44d9-878c-9c6d5ac62f21 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:36.930909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3dce4-9f0b-44d9-878c-9c6d5ac62f21 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:36.931324 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3dce4-9f0b-44d9-878c-9c6d5ac62f21 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:36.932747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 512/1025] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:36.938118 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b09fee-2a46-4535-b40d-354c008722ff None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:36.938409 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b09fee-2a46-4535-b40d-354c008722ff None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:36.938722 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b09fee-2a46-4535-b40d-354c008722ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:36.956108 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-b6b09fee-2a46-4535-b40d-354c008722ff None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:36.957280 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-b6b09fee-2a46-4535-b40d-354c008722ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:36.965175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:37.033690 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 514/1026] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:05:37.057074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3dce4-9f0b-44d9-878c-9c6d5ac62f21 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:37.120228 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:37.166839 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3dce4-9f0b-44d9-878c-9c6d5ac62f21 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:37.205756 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f19118-53d7-4875-8b11-3472943636cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:37.206508 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f19118-53d7-4875-8b11-3472943636cd None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:37.206785 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f19118-53d7-4875-8b11-3472943636cd None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:37.207050 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f19118-53d7-4875-8b11-3472943636cd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:37.218751 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f19118-53d7-4875-8b11-3472943636cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-f5f19118-53d7-4875-8b11-3472943636cd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:05:37.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:37.229242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:37.230575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:37.230575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:37.230575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:37.230575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:37.230575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:05:37.230575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:05:37.230575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:05:37.230575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:05:37.230575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:05:37.230575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:05:37.230575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:05:37.230575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:37.230575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 513/1027] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 20:05:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:37.234747 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b09fee-2a46-4535-b40d-354c008722ff None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:37.343547 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b09fee-2a46-4535-b40d-354c008722ff None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:37.378739 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ddf7aa-229f-4a97-8f24-0d9f19b8efd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:37.379460 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddf7aa-229f-4a97-8f24-0d9f19b8efd2 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:37.379748 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddf7aa-229f-4a97-8f24-0d9f19b8efd2 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:37.380019 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddf7aa-229f-4a97-8f24-0d9f19b8efd2 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:37.380777 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ddf7aa-229f-4a97-8f24-0d9f19b8efd2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:37.381084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ddf7aa-229f-4a97-8f24-0d9f19b8efd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:37.382479 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ddf7aa-229f-4a97-8f24-0d9f19b8efd2 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:37.391586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 515/1028] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 20:05:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:37.397318 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f19118-53d7-4875-8b11-3472943636cd None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:37.511490 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f19118-53d7-4875-8b11-3472943636cd None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:37.532466 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:37.551596 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97f329-1bfd-411b-9dda-0e319d419c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:37.552421 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97f329-1bfd-411b-9dda-0e319d419c6e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:37.552695 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97f329-1bfd-411b-9dda-0e319d419c6e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:37.553469 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97f329-1bfd-411b-9dda-0e319d419c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:37.566300 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97f329-1bfd-411b-9dda-0e319d419c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:37.566300 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97f329-1bfd-411b-9dda-0e319d419c6e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:37.585000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-9e97f329-1bfd-411b-9dda-0e319d419c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:37.585000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:37.585000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:37.585000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:37.585000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:37.585000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:37.585000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:37.585000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:37.585000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:37.585000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:37.585000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:37.585000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:37.585000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:37.585000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:37.585000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:37.585977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:37.592510 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:37.592510 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:37.592510 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:37.592510 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:37.592510 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 514/1029] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:37.595093 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ddf7aa-229f-4a97-8f24-0d9f19b8efd2 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:37.600178 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:37.640004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c3aec-db96-4aea-842c-50ed4c946c19 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:37.640376 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c3aec-db96-4aea-842c-50ed4c946c19 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:37.640736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c3aec-db96-4aea-842c-50ed4c946c19 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:37.642373 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 515/1030] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:05:37.647316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad19b59-5cd9-4de8-ab57-7d83680fe328 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:37.647604 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad19b59-5cd9-4de8-ab57-7d83680fe328 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:37.647884 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad19b59-5cd9-4de8-ab57-7d83680fe328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:37.667163 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-fad19b59-5cd9-4de8-ab57-7d83680fe328 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:37.668370 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-fad19b59-5cd9-4de8-ab57-7d83680fe328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:37.712524 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ddf7aa-229f-4a97-8f24-0d9f19b8efd2 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:37.741900 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 516/1031] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:05:37.755197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdeeec-4747-4cdc-9d74-79d1fe7f4b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:37.755880 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdeeec-4747-4cdc-9d74-79d1fe7f4b00 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:37.756166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdeeec-4747-4cdc-9d74-79d1fe7f4b00 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:37.756442 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdeeec-4747-4cdc-9d74-79d1fe7f4b00 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:37.757186 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cdeeec-4747-4cdc-9d74-79d1fe7f4b00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:37.757494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cdeeec-4747-4cdc-9d74-79d1fe7f4b00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:37.759656 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cdeeec-4747-4cdc-9d74-79d1fe7f4b00 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:37.765077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad19b59-5cd9-4de8-ab57-7d83680fe328 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:37.774504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 516/1032] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:37.778990 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdeeec-4747-4cdc-9d74-79d1fe7f4b00 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:37.848242 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:37.881984 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad19b59-5cd9-4de8-ab57-7d83680fe328 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:37.883757 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:37.888649 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdeeec-4747-4cdc-9d74-79d1fe7f4b00 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:37.922501 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ce01d-5c14-4afa-b187-f342d5796540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:37.923313 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ce01d-5c14-4afa-b187-f342d5796540 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:37.923598 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ce01d-5c14-4afa-b187-f342d5796540 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:37.923868 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ce01d-5c14-4afa-b187-f342d5796540 None admin] PATH_INFO: `/v3/services/image` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:37.924527 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ce01d-5c14-4afa-b187-f342d5796540 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:37.925386 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de298781-011f-4a93-8018-62a6fb0aff95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:37.925918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ce01d-5c14-4afa-b187-f342d5796540 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:37.926195 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de298781-011f-4a93-8018-62a6fb0aff95 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:37.926526 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de298781-011f-4a93-8018-62a6fb0aff95 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:37.926806 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de298781-011f-4a93-8018-62a6fb0aff95 None admin] PATH_INFO: `/v3/users/nova` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:37.935121 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de298781-011f-4a93-8018-62a6fb0aff95 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:37.936559 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de298781-011f-4a93-8018-62a6fb0aff95 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-e60ce01d-5c14-4afa-b187-f342d5796540 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:37.938793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:37.939886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:37.940826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:37.940826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:37.940826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:37.940826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:37.940826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:37.940826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:37.940826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:37.940826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:37.940826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:37.940826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:37.940826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:37.940826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:37.940826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:05:37.940826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:37.940826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 517/1033] 10.222.0.84 () {56 vars in 1105 bytes} [Thu May 19 20:05:37 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:37.946063 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ce01d-5c14-4afa-b187-f342d5796540 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-de298781-011f-4a93-8018-62a6fb0aff95 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:05:37.954946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:37.955959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:37.956983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:37.956983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:37.956983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:05:37.956983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:05:37.956983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:05:37.956983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:05:37.956983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:05:37.956983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:05:37.956983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:05:37.956983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:37.956983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 517/1034] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:05:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:37.961972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de298781-011f-4a93-8018-62a6fb0aff95 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:38.063341 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ce01d-5c14-4afa-b187-f342d5796540 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:38.068146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de298781-011f-4a93-8018-62a6fb0aff95 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:38.095984 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:38.102246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53346527-411c-4a68-a6f0-d87d02f65e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:38.103241 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53346527-411c-4a68-a6f0-d87d02f65e0c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:38.103691 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53346527-411c-4a68-a6f0-d87d02f65e0c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:38.103691 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53346527-411c-4a68-a6f0-d87d02f65e0c None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:38.104424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53346527-411c-4a68-a6f0-d87d02f65e0c None admin] RBAC: Adding query filter params (name=image) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:38.104737 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53346527-411c-4a68-a6f0-d87d02f65e0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:38.106268 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53346527-411c-4a68-a6f0-d87d02f65e0c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:38.108807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18cbfd-450d-440b-a466-9610a30be86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:38.109348 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18cbfd-450d-440b-a466-9610a30be86c None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:38.109624 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18cbfd-450d-440b-a466-9610a30be86c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:38.109892 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18cbfd-450d-440b-a466-9610a30be86c None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:38.110768 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18cbfd-450d-440b-a466-9610a30be86c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:38.111078 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18cbfd-450d-440b-a466-9610a30be86c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:38.112559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 518/1035] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:38.112672 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18cbfd-450d-440b-a466-9610a30be86c None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:38.118009 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53346527-411c-4a68-a6f0-d87d02f65e0c None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:38.136945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 518/1036] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:05:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:38.138716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18cbfd-450d-440b-a466-9610a30be86c None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:38.156586 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:38.235535 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53346527-411c-4a68-a6f0-d87d02f65e0c None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:38.248569 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18cbfd-450d-440b-a466-9610a30be86c None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:38.273025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77895ab0-de33-44a9-903b-d7c1e75ffca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:38.273699 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77895ab0-de33-44a9-903b-d7c1e75ffca2 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:38.273980 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77895ab0-de33-44a9-903b-d7c1e75ffca2 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:38.274254 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77895ab0-de33-44a9-903b-d7c1e75ffca2 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:38.275271 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77895ab0-de33-44a9-903b-d7c1e75ffca2 None admin] RBAC: Adding query filter params (type=image) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:38.276228 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77895ab0-de33-44a9-903b-d7c1e75ffca2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:38.277740 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77895ab0-de33-44a9-903b-d7c1e75ffca2 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:38.288857 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 519/1037] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:38.290632 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77895ab0-de33-44a9-903b-d7c1e75ffca2 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:38.296098 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad6146-4361-43df-8f06-4e978b40320f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:38.296767 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad6146-4361-43df-8f06-4e978b40320f None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:38.297044 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad6146-4361-43df-8f06-4e978b40320f None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:38.297314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad6146-4361-43df-8f06-4e978b40320f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:38.351861 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:38.373705 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad6146-4361-43df-8f06-4e978b40320f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:38.375329 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad6146-4361-43df-8f06-4e978b40320f None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:38.404195 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:38.411326 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77895ab0-de33-44a9-903b-d7c1e75ffca2 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:38.446058 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a0ee1-13d8-4cb8-bb36-f26d3b85bd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:38.446903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a0ee1-13d8-4cb8-bb36-f26d3b85bd0d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:38.447164 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a0ee1-13d8-4cb8-bb36-f26d3b85bd0d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:38.447416 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a0ee1-13d8-4cb8-bb36-f26d3b85bd0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:38.448204 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a0ee1-13d8-4cb8-bb36-f26d3b85bd0d None admin] RBAC: Adding query filter params (interface=public, service_id=572557f97db94b2988ad2bb260bd1018, region_id=RegionOne) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:38.448525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a0ee1-13d8-4cb8-bb36-f26d3b85bd0d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:38.449914 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a0ee1-13d8-4cb8-bb36-f26d3b85bd0d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:38.455677 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 520/1038] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:05:38 2022] GET /identity/v3/endpoints?service_id=572557f97db94b2988ad2bb260bd1018&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:38.461086 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a0ee1-13d8-4cb8-bb36-f26d3b85bd0d None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:38.484354 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 519/1039] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:05:38 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:05:38.490025 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad6146-4361-43df-8f06-4e978b40320f None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:38.571760 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a0ee1-13d8-4cb8-bb36-f26d3b85bd0d None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:38.584045 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:38.603587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad6146-4361-43df-8f06-4e978b40320f None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:38.607570 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a0836-61cf-4023-9090-be2bd2a51a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:38.608314 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a0836-61cf-4023-9090-be2bd2a51a21 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:38.608595 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a0836-61cf-4023-9090-be2bd2a51a21 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:38.608862 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a0836-61cf-4023-9090-be2bd2a51a21 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:38.609541 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a0836-61cf-4023-9090-be2bd2a51a21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:38.611062 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a0836-61cf-4023-9090-be2bd2a51a21 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:38.630300 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 521/1040] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:05:38 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:05:38.653425 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fb251-c8cd-401a-ac5d-99b8dc08a16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:38.654330 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fb251-c8cd-401a-ac5d-99b8dc08a16d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:38.654616 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fb251-c8cd-401a-ac5d-99b8dc08a16d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:38.654890 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fb251-c8cd-401a-ac5d-99b8dc08a16d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:38.667735 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588fb251-c8cd-401a-ac5d-99b8dc08a16d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:38.669236 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588fb251-c8cd-401a-ac5d-99b8dc08a16d None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:38.689751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-588fb251-c8cd-401a-ac5d-99b8dc08a16d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:38.689751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:38.689751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:38.689751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:38.689751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:38.689751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:38.689751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:38.689751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:38.689751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:38.689751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:38.689751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:38.689751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:38.689751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:38.689751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:38.689751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:38.691071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:38.692401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:38.692401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:38.692401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:38.692401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:38.692401 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 520/1041] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:38.697319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a0836-61cf-4023-9090-be2bd2a51a21 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:38.727222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:38.813537 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a0836-61cf-4023-9090-be2bd2a51a21 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:38.850025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1518e8-25c9-4486-8bc0-d690df385cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:38.850735 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1518e8-25c9-4486-8bc0-d690df385cf3 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:38.851017 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1518e8-25c9-4486-8bc0-d690df385cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:38.851305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1518e8-25c9-4486-8bc0-d690df385cf3 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:38.852016 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1518e8-25c9-4486-8bc0-d690df385cf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:38.852323 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1518e8-25c9-4486-8bc0-d690df385cf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:38.853841 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1518e8-25c9-4486-8bc0-d690df385cf3 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:38.856761 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b21397-9521-4a86-8a65-b4dc923e8e08 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:38.857056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b21397-9521-4a86-8a65-b4dc923e8e08 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:38.857329 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b21397-9521-4a86-8a65-b4dc923e8e08 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:38.858897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 521/1042] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:05:38.863842 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfabf9-1836-4ec4-beed-424b65447e5b None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:38.864572 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfabf9-1836-4ec4-beed-424b65447e5b None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:38.864853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfabf9-1836-4ec4-beed-424b65447e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:38.867503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 522/1043] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:38.873059 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1518e8-25c9-4486-8bc0-d690df385cf3 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:38.882172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-4ecfabf9-1836-4ec4-beed-424b65447e5b None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:38.883397 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-4ecfabf9-1836-4ec4-beed-424b65447e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:38.957164 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:38.959830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 522/1044] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:38.978947 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1518e8-25c9-4486-8bc0-d690df385cf3 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:38.983699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfabf9-1836-4ec4-beed-424b65447e5b None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:39.014754 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a4ecb-07ef-4117-a61d-acc3af3e3a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:39.015556 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a4ecb-07ef-4117-a61d-acc3af3e3a46 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:39.015836 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a4ecb-07ef-4117-a61d-acc3af3e3a46 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:39.016108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a4ecb-07ef-4117-a61d-acc3af3e3a46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:39.023188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a4ecb-07ef-4117-a61d-acc3af3e3a46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:39.024706 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a4ecb-07ef-4117-a61d-acc3af3e3a46 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:39.040621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-a29a4ecb-07ef-4117-a61d-acc3af3e3a46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:39.040621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:39.040621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:39.040621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:39.040621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:39.040621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:39.040621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:39.040621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:39.040621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:39.040621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:39.040621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:39.040621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:39.040621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:05:39.040621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:05:39.040621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:39.046688 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:39.047657 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:05:39.047657 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:05:39.047657 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:05:39.047657 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:05:39.047657 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:05:39.047657 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:05:39.047657 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:39.047657 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:39.047657 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 523/1045] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:39.047657 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a4ecb-07ef-4117-a61d-acc3af3e3a46 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:39.058898 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:39.097760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfabf9-1836-4ec4-beed-424b65447e5b None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:39.132298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0616c34-c477-4a7e-b5d9-92846cd4e604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:39.133003 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0616c34-c477-4a7e-b5d9-92846cd4e604 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:39.133281 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0616c34-c477-4a7e-b5d9-92846cd4e604 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:39.133553 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0616c34-c477-4a7e-b5d9-92846cd4e604 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:39.144566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0616c34-c477-4a7e-b5d9-92846cd4e604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-a0616c34-c477-4a7e-b5d9-92846cd4e604 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:05:39.152269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:39.153289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:39.154566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:39.154566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:39.154566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:39.154566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:39.154566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:05:39.154566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:05:39.154566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:05:39.154566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:05:39.154566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:05:39.154566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:05:39.154566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:05:39.154566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:39.154566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 523/1046] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:05:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:39.159392 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0616c34-c477-4a7e-b5d9-92846cd4e604 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:39.162517 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a4ecb-07ef-4117-a61d-acc3af3e3a46 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:39.200584 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f12d9-f48e-4c18-854c-f40968b03c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:39.201299 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f12d9-f48e-4c18-854c-f40968b03c0a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:39.201586 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f12d9-f48e-4c18-854c-f40968b03c0a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:39.201860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f12d9-f48e-4c18-854c-f40968b03c0a None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:39.202797 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f12d9-f48e-4c18-854c-f40968b03c0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:39.203111 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f12d9-f48e-4c18-854c-f40968b03c0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:39.204723 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f12d9-f48e-4c18-854c-f40968b03c0a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:39.223354 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 524/1047] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:05:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:39.224399 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f12d9-f48e-4c18-854c-f40968b03c0a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:39.228819 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:39.272885 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0616c34-c477-4a7e-b5d9-92846cd4e604 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:39.297565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:39.313319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ea7dd-b2cd-4823-af9d-c5a06a8b4d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:39.314055 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ea7dd-b2cd-4823-af9d-c5a06a8b4d11 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:39.314370 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ea7dd-b2cd-4823-af9d-c5a06a8b4d11 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:39.314694 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ea7dd-b2cd-4823-af9d-c5a06a8b4d11 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:39.315456 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ea7dd-b2cd-4823-af9d-c5a06a8b4d11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:39.315763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ea7dd-b2cd-4823-af9d-c5a06a8b4d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:39.317128 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ea7dd-b2cd-4823-af9d-c5a06a8b4d11 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:39.325826 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 524/1048] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:05:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:39.331355 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ea7dd-b2cd-4823-af9d-c5a06a8b4d11 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:39.333129 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f12d9-f48e-4c18-854c-f40968b03c0a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:39.370408 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561440b-1e05-41d2-b4cd-13d05e4a9ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:39.371106 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561440b-1e05-41d2-b4cd-13d05e4a9ce5 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:39.371389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561440b-1e05-41d2-b4cd-13d05e4a9ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:39.371664 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561440b-1e05-41d2-b4cd-13d05e4a9ce5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:39.372536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561440b-1e05-41d2-b4cd-13d05e4a9ce5 None admin] RBAC: Adding query filter params (role.id=326ab279cac14b3fbafe4155443ed9b7, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=378b046698c04b7b88194a9ffe5b045a) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:39.372850 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561440b-1e05-41d2-b4cd-13d05e4a9ce5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:39.374782 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561440b-1e05-41d2-b4cd-13d05e4a9ce5 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:39.382593 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 525/1049] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:05:39 2022] GET /identity/v3/role_assignments?user.id=378b046698c04b7b88194a9ffe5b045a&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=326ab279cac14b3fbafe4155443ed9b7 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:39.445138 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ea7dd-b2cd-4823-af9d-c5a06a8b4d11 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:39.484343 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b8a61-e8e8-4146-b110-41137b170921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:39.485259 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b8a61-e8e8-4146-b110-41137b170921 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:39.485540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b8a61-e8e8-4146-b110-41137b170921 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:39.485821 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b8a61-e8e8-4146-b110-41137b170921 None admin] PATH_INFO: `/v3/users/226fb5e8a5ea4437a1dced8fd5f0e3fd` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:39.501681 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b8a61-e8e8-4146-b110-41137b170921 None admin] RBAC: Authorizing `identity:get_user(user_id=226fb5e8a5ea4437a1dced8fd5f0e3fd)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:39.503188 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b8a61-e8e8-4146-b110-41137b170921 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:39.505162 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 525/1050] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:05:39 2022] GET /identity/v3/users/226fb5e8a5ea4437a1dced8fd5f0e3fd => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:39.510924 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561440b-1e05-41d2-b4cd-13d05e4a9ce5 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:39.591301 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:39.629416 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561440b-1e05-41d2-b4cd-13d05e4a9ce5 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:39.664465 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e73b28-2f33-49e1-8a22-67e00f0a99b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:39.665148 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e73b28-2f33-49e1-8a22-67e00f0a99b4 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:39.665421 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e73b28-2f33-49e1-8a22-67e00f0a99b4 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:39.665644 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e73b28-2f33-49e1-8a22-67e00f0a99b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:39.737754 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e73b28-2f33-49e1-8a22-67e00f0a99b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:39.739262 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e73b28-2f33-49e1-8a22-67e00f0a99b4 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:39.809003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:39.850742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 526/1051] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:05:39 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:05:39.856924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b8a61-e8e8-4146-b110-41137b170921 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:39.905550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:39.960742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b8a61-e8e8-4146-b110-41137b170921 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:39.997075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806ef0a-0c42-4610-9076-48e660e63ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:39.998033 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806ef0a-0c42-4610-9076-48e660e63ea4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:39.998398 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806ef0a-0c42-4610-9076-48e660e63ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:39.998677 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806ef0a-0c42-4610-9076-48e660e63ea4 None admin] PATH_INFO: `/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:39.999877 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8806ef0a-0c42-4610-9076-48e660e63ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=70e2a49df21f45ab94b5c0ec6af1c6ba)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:40.001314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8806ef0a-0c42-4610-9076-48e660e63ea4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:40.003261 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 526/1052] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:05:39 2022] GET /identity/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:40.009370 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e73b28-2f33-49e1-8a22-67e00f0a99b4 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:40.109163 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e73b28-2f33-49e1-8a22-67e00f0a99b4 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:40.143548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad445bd0-7661-4277-8b69-d15647e60e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:40.144207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad445bd0-7661-4277-8b69-d15647e60e2c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:40.144486 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad445bd0-7661-4277-8b69-d15647e60e2c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:40.144760 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad445bd0-7661-4277-8b69-d15647e60e2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:40.145591 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad445bd0-7661-4277-8b69-d15647e60e2c None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=70e2a49df21f45ab94b5c0ec6af1c6ba, user.id=226fb5e8a5ea4437a1dced8fd5f0e3fd) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:40.145902 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad445bd0-7661-4277-8b69-d15647e60e2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:40.147394 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad445bd0-7661-4277-8b69-d15647e60e2c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:40.153298 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 527/1053] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:05:40 2022] GET /identity/v3/role_assignments?user.id=226fb5e8a5ea4437a1dced8fd5f0e3fd&scope.project.id=70e2a49df21f45ab94b5c0ec6af1c6ba&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:40.909080 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfedfaa-db16-4299-980f-93aef021e9c6 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:40.909621 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfedfaa-db16-4299-980f-93aef021e9c6 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:40.909621 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfedfaa-db16-4299-980f-93aef021e9c6 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:40.911105 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 527/1054] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:40.917048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb9496-c5a9-4c7a-9ca5-c136b5f43586 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:40.917499 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb9496-c5a9-4c7a-9ca5-c136b5f43586 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:40.917603 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb9496-c5a9-4c7a-9ca5-c136b5f43586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:40.935883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-4acb9496-c5a9-4c7a-9ca5-c136b5f43586 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:40.937075 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-4acb9496-c5a9-4c7a-9ca5-c136b5f43586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:40.962567 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:41.014426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 528/1055] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:41.032761 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfedfaa-db16-4299-980f-93aef021e9c6 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:41.142556 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfedfaa-db16-4299-980f-93aef021e9c6 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:41.157384 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:41.181238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4a581-1e0f-4731-b064-3219ad660ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:41.182119 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4a581-1e0f-4731-b064-3219ad660ec1 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:41.182455 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4a581-1e0f-4731-b064-3219ad660ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:41.182736 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4a581-1e0f-4731-b064-3219ad660ec1 None admin] PATH_INFO: `/v3/services/image` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:41.183405 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4a581-1e0f-4731-b064-3219ad660ec1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:41.184814 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4a581-1e0f-4731-b064-3219ad660ec1 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-e2b4a581-1e0f-4731-b064-3219ad660ec1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:41.197192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:41.198415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:41.199422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:41.199422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:41.199422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:41.199422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:41.199422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:41.199422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:41.199422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:41.199422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:41.199422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:41.199422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:41.199422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:05:41.199422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:05:41.199422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:05:41.199422 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:41.202689 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 528/1056] 10.222.0.84 () {56 vars in 1105 bytes} [Thu May 19 20:05:41 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:41.204671 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb9496-c5a9-4c7a-9ca5-c136b5f43586 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:41.313475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb9496-c5a9-4c7a-9ca5-c136b5f43586 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:41.349449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed7dff-f2a2-4ac2-8811-3013ed410025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:41.350113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed7dff-f2a2-4ac2-8811-3013ed410025 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:41.350418 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed7dff-f2a2-4ac2-8811-3013ed410025 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:41.350798 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed7dff-f2a2-4ac2-8811-3013ed410025 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:41.351395 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed7dff-f2a2-4ac2-8811-3013ed410025 None admin] RBAC: Adding query filter params (name=image) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:41.351682 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed7dff-f2a2-4ac2-8811-3013ed410025 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:41.353426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed7dff-f2a2-4ac2-8811-3013ed410025 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:41.358876 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 529/1057] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:41.364472 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4a581-1e0f-4731-b064-3219ad660ec1 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:41.473171 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4a581-1e0f-4731-b064-3219ad660ec1 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:41.507191 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249aeaa4-c916-4b8a-ba80-ccc3bd7da104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:41.507868 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249aeaa4-c916-4b8a-ba80-ccc3bd7da104 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:41.508150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249aeaa4-c916-4b8a-ba80-ccc3bd7da104 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:41.508425 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249aeaa4-c916-4b8a-ba80-ccc3bd7da104 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:41.509324 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249aeaa4-c916-4b8a-ba80-ccc3bd7da104 None admin] RBAC: Adding query filter params (type=image) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:41.509721 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249aeaa4-c916-4b8a-ba80-ccc3bd7da104 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:41.511180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249aeaa4-c916-4b8a-ba80-ccc3bd7da104 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:41.516356 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 529/1058] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:41.521991 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed7dff-f2a2-4ac2-8811-3013ed410025 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:41.554190 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:41.580365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc45f9-b8bb-4e52-b984-10e652ce4247 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:41.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc45f9-b8bb-4e52-b984-10e652ce4247 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:41.580921 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc45f9-b8bb-4e52-b984-10e652ce4247 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:41.582467 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 530/1059] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:41.587205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcb137-3150-4fde-a5a2-6fd07c10b157 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:41.587494 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcb137-3150-4fde-a5a2-6fd07c10b157 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:41.587750 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcb137-3150-4fde-a5a2-6fd07c10b157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:41.606203 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-19bcb137-3150-4fde-a5a2-6fd07c10b157 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:41.607431 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-19bcb137-3150-4fde-a5a2-6fd07c10b157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:41.643056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:41.643258 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed7dff-f2a2-4ac2-8811-3013ed410025 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:41.677853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 531/1060] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:41.678063 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855f805-a53c-4dd4-b5db-55b643c0f1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:41.678705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855f805-a53c-4dd4-b5db-55b643c0f1e0 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:41.678985 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855f805-a53c-4dd4-b5db-55b643c0f1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:41.679255 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855f805-a53c-4dd4-b5db-55b643c0f1e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:41.679988 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855f805-a53c-4dd4-b5db-55b643c0f1e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:41.681566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855f805-a53c-4dd4-b5db-55b643c0f1e0 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:41.700471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcb137-3150-4fde-a5a2-6fd07c10b157 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:41.704773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 530/1061] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:05:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:05:41.817685 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcb137-3150-4fde-a5a2-6fd07c10b157 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:41.863294 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4922a04-637b-4bd0-be35-14abdcae9130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:41.864129 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4922a04-637b-4bd0-be35-14abdcae9130 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:41.864408 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4922a04-637b-4bd0-be35-14abdcae9130 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:41.864676 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4922a04-637b-4bd0-be35-14abdcae9130 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:41.876503 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4922a04-637b-4bd0-be35-14abdcae9130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-a4922a04-637b-4bd0-be35-14abdcae9130 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:05:41.884843 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:41.885903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:41.887323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:41.887323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:41.887323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:41.887323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:41.887323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:05:41.887323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:05:41.887323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:05:41.887323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:05:41.887323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:05:41.887323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:05:41.887323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:05:41.887323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:41.887323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 532/1062] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 20:05:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:41.892306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855f805-a53c-4dd4-b5db-55b643c0f1e0 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:41.978709 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:42.001789 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855f805-a53c-4dd4-b5db-55b643c0f1e0 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:42.035944 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393034f-8709-439e-bb5d-89fa19f49446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:42.036620 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393034f-8709-439e-bb5d-89fa19f49446 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:42.036903 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393034f-8709-439e-bb5d-89fa19f49446 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:42.037170 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393034f-8709-439e-bb5d-89fa19f49446 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:42.037916 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393034f-8709-439e-bb5d-89fa19f49446 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:42.038218 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393034f-8709-439e-bb5d-89fa19f49446 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:42.039622 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393034f-8709-439e-bb5d-89fa19f49446 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:42.048363 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 531/1063] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 20:05:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:42.053997 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4922a04-637b-4bd0-be35-14abdcae9130 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:42.082550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:42.156679 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4922a04-637b-4bd0-be35-14abdcae9130 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:42.189988 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44120a4f-32d1-40b5-af0c-ad2a00734bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:42.190820 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44120a4f-32d1-40b5-af0c-ad2a00734bca None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:42.191100 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44120a4f-32d1-40b5-af0c-ad2a00734bca None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:42.191372 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44120a4f-32d1-40b5-af0c-ad2a00734bca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:42.202319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44120a4f-32d1-40b5-af0c-ad2a00734bca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:42.203861 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44120a4f-32d1-40b5-af0c-ad2a00734bca None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:42.223573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-44120a4f-32d1-40b5-af0c-ad2a00734bca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:42.223573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:42.223573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:42.223573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:42.223573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:42.223573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:42.223573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:42.223573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:42.223573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:42.223573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:42.223573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:42.223573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:42.223573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:42.223573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:42.223573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:42.224674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:42.225659 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:42.225659 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:42.225659 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:42.225659 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:42.230049 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 533/1064] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:42.231214 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393034f-8709-439e-bb5d-89fa19f49446 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:42.341398 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393034f-8709-439e-bb5d-89fa19f49446 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:42.375724 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1434b-7f71-46e5-b407-5298f759ebec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:42.376390 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1434b-7f71-46e5-b407-5298f759ebec None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:42.376661 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1434b-7f71-46e5-b407-5298f759ebec None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:42.376933 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1434b-7f71-46e5-b407-5298f759ebec None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:42.377647 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1434b-7f71-46e5-b407-5298f759ebec None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:42.377948 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1434b-7f71-46e5-b407-5298f759ebec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:42.379355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1434b-7f71-46e5-b407-5298f759ebec None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:42.391449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 532/1065] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:42.396934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44120a4f-32d1-40b5-af0c-ad2a00734bca None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:42.406298 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37d4bc-7806-48b4-9f66-5f174c569798 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:42.406298 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37d4bc-7806-48b4-9f66-5f174c569798 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:42.406298 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37d4bc-7806-48b4-9f66-5f174c569798 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:42.407394 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 533/1066] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:42.412729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d0809-b8f2-4359-acd4-05590479e79f None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:42.413019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d0809-b8f2-4359-acd4-05590479e79f None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:42.413293 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d0809-b8f2-4359-acd4-05590479e79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:42.418710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:42.432315 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-b95d0809-b8f2-4359-acd4-05590479e79f None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:42.433550 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-b95d0809-b8f2-4359-acd4-05590479e79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:42.481973 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:42.508100 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44120a4f-32d1-40b5-af0c-ad2a00734bca None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:42.511188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 534/1067] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:42.530191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d0809-b8f2-4359-acd4-05590479e79f None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:42.547802 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f827ee-4d06-4475-97fe-27b3e53bd3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:42.548724 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f827ee-4d06-4475-97fe-27b3e53bd3ed None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:42.549199 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f827ee-4d06-4475-97fe-27b3e53bd3ed None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:42.549388 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f827ee-4d06-4475-97fe-27b3e53bd3ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:42.557442 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f827ee-4d06-4475-97fe-27b3e53bd3ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:42.559163 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f827ee-4d06-4475-97fe-27b3e53bd3ed None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-44f827ee-4d06-4475-97fe-27b3e53bd3ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:05:42.574550 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:42.575608 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:42.576650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:42.576650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:42.576650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:05:42.576650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:05:42.576650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:05:42.576650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:05:42.576650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:05:42.576650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:05:42.576650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:05:42.576650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:42.580742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 534/1068] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:05:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:42.581443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f827ee-4d06-4475-97fe-27b3e53bd3ed None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:42.644860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d0809-b8f2-4359-acd4-05590479e79f None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:42.681029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:42.686565 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1525e64-5fb0-4043-8f56-466b79b85ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:42.687262 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f827ee-4d06-4475-97fe-27b3e53bd3ed None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:42.687398 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1525e64-5fb0-4043-8f56-466b79b85ebb None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:42.687640 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1525e64-5fb0-4043-8f56-466b79b85ebb None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:42.687911 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1525e64-5fb0-4043-8f56-466b79b85ebb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:42.698387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1525e64-5fb0-4043-8f56-466b79b85ebb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-a1525e64-5fb0-4043-8f56-466b79b85ebb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:05:42.706048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:42.707247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:42.708189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:42.708189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:42.708189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:42.708189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:42.708189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:05:42.708189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:05:42.708189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:05:42.708189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:05:42.708189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:05:42.708189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:05:42.708189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:05:42.708189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:42.712285 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 535/1069] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:05:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:42.713173 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1525e64-5fb0-4043-8f56-466b79b85ebb None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:42.725383 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818b91b-fdda-4212-91bb-0db771f43cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:42.726480 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818b91b-fdda-4212-91bb-0db771f43cdb None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:42.726916 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818b91b-fdda-4212-91bb-0db771f43cdb None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:42.727195 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818b91b-fdda-4212-91bb-0db771f43cdb None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:42.728340 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818b91b-fdda-4212-91bb-0db771f43cdb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:42.728658 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818b91b-fdda-4212-91bb-0db771f43cdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:42.730100 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818b91b-fdda-4212-91bb-0db771f43cdb None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:42.738972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:42.750150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 535/1070] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:05:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:42.755929 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818b91b-fdda-4212-91bb-0db771f43cdb None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:42.820223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1525e64-5fb0-4043-8f56-466b79b85ebb None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:42.859907 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51039df3-2766-47ab-90a8-2fdc65a44d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:42.860865 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51039df3-2766-47ab-90a8-2fdc65a44d59 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:42.861146 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51039df3-2766-47ab-90a8-2fdc65a44d59 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:42.861423 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51039df3-2766-47ab-90a8-2fdc65a44d59 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:42.862166 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51039df3-2766-47ab-90a8-2fdc65a44d59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:42.862501 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51039df3-2766-47ab-90a8-2fdc65a44d59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:42.863880 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51039df3-2766-47ab-90a8-2fdc65a44d59 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:42.863988 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818b91b-fdda-4212-91bb-0db771f43cdb None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:42.877016 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 536/1071] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:05:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:42.878519 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51039df3-2766-47ab-90a8-2fdc65a44d59 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:42.904192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5ef76-e535-449e-8585-b2305b296f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:42.905561 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5ef76-e535-449e-8585-b2305b296f2f None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:42.906093 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5ef76-e535-449e-8585-b2305b296f2f None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:42.906663 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5ef76-e535-449e-8585-b2305b296f2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:42.919125 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5ef76-e535-449e-8585-b2305b296f2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:42.920844 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5ef76-e535-449e-8585-b2305b296f2f None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:42.938863 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:42.942335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-e4a5ef76-e535-449e-8585-b2305b296f2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:42.942335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:42.942335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:42.942335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:42.942335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:42.942335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:42.942335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:42.942335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:42.942335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:42.942335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:42.942335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:42.942335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:42.942335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:42.942335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:42.942335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:42.943537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:42.944742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:42.944742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:42.944742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:42.944742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:42.949776 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 536/1072] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:42.950666 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5ef76-e535-449e-8585-b2305b296f2f None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:42.990053 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51039df3-2766-47ab-90a8-2fdc65a44d59 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:43.002565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:43.028211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228831a1-a980-4965-9212-b013f0ad8dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:43.029130 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228831a1-a980-4965-9212-b013f0ad8dcc None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:43.029407 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228831a1-a980-4965-9212-b013f0ad8dcc None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:43.029674 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228831a1-a980-4965-9212-b013f0ad8dcc None admin] PATH_INFO: `/v3/users/226fb5e8a5ea4437a1dced8fd5f0e3fd` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:43.030916 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228831a1-a980-4965-9212-b013f0ad8dcc None admin] RBAC: Authorizing `identity:get_user(user_id=226fb5e8a5ea4437a1dced8fd5f0e3fd)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:43.032369 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228831a1-a980-4965-9212-b013f0ad8dcc None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:43.034003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 537/1073] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:05:42 2022] GET /identity/v3/users/226fb5e8a5ea4437a1dced8fd5f0e3fd => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:43.039468 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228831a1-a980-4965-9212-b013f0ad8dcc None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:43.065068 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5ef76-e535-449e-8585-b2305b296f2f None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:43.105846 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5a73e-6f78-4b89-a530-6b3da4b067c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:43.106924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5a73e-6f78-4b89-a530-6b3da4b067c0 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:43.107443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5a73e-6f78-4b89-a530-6b3da4b067c0 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:43.107931 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5a73e-6f78-4b89-a530-6b3da4b067c0 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:43.108877 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5a73e-6f78-4b89-a530-6b3da4b067c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:43.109395 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5a73e-6f78-4b89-a530-6b3da4b067c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:43.110991 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5a73e-6f78-4b89-a530-6b3da4b067c0 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:43.122905 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 537/1074] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:43.128109 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5a73e-6f78-4b89-a530-6b3da4b067c0 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:43.156452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228831a1-a980-4965-9212-b013f0ad8dcc None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:43.182397 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:43.194293 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d11e5f-0556-4e21-a48a-e220b747a2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:43.195174 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d11e5f-0556-4e21-a48a-e220b747a2bf None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:43.195453 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d11e5f-0556-4e21-a48a-e220b747a2bf None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:43.195754 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d11e5f-0556-4e21-a48a-e220b747a2bf None admin] PATH_INFO: `/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:43.197000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d11e5f-0556-4e21-a48a-e220b747a2bf None admin] RBAC: Authorizing `identity:get_project(project_id=70e2a49df21f45ab94b5c0ec6af1c6ba)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:43.198480 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d11e5f-0556-4e21-a48a-e220b747a2bf None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:43.200348 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 538/1075] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:05:43 2022] GET /identity/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:43.206641 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d11e5f-0556-4e21-a48a-e220b747a2bf None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:43.241541 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5a73e-6f78-4b89-a530-6b3da4b067c0 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:43.251024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:43.280315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ba931-44c7-4db9-afe4-df43a1006d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:43.281973 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ba931-44c7-4db9-afe4-df43a1006d98 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:43.282274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ba931-44c7-4db9-afe4-df43a1006d98 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:43.282576 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ba931-44c7-4db9-afe4-df43a1006d98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:43.291433 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ba931-44c7-4db9-afe4-df43a1006d98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:43.293664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ba931-44c7-4db9-afe4-df43a1006d98 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:43.311555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d11e5f-0556-4e21-a48a-e220b747a2bf None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:43.313443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-492ba931-44c7-4db9-afe4-df43a1006d98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:43.313443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:43.313443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:43.313443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:43.313443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:43.313443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:43.313443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:43.313443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:43.313443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:43.313443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:43.313443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:43.313443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:43.313443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:05:43.313443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:05:43.313443 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:43.314537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:43.315538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:05:43.315538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:05:43.315538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:05:43.315538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:05:43.315538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:05:43.315538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:05:43.315538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:43.315538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:43.315538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 538/1076] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:43.320999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ba931-44c7-4db9-afe4-df43a1006d98 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:43.351965 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8b517-20e4-4fd9-9d57-02620a0d71a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:43.352973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8b517-20e4-4fd9-9d57-02620a0d71a0 None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:43.353331 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8b517-20e4-4fd9-9d57-02620a0d71a0 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:43.353611 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8b517-20e4-4fd9-9d57-02620a0d71a0 None admin] PATH_INFO: `/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba/users/226fb5e8a5ea4437a1dced8fd5f0e3fd/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:43.355829 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb8b517-20e4-4fd9-9d57-02620a0d71a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=70e2a49df21f45ab94b5c0ec6af1c6ba, user_id=226fb5e8a5ea4437a1dced8fd5f0e3fd, role_id=b34e4ad4fe7b4a60bd2fc669e2b31a66)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:43.357043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb8b517-20e4-4fd9-9d57-02620a0d71a0 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:43.372114 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 539/1077] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:05:43 2022] PUT /identity/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba/users/226fb5e8a5ea4437a1dced8fd5f0e3fd/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:05:43.438736 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ba931-44c7-4db9-afe4-df43a1006d98 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:43.478378 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a25dc-a86a-4080-b3a0-a5a688010aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:43.479081 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a25dc-a86a-4080-b3a0-a5a688010aa7 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:43.479361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a25dc-a86a-4080-b3a0-a5a688010aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:43.479636 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a25dc-a86a-4080-b3a0-a5a688010aa7 None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:43.480447 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a25dc-a86a-4080-b3a0-a5a688010aa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:43.480761 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a25dc-a86a-4080-b3a0-a5a688010aa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:43.482210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a25dc-a86a-4080-b3a0-a5a688010aa7 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:43.500021 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 539/1078] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:05:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:43.501601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8b517-20e4-4fd9-9d57-02620a0d71a0 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:43.572548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:43.625321 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8b517-20e4-4fd9-9d57-02620a0d71a0 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:43.661050 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fe933-440b-4477-a70c-549d561ea3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:43.661951 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fe933-440b-4477-a70c-549d561ea3bd None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:43.662230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fe933-440b-4477-a70c-549d561ea3bd None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:43.662530 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fe933-440b-4477-a70c-549d561ea3bd None admin] PATH_INFO: `/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/378b046698c04b7b88194a9ffe5b045a/roles/326ab279cac14b3fbafe4155443ed9b7` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:43.664794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fe933-440b-4477-a70c-549d561ea3bd None admin] RBAC: Authorizing `identity:create_grant(project_id=ba22fdc274b74521847e5fceb2d307b1, user_id=378b046698c04b7b88194a9ffe5b045a, role_id=326ab279cac14b3fbafe4155443ed9b7)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:43.666346 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fe933-440b-4477-a70c-549d561ea3bd None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:43.680453 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 540/1079] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:05:43 2022] PUT /identity/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/378b046698c04b7b88194a9ffe5b045a/roles/326ab279cac14b3fbafe4155443ed9b7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:05:44.019054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455119c8-8959-43e6-8c34-1581a6227ad4 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:44.019315 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455119c8-8959-43e6-8c34-1581a6227ad4 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:44.019589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455119c8-8959-43e6-8c34-1581a6227ad4 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:44.022198 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 540/1080] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:44.026843 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6e9da-f47c-4170-bf7c-c928a13540ae None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:44.027345 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6e9da-f47c-4170-bf7c-c928a13540ae None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:44.027345 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6e9da-f47c-4170-bf7c-c928a13540ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:44.045042 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-48b6e9da-f47c-4170-bf7c-c928a13540ae None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:44.046231 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-48b6e9da-f47c-4170-bf7c-c928a13540ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:44.122307 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 541/1081] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:44.141012 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455119c8-8959-43e6-8c34-1581a6227ad4 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:44.170972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:44.253509 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455119c8-8959-43e6-8c34-1581a6227ad4 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:44.289419 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613dbef4-177b-4722-a250-ae0797c1ec9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:44.290211 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dbef4-177b-4722-a250-ae0797c1ec9f None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:44.290600 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dbef4-177b-4722-a250-ae0797c1ec9f None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:44.290797 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dbef4-177b-4722-a250-ae0797c1ec9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:44.362303 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613dbef4-177b-4722-a250-ae0797c1ec9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:44.363777 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613dbef4-177b-4722-a250-ae0797c1ec9f None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:44.391363 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:44.470741 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 541/1082] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:05:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:05:44.472003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6e9da-f47c-4170-bf7c-c928a13540ae None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:44.517592 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:44.593794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6e9da-f47c-4170-bf7c-c928a13540ae None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:44.631808 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aad8cb-9033-40fd-9810-012ee17d2166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:44.632608 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aad8cb-9033-40fd-9810-012ee17d2166 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:44.632886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aad8cb-9033-40fd-9810-012ee17d2166 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:44.633156 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aad8cb-9033-40fd-9810-012ee17d2166 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:44.643500 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aad8cb-9033-40fd-9810-012ee17d2166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:44.644963 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aad8cb-9033-40fd-9810-012ee17d2166 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:44.662705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-e8aad8cb-9033-40fd-9810-012ee17d2166 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:44.662705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:44.662705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:44.662705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:44.662705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:44.662705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:44.662705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:44.662705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:44.662705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:44.662705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:44.662705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:44.662705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:44.662705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:44.662705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:44.662705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:44.663710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:44.665361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:44.665361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:44.665361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:44.665361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:44.665361 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 542/1083] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:44.670051 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613dbef4-177b-4722-a250-ae0797c1ec9f None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:44.768424 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613dbef4-177b-4722-a250-ae0797c1ec9f None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:44.794936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:44.804266 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb38e9-3c7b-4ed1-ab05-6d55409f5580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:44.804943 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb38e9-3c7b-4ed1-ab05-6d55409f5580 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:44.805225 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb38e9-3c7b-4ed1-ab05-6d55409f5580 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:44.805625 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb38e9-3c7b-4ed1-ab05-6d55409f5580 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:44.806293 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb38e9-3c7b-4ed1-ab05-6d55409f5580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:44.806598 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb38e9-3c7b-4ed1-ab05-6d55409f5580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:44.807962 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb38e9-3c7b-4ed1-ab05-6d55409f5580 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:44.819694 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 542/1084] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:45.616496 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ece30f-7e3b-4507-b900-76262f9ed9e4 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:45.617058 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ece30f-7e3b-4507-b900-76262f9ed9e4 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:45.617058 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ece30f-7e3b-4507-b900-76262f9ed9e4 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:45.618488 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 543/1085] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:45.623945 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b144162-b2e9-43a9-a30d-c58c1ce3ca6c None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:45.624439 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b144162-b2e9-43a9-a30d-c58c1ce3ca6c None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:45.624541 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b144162-b2e9-43a9-a30d-c58c1ce3ca6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:45.641757 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-0b144162-b2e9-43a9-a30d-c58c1ce3ca6c None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:45.642959 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-0b144162-b2e9-43a9-a30d-c58c1ce3ca6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:45.713056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 543/1086] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:45.736732 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ece30f-7e3b-4507-b900-76262f9ed9e4 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:45.838967 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:45.850736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ece30f-7e3b-4507-b900-76262f9ed9e4 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:45.886726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac00688-3ad7-41b6-912a-293b2fb8a3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:45.887525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac00688-3ad7-41b6-912a-293b2fb8a3a2 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:45.887805 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac00688-3ad7-41b6-912a-293b2fb8a3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:45.888077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac00688-3ad7-41b6-912a-293b2fb8a3a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:45.898898 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac00688-3ad7-41b6-912a-293b2fb8a3a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-eac00688-3ad7-41b6-912a-293b2fb8a3a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:05:45.906727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:45.907810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:45.908736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:45.908736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:45.908736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:45.908736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:45.908736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:05:45.908736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:05:45.908736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:05:45.908736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:05:45.908736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:05:45.908736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:05:45.908736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:05:45.908736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:45.908736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 544/1087] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:05:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:45.914140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b144162-b2e9-43a9-a30d-c58c1ce3ca6c None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:45.978986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa2891-72ec-4e31-a8a2-4bbf22935beb None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:45.979237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa2891-72ec-4e31-a8a2-4bbf22935beb None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:45.979537 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa2891-72ec-4e31-a8a2-4bbf22935beb None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:45.980917 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 545/1088] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:45.986080 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2774b3-7566-4ac7-ba93-a665b80c4b30 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:45.986401 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2774b3-7566-4ac7-ba93-a665b80c4b30 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:45.986680 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2774b3-7566-4ac7-ba93-a665b80c4b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:46.005690 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-ed2774b3-7566-4ac7-ba93-a665b80c4b30 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:46.006918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-ed2774b3-7566-4ac7-ba93-a665b80c4b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:46.016085 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b144162-b2e9-43a9-a30d-c58c1ce3ca6c None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:46.034526 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:46.052345 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e67d48-9991-4aae-9d05-3af17f9a2497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:46.053018 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e67d48-9991-4aae-9d05-3af17f9a2497 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:46.053301 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e67d48-9991-4aae-9d05-3af17f9a2497 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:46.053572 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e67d48-9991-4aae-9d05-3af17f9a2497 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:46.054377 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e67d48-9991-4aae-9d05-3af17f9a2497 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:46.054687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e67d48-9991-4aae-9d05-3af17f9a2497 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:46.056281 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e67d48-9991-4aae-9d05-3af17f9a2497 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:46.064782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 544/1089] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:05:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:46.070613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e67d48-9991-4aae-9d05-3af17f9a2497 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:46.077492 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 546/1090] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:46.101296 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2774b3-7566-4ac7-ba93-a665b80c4b30 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:46.174364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e67d48-9991-4aae-9d05-3af17f9a2497 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:46.186064 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:46.204524 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2774b3-7566-4ac7-ba93-a665b80c4b30 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:46.207709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7ab0f-ef2e-40a4-938b-f9c561e08dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:46.208589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7ab0f-ef2e-40a4-938b-f9c561e08dc0 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:46.208865 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7ab0f-ef2e-40a4-938b-f9c561e08dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:46.209134 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7ab0f-ef2e-40a4-938b-f9c561e08dc0 None admin] PATH_INFO: `/v3/users/226fb5e8a5ea4437a1dced8fd5f0e3fd` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:46.210352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7ab0f-ef2e-40a4-938b-f9c561e08dc0 None admin] RBAC: Authorizing `identity:get_user(user_id=226fb5e8a5ea4437a1dced8fd5f0e3fd)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:46.211848 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7ab0f-ef2e-40a4-938b-f9c561e08dc0 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:46.213485 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 545/1091] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:05:46 2022] GET /identity/v3/users/226fb5e8a5ea4437a1dced8fd5f0e3fd => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:46.219595 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7ab0f-ef2e-40a4-938b-f9c561e08dc0 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:46.243034 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53087b-bfbf-4fab-ba98-415739ebd36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:46.244018 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53087b-bfbf-4fab-ba98-415739ebd36a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:46.244294 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53087b-bfbf-4fab-ba98-415739ebd36a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:46.244563 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53087b-bfbf-4fab-ba98-415739ebd36a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:46.254815 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53087b-bfbf-4fab-ba98-415739ebd36a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-ed53087b-bfbf-4fab-ba98-415739ebd36a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:05:46.262870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:46.263928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:46.264855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:46.264855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:46.264855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:46.264855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:46.264855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:05:46.264855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:05:46.264855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:05:46.264855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:05:46.264855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:05:46.264855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:05:46.264855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:05:46.264855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:46.264855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 547/1092] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 20:05:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:46.269752 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:46.270023 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53087b-bfbf-4fab-ba98-415739ebd36a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:46.323056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7ab0f-ef2e-40a4-938b-f9c561e08dc0 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:46.357430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347306f6-6c4a-4249-856d-d3ac66c8d4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:46.358124 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347306f6-6c4a-4249-856d-d3ac66c8d4d4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:46.358496 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347306f6-6c4a-4249-856d-d3ac66c8d4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:46.358855 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347306f6-6c4a-4249-856d-d3ac66c8d4d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:46.383126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53087b-bfbf-4fab-ba98-415739ebd36a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:46.421003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661ffb0-6837-40d2-9a55-e743c10a24f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:46.421968 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661ffb0-6837-40d2-9a55-e743c10a24f5 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:46.422246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661ffb0-6837-40d2-9a55-e743c10a24f5 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:46.422550 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661ffb0-6837-40d2-9a55-e743c10a24f5 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:46.423305 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661ffb0-6837-40d2-9a55-e743c10a24f5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:46.423618 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661ffb0-6837-40d2-9a55-e743c10a24f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:46.425025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661ffb0-6837-40d2-9a55-e743c10a24f5 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:46.428700 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347306f6-6c4a-4249-856d-d3ac66c8d4d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:46.430160 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347306f6-6c4a-4249-856d-d3ac66c8d4d4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:46.433541 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 548/1093] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 20:05:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:46.440052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661ffb0-6837-40d2-9a55-e743c10a24f5 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:46.445441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:46.483829 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:46.537012 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 546/1094] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:05:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:05:46.543314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347306f6-6c4a-4249-856d-d3ac66c8d4d4 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:46.548768 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661ffb0-6837-40d2-9a55-e743c10a24f5 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:46.586338 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fcc8f0-663e-4010-b217-2e5f07440bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:46.587172 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcc8f0-663e-4010-b217-2e5f07440bf3 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:46.587592 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcc8f0-663e-4010-b217-2e5f07440bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:46.587843 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcc8f0-663e-4010-b217-2e5f07440bf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:46.599292 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fcc8f0-663e-4010-b217-2e5f07440bf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:46.600758 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fcc8f0-663e-4010-b217-2e5f07440bf3 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:46.617969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-b6fcc8f0-663e-4010-b217-2e5f07440bf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:46.617969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:46.617969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:46.617969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:46.617969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:46.617969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:46.617969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:46.617969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:46.617969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:46.617969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:46.617969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:46.617969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:46.617969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:46.617969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:46.617969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:46.619452 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:46.620421 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:46.620421 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:46.620421 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:46.620421 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:46.620421 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 549/1095] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:46.625073 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fcc8f0-663e-4010-b217-2e5f07440bf3 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:46.647284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347306f6-6c4a-4249-856d-d3ac66c8d4d4 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:46.683106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e4fb9-7231-4113-bef2-6c2284951c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:46.683975 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e4fb9-7231-4113-bef2-6c2284951c31 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:46.684255 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e4fb9-7231-4113-bef2-6c2284951c31 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:46.684529 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e4fb9-7231-4113-bef2-6c2284951c31 None admin] PATH_INFO: `/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:46.685719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e4fb9-7231-4113-bef2-6c2284951c31 None admin] RBAC: Authorizing `identity:get_project(project_id=70e2a49df21f45ab94b5c0ec6af1c6ba)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:46.687195 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e4fb9-7231-4113-bef2-6c2284951c31 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:46.688905 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 547/1096] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:05:46 2022] GET /identity/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:46.689750 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:46.695107 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e4fb9-7231-4113-bef2-6c2284951c31 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:46.713453 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:46.735916 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fcc8f0-663e-4010-b217-2e5f07440bf3 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:46.769599 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7fce1-a0fc-4452-9e73-6beb46ace04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:46.770283 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7fce1-a0fc-4452-9e73-6beb46ace04b None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:46.770576 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7fce1-a0fc-4452-9e73-6beb46ace04b None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:46.770850 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7fce1-a0fc-4452-9e73-6beb46ace04b None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:46.771576 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7fce1-a0fc-4452-9e73-6beb46ace04b None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:46.771881 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7fce1-a0fc-4452-9e73-6beb46ace04b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:46.773277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7fce1-a0fc-4452-9e73-6beb46ace04b None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:46.784863 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 550/1097] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:05:46.790490 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7fce1-a0fc-4452-9e73-6beb46ace04b None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:46.797172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e4fb9-7231-4113-bef2-6c2284951c31 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:46.834684 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7381c3-52b4-45da-95f8-cc2ba409c18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:46.835421 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7381c3-52b4-45da-95f8-cc2ba409c18f None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:46.835700 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7381c3-52b4-45da-95f8-cc2ba409c18f None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:46.836085 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7381c3-52b4-45da-95f8-cc2ba409c18f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:46.836980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7381c3-52b4-45da-95f8-cc2ba409c18f None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=70e2a49df21f45ab94b5c0ec6af1c6ba, user.id=226fb5e8a5ea4437a1dced8fd5f0e3fd) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:46.837333 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7381c3-52b4-45da-95f8-cc2ba409c18f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:46.838789 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7381c3-52b4-45da-95f8-cc2ba409c18f None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:46.845234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 548/1098] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:05:46 2022] GET /identity/v3/role_assignments?user.id=226fb5e8a5ea4437a1dced8fd5f0e3fd&scope.project.id=70e2a49df21f45ab94b5c0ec6af1c6ba&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:46.896104 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7fce1-a0fc-4452-9e73-6beb46ace04b None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:46.917936 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:46.939673 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbafbb-6334-49cc-b065-945066f7805a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:46.940498 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbafbb-6334-49cc-b065-945066f7805a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:46.940778 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbafbb-6334-49cc-b065-945066f7805a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:46.941057 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbafbb-6334-49cc-b065-945066f7805a None admin] PATH_INFO: `/v3/users/nova` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:46.950323 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbafbb-6334-49cc-b065-945066f7805a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:46.951865 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbafbb-6334-49cc-b065-945066f7805a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-8bfbafbb-6334-49cc-b065-945066f7805a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:05:46.969464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:46.970676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:46.971750 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:46.971750 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:46.971750 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:05:46.971750 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:05:46.971750 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:05:46.971750 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:05:46.971750 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:05:46.971750 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:05:46.971750 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:05:46.971750 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:05:46.971750 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 551/1099] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:05:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:05:46.977847 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7381c3-52b4-45da-95f8-cc2ba409c18f None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:47.068533 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:47.089030 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7381c3-52b4-45da-95f8-cc2ba409c18f None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:47.127985 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69349829-7a16-4d56-985c-6acdda8331f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:47.128664 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69349829-7a16-4d56-985c-6acdda8331f7 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:47.128940 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69349829-7a16-4d56-985c-6acdda8331f7 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:47.129212 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69349829-7a16-4d56-985c-6acdda8331f7 None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:47.130052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69349829-7a16-4d56-985c-6acdda8331f7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:47.130388 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69349829-7a16-4d56-985c-6acdda8331f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:47.131838 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69349829-7a16-4d56-985c-6acdda8331f7 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:47.149246 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 549/1100] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:05:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:47.155189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbafbb-6334-49cc-b065-945066f7805a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:47.257975 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbafbb-6334-49cc-b065-945066f7805a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:47.294585 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd08ef1-b04d-4536-b311-3ab607c1d555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:47.295239 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd08ef1-b04d-4536-b311-3ab607c1d555 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:47.295521 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd08ef1-b04d-4536-b311-3ab607c1d555 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:47.295794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd08ef1-b04d-4536-b311-3ab607c1d555 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:47.342569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:47.369600 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd08ef1-b04d-4536-b311-3ab607c1d555 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:47.371143 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd08ef1-b04d-4536-b311-3ab607c1d555 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:47.475759 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 552/1101] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:05:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:05:47.481960 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69349829-7a16-4d56-985c-6acdda8331f7 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:47.589357 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69349829-7a16-4d56-985c-6acdda8331f7 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:47.625385 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886151b-034e-411b-bb37-9e329cbc25e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:47.626210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886151b-034e-411b-bb37-9e329cbc25e3 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:47.626542 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886151b-034e-411b-bb37-9e329cbc25e3 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:47.626817 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886151b-034e-411b-bb37-9e329cbc25e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:47.634594 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:47.639065 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886151b-034e-411b-bb37-9e329cbc25e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:47.640971 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886151b-034e-411b-bb37-9e329cbc25e3 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:47.659283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-5886151b-034e-411b-bb37-9e329cbc25e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:47.659283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:47.659283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:47.659283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:47.659283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:47.659283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:47.659283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:47.659283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:47.659283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:47.659283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:47.659283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:47.659283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:47.659283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:05:47.659283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:05:47.659283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:47.660727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:47.661709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:05:47.661709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:05:47.661709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:05:47.661709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:47.661709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 550/1102] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:47.666863 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd08ef1-b04d-4536-b311-3ab607c1d555 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:47.745557 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:47.769271 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd08ef1-b04d-4536-b311-3ab607c1d555 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:47.802436 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2c94d-ca1a-48aa-94fb-092a0b5cb92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:47.803098 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2c94d-ca1a-48aa-94fb-092a0b5cb92a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:47.803377 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2c94d-ca1a-48aa-94fb-092a0b5cb92a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:47.803660 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2c94d-ca1a-48aa-94fb-092a0b5cb92a None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:47.804378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2c94d-ca1a-48aa-94fb-092a0b5cb92a None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:47.804680 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2c94d-ca1a-48aa-94fb-092a0b5cb92a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:47.806091 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2c94d-ca1a-48aa-94fb-092a0b5cb92a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:47.817291 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 553/1103] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:47.823172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886151b-034e-411b-bb37-9e329cbc25e3 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:47.925596 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886151b-034e-411b-bb37-9e329cbc25e3 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:47.960892 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef9fc2-1b7e-40cb-91fd-1e72d4e6533e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:47.961717 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef9fc2-1b7e-40cb-91fd-1e72d4e6533e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:47.962003 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef9fc2-1b7e-40cb-91fd-1e72d4e6533e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:47.962483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef9fc2-1b7e-40cb-91fd-1e72d4e6533e None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:47.970073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef9fc2-1b7e-40cb-91fd-1e72d4e6533e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:47.971561 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef9fc2-1b7e-40cb-91fd-1e72d4e6533e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:47.985813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-f9ef9fc2-1b7e-40cb-91fd-1e72d4e6533e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:47.985813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:05:47.985813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:05:47.985813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:05:47.985813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:05:47.985813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:05:47.985813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:05:47.985813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:05:47.985813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:05:47.985813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:05:47.985813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:05:47.985813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:05:47.985813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:05:47.985813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:05:47.985813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:05:47.987006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:05:47.988084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:05:47.988084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:05:47.988084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:05:47.988084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:05:47.988084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:05:47.988084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:05:47.988084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:05:47.988084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:05:47.988084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 551/1104] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:05:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:05:47.993509 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2c94d-ca1a-48aa-94fb-092a0b5cb92a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:48.096851 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2c94d-ca1a-48aa-94fb-092a0b5cb92a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:48.132330 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a17d9-901c-46af-ab7a-2d39aa4497fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:48.133000 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a17d9-901c-46af-ab7a-2d39aa4497fc None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:48.133281 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a17d9-901c-46af-ab7a-2d39aa4497fc None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:48.133618 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a17d9-901c-46af-ab7a-2d39aa4497fc None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:48.134432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a17d9-901c-46af-ab7a-2d39aa4497fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:48.134746 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a17d9-901c-46af-ab7a-2d39aa4497fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:48.136188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a17d9-901c-46af-ab7a-2d39aa4497fc None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:48.138765 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:48.149215 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 554/1105] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:05:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:48.155520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef9fc2-1b7e-40cb-91fd-1e72d4e6533e None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:48.209771 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:48.261251 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef9fc2-1b7e-40cb-91fd-1e72d4e6533e None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:48.298857 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ef99b-e6cb-4e4b-a83b-28c71a467cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:48.299529 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ef99b-e6cb-4e4b-a83b-28c71a467cb0 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:48.299815 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ef99b-e6cb-4e4b-a83b-28c71a467cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:48.300088 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ef99b-e6cb-4e4b-a83b-28c71a467cb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:48.300933 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ef99b-e6cb-4e4b-a83b-28c71a467cb0 None admin] RBAC: Adding query filter params (role.id=326ab279cac14b3fbafe4155443ed9b7, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=378b046698c04b7b88194a9ffe5b045a) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:48.301248 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ef99b-e6cb-4e4b-a83b-28c71a467cb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:48.302727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ef99b-e6cb-4e4b-a83b-28c71a467cb0 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:48.309633 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 552/1106] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:05:48 2022] GET /identity/v3/role_assignments?user.id=378b046698c04b7b88194a9ffe5b045a&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=326ab279cac14b3fbafe4155443ed9b7 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:49.100473 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49716710-3d88-4c2d-bc97-5c6d8dcde9f2 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:49.101047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49716710-3d88-4c2d-bc97-5c6d8dcde9f2 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:49.101047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49716710-3d88-4c2d-bc97-5c6d8dcde9f2 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:49.102549 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 555/1107] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:49.109271 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c968ee9-a6d7-4047-bcc4-27f997ca4116 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:49.109740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c968ee9-a6d7-4047-bcc4-27f997ca4116 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:49.109856 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c968ee9-a6d7-4047-bcc4-27f997ca4116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:49.131024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-7c968ee9-a6d7-4047-bcc4-27f997ca4116 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:49.132218 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-7c968ee9-a6d7-4047-bcc4-27f997ca4116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:49.206102 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 553/1108] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:49.232384 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49716710-3d88-4c2d-bc97-5c6d8dcde9f2 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:49.370927 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49716710-3d88-4c2d-bc97-5c6d8dcde9f2 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:49.406816 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b91a74-3baa-46ab-a7f2-a0624ef3089e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:49.407603 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b91a74-3baa-46ab-a7f2-a0624ef3089e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:49.407875 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b91a74-3baa-46ab-a7f2-a0624ef3089e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:49.408146 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b91a74-3baa-46ab-a7f2-a0624ef3089e None admin] PATH_INFO: `/v3/domains/default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:49.409364 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b91a74-3baa-46ab-a7f2-a0624ef3089e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:49.410845 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b91a74-3baa-46ab-a7f2-a0624ef3089e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:49.412453 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 556/1109] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:05:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:49.419600 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c968ee9-a6d7-4047-bcc4-27f997ca4116 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:49.467104 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:49.523495 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c968ee9-a6d7-4047-bcc4-27f997ca4116 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:49.563006 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fc198-da92-4d46-b155-89cc34601362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:49.563687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fc198-da92-4d46-b155-89cc34601362 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:49.563963 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fc198-da92-4d46-b155-89cc34601362 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:49.564230 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fc198-da92-4d46-b155-89cc34601362 None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:49.564942 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fc198-da92-4d46-b155-89cc34601362 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:49.566418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fc198-da92-4d46-b155-89cc34601362 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:49.587641 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 554/1110] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:05:49 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:05:51.966534 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5629d-85cf-41a1-8043-9e12a695716c None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:51.967100 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5629d-85cf-41a1-8043-9e12a695716c None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:51.967100 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5629d-85cf-41a1-8043-9e12a695716c None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:51.968387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 557/1111] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:51.975229 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b40c56-f4fc-4b8a-9465-1e1dbfe6a761 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:51.975980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b40c56-f4fc-4b8a-9465-1e1dbfe6a761 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:51.976098 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b40c56-f4fc-4b8a-9465-1e1dbfe6a761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:51.996411 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-c8b40c56-f4fc-4b8a-9465-1e1dbfe6a761 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:51.997627 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-c8b40c56-f4fc-4b8a-9465-1e1dbfe6a761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:52.068586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 555/1112] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:52.092109 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5629d-85cf-41a1-8043-9e12a695716c None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:52.160320 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:52.205569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5629d-85cf-41a1-8043-9e12a695716c None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:52.245315 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5e049-b615-4049-8333-db450b275047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:52.246103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5e049-b615-4049-8333-db450b275047 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:52.246447 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5e049-b615-4049-8333-db450b275047 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:52.246721 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5e049-b615-4049-8333-db450b275047 None admin] PATH_INFO: `/v3/roles/b21c1004651d40f1b44fb0b7becbe3c8` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:52.248334 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5e049-b615-4049-8333-db450b275047 None admin] RBAC: Authorizing `identity:get_role(role_id=b21c1004651d40f1b44fb0b7becbe3c8)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:52.249728 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5e049-b615-4049-8333-db450b275047 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:52.251149 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 558/1113] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:05:52 2022] GET /identity/v3/roles/b21c1004651d40f1b44fb0b7becbe3c8 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:52.260333 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b40c56-f4fc-4b8a-9465-1e1dbfe6a761 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:52.300232 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:52.372610 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b40c56-f4fc-4b8a-9465-1e1dbfe6a761 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:52.408791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e9841-4967-4af7-b42e-e8bd2e7a4470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:52.409667 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e9841-4967-4af7-b42e-e8bd2e7a4470 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:52.409946 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e9841-4967-4af7-b42e-e8bd2e7a4470 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:52.410218 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e9841-4967-4af7-b42e-e8bd2e7a4470 None admin] PATH_INFO: `/v3/users/e408d2c9e5da43769417b7e9f96bcaad` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:52.434246 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e9841-4967-4af7-b42e-e8bd2e7a4470 None admin] RBAC: Authorizing `identity:get_user(user_id=e408d2c9e5da43769417b7e9f96bcaad)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:52.435740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e9841-4967-4af7-b42e-e8bd2e7a4470 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:52.437483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 556/1114] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:05:52 2022] GET /identity/v3/users/e408d2c9e5da43769417b7e9f96bcaad => generated 286 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:52.446568 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5e049-b615-4049-8333-db450b275047 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:52.565641 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5e049-b615-4049-8333-db450b275047 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:52.598513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:52.604640 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2196119-9d7a-49e5-b466-221a13343cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:52.605292 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2196119-9d7a-49e5-b466-221a13343cdc None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:52.605638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2196119-9d7a-49e5-b466-221a13343cdc None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:52.605918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2196119-9d7a-49e5-b466-221a13343cdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:52.685300 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2196119-9d7a-49e5-b466-221a13343cdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:52.686800 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2196119-9d7a-49e5-b466-221a13343cdc None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:52.790886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 559/1115] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:05:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:05:52.797372 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e9841-4967-4af7-b42e-e8bd2e7a4470 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:52.888596 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:52.902158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e9841-4967-4af7-b42e-e8bd2e7a4470 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:52.939009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3faa31-140e-4486-9d9c-0548bf640f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:52.939879 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3faa31-140e-4486-9d9c-0548bf640f97 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:52.940155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3faa31-140e-4486-9d9c-0548bf640f97 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:52.940429 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3faa31-140e-4486-9d9c-0548bf640f97 None admin] PATH_INFO: `/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:52.941736 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3faa31-140e-4486-9d9c-0548bf640f97 None admin] RBAC: Authorizing `identity:get_project(project_id=70e2a49df21f45ab94b5c0ec6af1c6ba)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:52.943227 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3faa31-140e-4486-9d9c-0548bf640f97 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:52.944901 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 557/1116] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:05:52 2022] GET /identity/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:52.951372 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2196119-9d7a-49e5-b466-221a13343cdc None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:52.986804 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:53.058661 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2196119-9d7a-49e5-b466-221a13343cdc None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:53.094931 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728b9b1-6a51-4ce1-8478-cd11bbad4d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:53.095583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728b9b1-6a51-4ce1-8478-cd11bbad4d6c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:53.095860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728b9b1-6a51-4ce1-8478-cd11bbad4d6c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:53.096133 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728b9b1-6a51-4ce1-8478-cd11bbad4d6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:53.096964 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728b9b1-6a51-4ce1-8478-cd11bbad4d6c None admin] RBAC: Adding query filter params (role.id=b21c1004651d40f1b44fb0b7becbe3c8, scope.project.id=70e2a49df21f45ab94b5c0ec6af1c6ba, user.id=e408d2c9e5da43769417b7e9f96bcaad) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:53.097279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728b9b1-6a51-4ce1-8478-cd11bbad4d6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:53.098761 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728b9b1-6a51-4ce1-8478-cd11bbad4d6c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:53.104429 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 560/1117] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:05:52 2022] GET /identity/v3/role_assignments?user.id=e408d2c9e5da43769417b7e9f96bcaad&scope.project.id=70e2a49df21f45ab94b5c0ec6af1c6ba&role.id=b21c1004651d40f1b44fb0b7becbe3c8 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:55.267344 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa4bf1-88de-497d-8068-d8b419a619fa None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:55.268279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa4bf1-88de-497d-8068-d8b419a619fa None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:55.268279 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa4bf1-88de-497d-8068-d8b419a619fa None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:55.270511 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 558/1118] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:55.277931 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b521aec-e9fa-4dd1-a972-a6e70e1837a8 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:55.278429 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b521aec-e9fa-4dd1-a972-a6e70e1837a8 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:55.278534 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b521aec-e9fa-4dd1-a972-a6e70e1837a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:55.297584 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-0b521aec-e9fa-4dd1-a972-a6e70e1837a8 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:55.298828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-0b521aec-e9fa-4dd1-a972-a6e70e1837a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:55.370525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 561/1119] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:05:55.393742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa4bf1-88de-497d-8068-d8b419a619fa None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:55.501482 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa4bf1-88de-497d-8068-d8b419a619fa None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:55.536870 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a843ee4-1a76-4857-bd14-defef870e134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:55.537719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a843ee4-1a76-4857-bd14-defef870e134 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:55.537996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a843ee4-1a76-4857-bd14-defef870e134 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:55.538303 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a843ee4-1a76-4857-bd14-defef870e134 None admin] PATH_INFO: `/v3/roles/b21c1004651d40f1b44fb0b7becbe3c8` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:55.539867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a843ee4-1a76-4857-bd14-defef870e134 None admin] RBAC: Authorizing `identity:get_role(role_id=b21c1004651d40f1b44fb0b7becbe3c8)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:55.541243 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a843ee4-1a76-4857-bd14-defef870e134 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:55.542656 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 559/1120] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:05:55 2022] GET /identity/v3/roles/b21c1004651d40f1b44fb0b7becbe3c8 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:55.549237 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b521aec-e9fa-4dd1-a972-a6e70e1837a8 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:55.584711 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:55.660663 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b521aec-e9fa-4dd1-a972-a6e70e1837a8 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:55.696092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076e4cc-5f67-4651-97f1-07675471e35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:55.696967 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076e4cc-5f67-4651-97f1-07675471e35d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:55.697246 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076e4cc-5f67-4651-97f1-07675471e35d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:55.697523 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076e4cc-5f67-4651-97f1-07675471e35d None admin] PATH_INFO: `/v3/users/e408d2c9e5da43769417b7e9f96bcaad` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:55.698807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076e4cc-5f67-4651-97f1-07675471e35d None admin] RBAC: Authorizing `identity:get_user(user_id=e408d2c9e5da43769417b7e9f96bcaad)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:55.700261 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076e4cc-5f67-4651-97f1-07675471e35d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:55.701898 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 562/1121] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:05:55 2022] GET /identity/v3/users/e408d2c9e5da43769417b7e9f96bcaad => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:55.708106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a843ee4-1a76-4857-bd14-defef870e134 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:55.735255 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:55.812079 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a843ee4-1a76-4857-bd14-defef870e134 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:55.850302 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29128971-1512-4aad-ae5a-697f6e470101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:55.851197 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29128971-1512-4aad-ae5a-697f6e470101 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:55.851453 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29128971-1512-4aad-ae5a-697f6e470101 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:55.851738 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29128971-1512-4aad-ae5a-697f6e470101 None admin] PATH_INFO: `/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:55.853048 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29128971-1512-4aad-ae5a-697f6e470101 None admin] RBAC: Authorizing `identity:get_project(project_id=70e2a49df21f45ab94b5c0ec6af1c6ba)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:55.854525 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29128971-1512-4aad-ae5a-697f6e470101 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:55.856260 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 560/1122] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:05:55 2022] GET /identity/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:55.862699 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076e4cc-5f67-4651-97f1-07675471e35d None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:55.966811 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:55.974716 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076e4cc-5f67-4651-97f1-07675471e35d None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:56.015151 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03d8ad-6ce9-4b21-b4a5-fbe4e9eb46f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:56.016043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03d8ad-6ce9-4b21-b4a5-fbe4e9eb46f0 None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:56.016319 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03d8ad-6ce9-4b21-b4a5-fbe4e9eb46f0 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:56.016594 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03d8ad-6ce9-4b21-b4a5-fbe4e9eb46f0 None admin] PATH_INFO: `/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba/users/e408d2c9e5da43769417b7e9f96bcaad/roles/b21c1004651d40f1b44fb0b7becbe3c8` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:56.019157 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03d8ad-6ce9-4b21-b4a5-fbe4e9eb46f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=70e2a49df21f45ab94b5c0ec6af1c6ba, user_id=e408d2c9e5da43769417b7e9f96bcaad, role_id=b21c1004651d40f1b44fb0b7becbe3c8)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:56.020611 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03d8ad-6ce9-4b21-b4a5-fbe4e9eb46f0 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:56.041909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 563/1123] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:05:55 2022] PUT /identity/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba/users/e408d2c9e5da43769417b7e9f96bcaad/roles/b21c1004651d40f1b44fb0b7becbe3c8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:05:58.271001 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95abff-f61d-4d6e-98b1-74c3a93b1e02 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:58.271876 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95abff-f61d-4d6e-98b1-74c3a93b1e02 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:58.271876 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95abff-f61d-4d6e-98b1-74c3a93b1e02 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:58.274699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 561/1124] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:05:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:05:58.282643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217bb99-7490-4bdb-a0bb-077072e0a27d None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:58.283052 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217bb99-7490-4bdb-a0bb-077072e0a27d None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:58.283240 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217bb99-7490-4bdb-a0bb-077072e0a27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:58.309501 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-6217bb99-7490-4bdb-a0bb-077072e0a27d None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:05:58.310739 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-6217bb99-7490-4bdb-a0bb-077072e0a27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:05:58.384644 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 564/1125] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:05:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:05:58.409085 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95abff-f61d-4d6e-98b1-74c3a93b1e02 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:58.526352 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:58.537337 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95abff-f61d-4d6e-98b1-74c3a93b1e02 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:58.576612 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6e3cd-ae30-46d7-a6e5-ad05b35bce90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:58.577537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6e3cd-ae30-46d7-a6e5-ad05b35bce90 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:58.577815 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6e3cd-ae30-46d7-a6e5-ad05b35bce90 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:58.578088 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6e3cd-ae30-46d7-a6e5-ad05b35bce90 None admin] PATH_INFO: `/v3/roles/b21c1004651d40f1b44fb0b7becbe3c8` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:58.579667 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6e3cd-ae30-46d7-a6e5-ad05b35bce90 None admin] RBAC: Authorizing `identity:get_role(role_id=b21c1004651d40f1b44fb0b7becbe3c8)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:58.581052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6e3cd-ae30-46d7-a6e5-ad05b35bce90 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:58.582484 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 562/1126] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:05:58 2022] GET /identity/v3/roles/b21c1004651d40f1b44fb0b7becbe3c8 => generated 223 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:58.588928 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217bb99-7490-4bdb-a0bb-077072e0a27d None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:58.677290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:58.697239 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217bb99-7490-4bdb-a0bb-077072e0a27d None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:58.736745 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d09d4-3ad8-40c7-9bd5-2e2fed7ab037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:58.737632 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d09d4-3ad8-40c7-9bd5-2e2fed7ab037 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:58.737917 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d09d4-3ad8-40c7-9bd5-2e2fed7ab037 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:58.738185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d09d4-3ad8-40c7-9bd5-2e2fed7ab037 None admin] PATH_INFO: `/v3/users/e408d2c9e5da43769417b7e9f96bcaad` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:58.739564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d09d4-3ad8-40c7-9bd5-2e2fed7ab037 None admin] RBAC: Authorizing `identity:get_user(user_id=e408d2c9e5da43769417b7e9f96bcaad)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:58.741022 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d09d4-3ad8-40c7-9bd5-2e2fed7ab037 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:58.743045 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 565/1127] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:05:58 2022] GET /identity/v3/users/e408d2c9e5da43769417b7e9f96bcaad => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:58.751821 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6e3cd-ae30-46d7-a6e5-ad05b35bce90 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:58.861364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6e3cd-ae30-46d7-a6e5-ad05b35bce90 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:58.896602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef7410-cb5c-4605-90ef-731f111a839b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:58.897243 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef7410-cb5c-4605-90ef-731f111a839b None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:58.897521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef7410-cb5c-4605-90ef-731f111a839b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:58.897794 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef7410-cb5c-4605-90ef-731f111a839b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:58.926249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:58.968035 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef7410-cb5c-4605-90ef-731f111a839b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:58.969817 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef7410-cb5c-4605-90ef-731f111a839b None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:59.071194 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 563/1128] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:05:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:05:59.077560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d09d4-3ad8-40c7-9bd5-2e2fed7ab037 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:59.202203 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d09d4-3ad8-40c7-9bd5-2e2fed7ab037 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:59.241154 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e79242-1662-49a6-9fe1-5d9069714359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:59.242434 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e79242-1662-49a6-9fe1-5d9069714359 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:59.242812 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e79242-1662-49a6-9fe1-5d9069714359 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:59.243195 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e79242-1662-49a6-9fe1-5d9069714359 None admin] PATH_INFO: `/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:59.245242 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e79242-1662-49a6-9fe1-5d9069714359 None admin] RBAC: Authorizing `identity:get_project(project_id=70e2a49df21f45ab94b5c0ec6af1c6ba)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:59.247571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e79242-1662-49a6-9fe1-5d9069714359 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:59.250213 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 566/1129] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:05:59 2022] GET /identity/v3/projects/70e2a49df21f45ab94b5c0ec6af1c6ba => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:05:59.256436 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef7410-cb5c-4605-90ef-731f111a839b None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:05:59.322148 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:05:59.364140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef7410-cb5c-4605-90ef-731f111a839b None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:05:59.401631 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e62f2-5740-41ce-a5ff-7f02ba656829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:05:59.402309 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e62f2-5740-41ce-a5ff-7f02ba656829 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:59.402583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e62f2-5740-41ce-a5ff-7f02ba656829 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:59.402865 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e62f2-5740-41ce-a5ff-7f02ba656829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:59.403697 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e62f2-5740-41ce-a5ff-7f02ba656829 None admin] RBAC: Adding query filter params (role.id=b21c1004651d40f1b44fb0b7becbe3c8, scope.project.id=70e2a49df21f45ab94b5c0ec6af1c6ba, user.id=e408d2c9e5da43769417b7e9f96bcaad) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:05:59.404009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e62f2-5740-41ce-a5ff-7f02ba656829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:05:59.405447 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e62f2-5740-41ce-a5ff-7f02ba656829 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:05:59.412119 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 564/1130] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:05:59 2022] GET /identity/v3/role_assignments?user.id=e408d2c9e5da43769417b7e9f96bcaad&scope.project.id=70e2a49df21f45ab94b5c0ec6af1c6ba&role.id=b21c1004651d40f1b44fb0b7becbe3c8 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:01.664628 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c09c6-7ae2-480a-b09f-0b5d73112123 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:01.665189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c09c6-7ae2-480a-b09f-0b5d73112123 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:01.665189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c09c6-7ae2-480a-b09f-0b5d73112123 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:01.667323 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 567/1131] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:06:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:01.676390 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f4faf-65c2-4282-b719-8a80ad724691 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:01.677106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f4faf-65c2-4282-b719-8a80ad724691 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:01.677257 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f4faf-65c2-4282-b719-8a80ad724691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:01.702830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-f81f4faf-65c2-4282-b719-8a80ad724691 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:01.704031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-f81f4faf-65c2-4282-b719-8a80ad724691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:01.775379 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 565/1132] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:06:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:06:01.799761 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c09c6-7ae2-480a-b09f-0b5d73112123 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:01.825218 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:01.926418 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c09c6-7ae2-480a-b09f-0b5d73112123 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:01.963934 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7996e1-35dc-4b72-82d9-96fb44ae3bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:01.964730 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7996e1-35dc-4b72-82d9-96fb44ae3bfc None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:01.965009 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7996e1-35dc-4b72-82d9-96fb44ae3bfc None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:01.965274 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7996e1-35dc-4b72-82d9-96fb44ae3bfc None admin] PATH_INFO: `/v3/domains/default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:01.966725 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7996e1-35dc-4b72-82d9-96fb44ae3bfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:01.968194 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7996e1-35dc-4b72-82d9-96fb44ae3bfc None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:01.970156 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 568/1133] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:06:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:01.979173 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f4faf-65c2-4282-b719-8a80ad724691 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:02.038653 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:02.091739 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f4faf-65c2-4282-b719-8a80ad724691 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:02.128475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb92e0c-d542-4547-95ed-9532ac105957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:02.129140 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb92e0c-d542-4547-95ed-9532ac105957 None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:02.129420 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb92e0c-d542-4547-95ed-9532ac105957 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:02.129693 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb92e0c-d542-4547-95ed-9532ac105957 None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:02.130437 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb92e0c-d542-4547-95ed-9532ac105957 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:02.131886 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb92e0c-d542-4547-95ed-9532ac105957 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:02.160065 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 566/1134] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:06:01 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:06:04.411726 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5efdb5-868a-447d-a580-c2bce2f9f110 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:04.412337 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5efdb5-868a-447d-a580-c2bce2f9f110 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:04.412337 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5efdb5-868a-447d-a580-c2bce2f9f110 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:04.413649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 569/1135] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:04.420397 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f0280-243d-4e33-8a3c-8867fd904fb1 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:04.421142 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f0280-243d-4e33-8a3c-8867fd904fb1 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:04.421287 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f0280-243d-4e33-8a3c-8867fd904fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:04.448540 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-ee8f0280-243d-4e33-8a3c-8867fd904fb1 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:04.449719 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-ee8f0280-243d-4e33-8a3c-8867fd904fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:04.519954 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 567/1136] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:06:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:06:04.550198 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5efdb5-868a-447d-a580-c2bce2f9f110 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:04.647227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:04.663625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5efdb5-868a-447d-a580-c2bce2f9f110 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:04.699402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0a616-4081-44bc-8ff2-ef261e1353d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:04.700223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0a616-4081-44bc-8ff2-ef261e1353d1 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:04.700501 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0a616-4081-44bc-8ff2-ef261e1353d1 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:04.700749 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0a616-4081-44bc-8ff2-ef261e1353d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:04.702071 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0a616-4081-44bc-8ff2-ef261e1353d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:04.703563 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0a616-4081-44bc-8ff2-ef261e1353d1 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:04.705195 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 570/1137] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:06:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:04.711654 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f0280-243d-4e33-8a3c-8867fd904fb1 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:04.750284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:04.817440 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f0280-243d-4e33-8a3c-8867fd904fb1 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:04.855019 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ec209-3722-4b2d-a525-5cb894aaf2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:04.855702 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ec209-3722-4b2d-a525-5cb894aaf2ab None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:04.855981 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ec209-3722-4b2d-a525-5cb894aaf2ab None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:04.856248 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ec209-3722-4b2d-a525-5cb894aaf2ab None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:04.856958 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ec209-3722-4b2d-a525-5cb894aaf2ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:04.858427 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ec209-3722-4b2d-a525-5cb894aaf2ab None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:04.882846 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 568/1138] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:06:04 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:06:07.134640 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f83884-3977-4683-a9d2-c141a4e69583 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:07.135495 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f83884-3977-4683-a9d2-c141a4e69583 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:07.135495 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f83884-3977-4683-a9d2-c141a4e69583 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:07.137751 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 571/1139] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:06:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:07.144067 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb7d6c-edb8-4c73-9ba3-a755eb2da56b None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:07.144779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb7d6c-edb8-4c73-9ba3-a755eb2da56b None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:07.144991 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb7d6c-edb8-4c73-9ba3-a755eb2da56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:07.164622 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-84eb7d6c-edb8-4c73-9ba3-a755eb2da56b None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:07.165979 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-84eb7d6c-edb8-4c73-9ba3-a755eb2da56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:07.244739 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 569/1140] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:06:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:06:07.268915 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f83884-3977-4683-a9d2-c141a4e69583 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:07.393359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f83884-3977-4683-a9d2-c141a4e69583 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:07.435072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046b364-3759-4d1b-935b-8fa50869f72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:07.435872 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046b364-3759-4d1b-935b-8fa50869f72f None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:07.436150 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046b364-3759-4d1b-935b-8fa50869f72f None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:07.436429 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046b364-3759-4d1b-935b-8fa50869f72f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:07.448291 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046b364-3759-4d1b-935b-8fa50869f72f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-a046b364-3759-4d1b-935b-8fa50869f72f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:07.455958 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:07.458977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:07.460189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:07.460189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:07.460189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:07.460189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:07.460189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:07.460189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:07.460189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:07.460189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:07.460189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:07.460189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:07.460189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:07.460189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:06:07.460189 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 572/1141] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:06:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:07.464933 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb7d6c-edb8-4c73-9ba3-a755eb2da56b None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:07.490807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:07.576005 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb7d6c-edb8-4c73-9ba3-a755eb2da56b None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:07.612830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44c9fa-a9b2-476a-a6f8-73e903ad1281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:07.613497 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44c9fa-a9b2-476a-a6f8-73e903ad1281 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:07.613779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44c9fa-a9b2-476a-a6f8-73e903ad1281 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:07.614056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44c9fa-a9b2-476a-a6f8-73e903ad1281 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:07.614838 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44c9fa-a9b2-476a-a6f8-73e903ad1281 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:07.615145 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44c9fa-a9b2-476a-a6f8-73e903ad1281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:07.616529 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44c9fa-a9b2-476a-a6f8-73e903ad1281 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:07.628538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 570/1142] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:06:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:07.634855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046b364-3759-4d1b-935b-8fa50869f72f None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:07.655708 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:07.741888 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046b364-3759-4d1b-935b-8fa50869f72f None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:07.776913 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d6edc-c776-411a-984c-0a3dfb648a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:07.777773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d6edc-c776-411a-984c-0a3dfb648a93 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:07.778048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d6edc-c776-411a-984c-0a3dfb648a93 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:07.778316 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d6edc-c776-411a-984c-0a3dfb648a93 None admin] PATH_INFO: `/v3/users/3225cd01256a4fad85296a6b200e2c10` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:07.792277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d6edc-c776-411a-984c-0a3dfb648a93 None admin] RBAC: Authorizing `identity:get_user(user_id=3225cd01256a4fad85296a6b200e2c10)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:07.793728 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d6edc-c776-411a-984c-0a3dfb648a93 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:07.795402 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 573/1143] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:06:07 2022] GET /identity/v3/users/3225cd01256a4fad85296a6b200e2c10 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:07.801597 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44c9fa-a9b2-476a-a6f8-73e903ad1281 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:07.892537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:07.910627 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44c9fa-a9b2-476a-a6f8-73e903ad1281 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:07.946623 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee80a-bc4e-427f-bb6c-70e469068585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:07.947364 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee80a-bc4e-427f-bb6c-70e469068585 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:07.947646 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee80a-bc4e-427f-bb6c-70e469068585 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:07.947932 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee80a-bc4e-427f-bb6c-70e469068585 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:08.018730 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee80a-bc4e-427f-bb6c-70e469068585 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:08.020196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee80a-bc4e-427f-bb6c-70e469068585 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:08.109877 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:08.130668 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 571/1144] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:06:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:06:08.137232 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d6edc-c776-411a-984c-0a3dfb648a93 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:08.228022 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:08.263731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d6edc-c776-411a-984c-0a3dfb648a93 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:08.299171 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af39cce-717a-47dc-a439-235b1b4f8238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:08.300025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af39cce-717a-47dc-a439-235b1b4f8238 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:08.300299 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af39cce-717a-47dc-a439-235b1b4f8238 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:08.300563 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af39cce-717a-47dc-a439-235b1b4f8238 None admin] PATH_INFO: `/v3/projects/1cb69f86881b4e108d2ee37877b17937` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:08.301861 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af39cce-717a-47dc-a439-235b1b4f8238 None admin] RBAC: Authorizing `identity:get_project(project_id=1cb69f86881b4e108d2ee37877b17937)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:08.303332 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af39cce-717a-47dc-a439-235b1b4f8238 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:08.305041 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 574/1145] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:06:08 2022] GET /identity/v3/projects/1cb69f86881b4e108d2ee37877b17937 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:08.312449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee80a-bc4e-427f-bb6c-70e469068585 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:08.435776 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee80a-bc4e-427f-bb6c-70e469068585 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:08.474187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba399e9c-80c3-40fa-b035-c7a768b2de83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:08.474873 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba399e9c-80c3-40fa-b035-c7a768b2de83 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:08.475153 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba399e9c-80c3-40fa-b035-c7a768b2de83 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:08.475430 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba399e9c-80c3-40fa-b035-c7a768b2de83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:08.476268 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba399e9c-80c3-40fa-b035-c7a768b2de83 None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=1cb69f86881b4e108d2ee37877b17937, user.id=3225cd01256a4fad85296a6b200e2c10) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:08.476590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba399e9c-80c3-40fa-b035-c7a768b2de83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:08.478727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba399e9c-80c3-40fa-b035-c7a768b2de83 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:08.484333 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 572/1146] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:06:08 2022] GET /identity/v3/role_assignments?user.id=3225cd01256a4fad85296a6b200e2c10&scope.project.id=1cb69f86881b4e108d2ee37877b17937&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:10.709447 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410d993-571a-422c-a215-55cce990821c None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:10.710074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410d993-571a-422c-a215-55cce990821c None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:10.710074 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410d993-571a-422c-a215-55cce990821c None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:10.711569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 575/1147] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:10.718177 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bd713-39ca-42b0-abb3-ada6beb82041 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:10.718973 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bd713-39ca-42b0-abb3-ada6beb82041 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:10.719463 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bd713-39ca-42b0-abb3-ada6beb82041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:10.741598 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-6f1bd713-39ca-42b0-abb3-ada6beb82041 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:10.743035 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-6f1bd713-39ca-42b0-abb3-ada6beb82041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:10.793391 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:10.819558 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 573/1148] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:06:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:06:10.842519 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410d993-571a-422c-a215-55cce990821c None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:10.958392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410d993-571a-422c-a215-55cce990821c None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:10.995449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e5a20-6a71-4545-857b-4c129e897713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:10.996236 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e5a20-6a71-4545-857b-4c129e897713 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:10.996514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e5a20-6a71-4545-857b-4c129e897713 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:10.996785 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e5a20-6a71-4545-857b-4c129e897713 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:11.008224 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e5a20-6a71-4545-857b-4c129e897713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-ee3e5a20-6a71-4545-857b-4c129e897713 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:11.015772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:11.017010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:11.018030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:11.018030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:11.018030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:11.018030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:11.018030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:11.018030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:11.018030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:11.018030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:11.018030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:11.018030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:11.018030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:11.018030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:06:11.018030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 576/1149] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:06:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:11.023949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1bd713-39ca-42b0-abb3-ada6beb82041 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:11.137667 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1bd713-39ca-42b0-abb3-ada6beb82041 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:11.176251 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0ea0b-c77e-4187-8563-8a8f3faa7f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:11.176929 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0ea0b-c77e-4187-8563-8a8f3faa7f17 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:11.177212 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0ea0b-c77e-4187-8563-8a8f3faa7f17 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:11.177488 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0ea0b-c77e-4187-8563-8a8f3faa7f17 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:11.178238 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0ea0b-c77e-4187-8563-8a8f3faa7f17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:11.178586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0ea0b-c77e-4187-8563-8a8f3faa7f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:11.180054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0ea0b-c77e-4187-8563-8a8f3faa7f17 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:11.189607 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 574/1150] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:06:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:11.195818 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e5a20-6a71-4545-857b-4c129e897713 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:11.199138 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:11.317188 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e5a20-6a71-4545-857b-4c129e897713 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:11.359090 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf698b9-5550-4bd5-9121-4540e89c0d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:11.360001 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf698b9-5550-4bd5-9121-4540e89c0d27 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:11.360282 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf698b9-5550-4bd5-9121-4540e89c0d27 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:11.360558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf698b9-5550-4bd5-9121-4540e89c0d27 None admin] PATH_INFO: `/v3/users/3225cd01256a4fad85296a6b200e2c10` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:11.361963 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf698b9-5550-4bd5-9121-4540e89c0d27 None admin] RBAC: Authorizing `identity:get_user(user_id=3225cd01256a4fad85296a6b200e2c10)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:11.363473 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf698b9-5550-4bd5-9121-4540e89c0d27 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:11.365104 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 577/1151] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:06:11 2022] GET /identity/v3/users/3225cd01256a4fad85296a6b200e2c10 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:11.371437 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0ea0b-c77e-4187-8563-8a8f3faa7f17 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:11.415044 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:11.480769 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0ea0b-c77e-4187-8563-8a8f3faa7f17 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:11.517640 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c14ea2-878f-464d-aea8-129f826763c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:11.518528 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c14ea2-878f-464d-aea8-129f826763c5 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:11.518804 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c14ea2-878f-464d-aea8-129f826763c5 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:11.519073 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c14ea2-878f-464d-aea8-129f826763c5 None admin] PATH_INFO: `/v3/projects/1cb69f86881b4e108d2ee37877b17937` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:11.520365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c14ea2-878f-464d-aea8-129f826763c5 None admin] RBAC: Authorizing `identity:get_project(project_id=1cb69f86881b4e108d2ee37877b17937)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:11.521802 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c14ea2-878f-464d-aea8-129f826763c5 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:11.523533 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 575/1152] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:06:11 2022] GET /identity/v3/projects/1cb69f86881b4e108d2ee37877b17937 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:11.530690 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf698b9-5550-4bd5-9121-4540e89c0d27 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:11.609329 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:11.640918 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf698b9-5550-4bd5-9121-4540e89c0d27 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:11.683141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0fe68-1980-4d34-b3a9-32fbd118ba62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:11.683965 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0fe68-1980-4d34-b3a9-32fbd118ba62 None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:11.684244 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0fe68-1980-4d34-b3a9-32fbd118ba62 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:11.684525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0fe68-1980-4d34-b3a9-32fbd118ba62 None admin] PATH_INFO: `/v3/projects/1cb69f86881b4e108d2ee37877b17937/users/3225cd01256a4fad85296a6b200e2c10/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:11.686681 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0fe68-1980-4d34-b3a9-32fbd118ba62 None admin] RBAC: Authorizing `identity:create_grant(project_id=1cb69f86881b4e108d2ee37877b17937, user_id=3225cd01256a4fad85296a6b200e2c10, role_id=b34e4ad4fe7b4a60bd2fc669e2b31a66)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:11.688142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0fe68-1980-4d34-b3a9-32fbd118ba62 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:11.703787 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 578/1153] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:06:11 2022] PUT /identity/v3/projects/1cb69f86881b4e108d2ee37877b17937/users/3225cd01256a4fad85296a6b200e2c10/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:06:13.922983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2c20e-4c56-4f04-982f-68644bfa55a8 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:13.923832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2c20e-4c56-4f04-982f-68644bfa55a8 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:13.923832 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2c20e-4c56-4f04-982f-68644bfa55a8 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:13.925971 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 576/1154] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:06:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:13.931567 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037ce4c-ca4d-4288-8bb5-b0cf3c2af41c None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:13.932106 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037ce4c-ca4d-4288-8bb5-b0cf3c2af41c None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:13.932106 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037ce4c-ca4d-4288-8bb5-b0cf3c2af41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:13.950874 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-3037ce4c-ca4d-4288-8bb5-b0cf3c2af41c None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:13.952038 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-3037ce4c-ca4d-4288-8bb5-b0cf3c2af41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:14.022324 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 579/1155] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:06:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:06:14.045854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2c20e-4c56-4f04-982f-68644bfa55a8 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:14.167056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2c20e-4c56-4f04-982f-68644bfa55a8 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:14.176498 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:14.207209 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f5dcd-7382-47af-8114-9b8f2539f757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:14.208016 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f5dcd-7382-47af-8114-9b8f2539f757 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:14.208303 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f5dcd-7382-47af-8114-9b8f2539f757 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:14.208584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f5dcd-7382-47af-8114-9b8f2539f757 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:14.221224 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f5dcd-7382-47af-8114-9b8f2539f757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-667f5dcd-7382-47af-8114-9b8f2539f757 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:14.229042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:14.230205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:14.231175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:14.231175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:14.231175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:14.231175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:14.231175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:14.231175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:14.231175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:14.231175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:14.231175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:14.231175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:14.231175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:14.231175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:06:14.231175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 577/1156] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:06:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:14.236487 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037ce4c-ca4d-4288-8bb5-b0cf3c2af41c None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:14.300870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:14.344818 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037ce4c-ca4d-4288-8bb5-b0cf3c2af41c None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:14.380986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f7150-a1eb-4b23-ab22-6fc4afaf0cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:14.381682 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f7150-a1eb-4b23-ab22-6fc4afaf0cdc None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:14.381961 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f7150-a1eb-4b23-ab22-6fc4afaf0cdc None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:14.382234 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f7150-a1eb-4b23-ab22-6fc4afaf0cdc None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:14.383129 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f7150-a1eb-4b23-ab22-6fc4afaf0cdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:14.383440 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f7150-a1eb-4b23-ab22-6fc4afaf0cdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:14.384845 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f7150-a1eb-4b23-ab22-6fc4afaf0cdc None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:14.393921 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 580/1157] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:14.403380 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f5dcd-7382-47af-8114-9b8f2539f757 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:14.522094 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f5dcd-7382-47af-8114-9b8f2539f757 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:14.561346 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360865bc-af3f-4986-88f0-fed87cafafc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:14.562234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360865bc-af3f-4986-88f0-fed87cafafc0 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:14.562548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360865bc-af3f-4986-88f0-fed87cafafc0 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:14.562823 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360865bc-af3f-4986-88f0-fed87cafafc0 None admin] PATH_INFO: `/v3/users/3225cd01256a4fad85296a6b200e2c10` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:14.564060 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360865bc-af3f-4986-88f0-fed87cafafc0 None admin] RBAC: Authorizing `identity:get_user(user_id=3225cd01256a4fad85296a6b200e2c10)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:14.565526 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360865bc-af3f-4986-88f0-fed87cafafc0 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:14.567205 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 578/1158] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:06:14 2022] GET /identity/v3/users/3225cd01256a4fad85296a6b200e2c10 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:14.573824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f7150-a1eb-4b23-ab22-6fc4afaf0cdc None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:14.693082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f7150-a1eb-4b23-ab22-6fc4afaf0cdc None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:14.716222 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:14.730753 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660294a1-1f09-48df-acef-fb50ce16f1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:14.731430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660294a1-1f09-48df-acef-fb50ce16f1c1 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:14.731710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660294a1-1f09-48df-acef-fb50ce16f1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:14.731990 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660294a1-1f09-48df-acef-fb50ce16f1c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:14.802999 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660294a1-1f09-48df-acef-fb50ce16f1c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:14.804498 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660294a1-1f09-48df-acef-fb50ce16f1c1 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:14.919880 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 581/1159] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:06:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:06:14.929253 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360865bc-af3f-4986-88f0-fed87cafafc0 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:14.976258 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:15.045966 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360865bc-af3f-4986-88f0-fed87cafafc0 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:15.083644 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9836a7-eede-4bcd-9f68-51b82fe33c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:15.084438 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9836a7-eede-4bcd-9f68-51b82fe33c8e None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:15.084720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9836a7-eede-4bcd-9f68-51b82fe33c8e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:15.084994 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9836a7-eede-4bcd-9f68-51b82fe33c8e None admin] PATH_INFO: `/v3/projects/1cb69f86881b4e108d2ee37877b17937` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:15.086307 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9836a7-eede-4bcd-9f68-51b82fe33c8e None admin] RBAC: Authorizing `identity:get_project(project_id=1cb69f86881b4e108d2ee37877b17937)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:15.087851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9836a7-eede-4bcd-9f68-51b82fe33c8e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:15.089580 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 579/1160] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:06:14 2022] GET /identity/v3/projects/1cb69f86881b4e108d2ee37877b17937 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:15.097546 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660294a1-1f09-48df-acef-fb50ce16f1c1 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:15.125972 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:15.219790 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660294a1-1f09-48df-acef-fb50ce16f1c1 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:15.261786 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ffee1-b07d-48b0-ae26-2e2013cd18df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:15.262487 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ffee1-b07d-48b0-ae26-2e2013cd18df None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:15.262882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ffee1-b07d-48b0-ae26-2e2013cd18df None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:15.263035 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ffee1-b07d-48b0-ae26-2e2013cd18df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:15.263882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ffee1-b07d-48b0-ae26-2e2013cd18df None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=1cb69f86881b4e108d2ee37877b17937, user.id=3225cd01256a4fad85296a6b200e2c10) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:15.264197 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ffee1-b07d-48b0-ae26-2e2013cd18df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:15.265667 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ffee1-b07d-48b0-ae26-2e2013cd18df None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:15.272993 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 582/1161] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:06:15 2022] GET /identity/v3/role_assignments?user.id=3225cd01256a4fad85296a6b200e2c10&scope.project.id=1cb69f86881b4e108d2ee37877b17937&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:17.658686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9ff3b-503f-4b87-baee-d6ae29378e52 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:17.659615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9ff3b-503f-4b87-baee-d6ae29378e52 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:17.659615 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9ff3b-503f-4b87-baee-d6ae29378e52 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:17.661800 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 580/1162] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:06:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:17.667420 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd3978-45cb-4cae-9d65-a189c18de205 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:17.667860 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd3978-45cb-4cae-9d65-a189c18de205 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:17.667965 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd3978-45cb-4cae-9d65-a189c18de205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:17.686290 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-37cd3978-45cb-4cae-9d65-a189c18de205 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:17.687485 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-37cd3978-45cb-4cae-9d65-a189c18de205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:17.763684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 583/1163] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:06:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:06:17.787522 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9ff3b-503f-4b87-baee-d6ae29378e52 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:17.903545 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9ff3b-503f-4b87-baee-d6ae29378e52 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:17.905857 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:17.942539 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe47cb0-bc74-48bf-a6e1-5a8376286934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:17.943191 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe47cb0-bc74-48bf-a6e1-5a8376286934 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:17.943474 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe47cb0-bc74-48bf-a6e1-5a8376286934 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:17.943752 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe47cb0-bc74-48bf-a6e1-5a8376286934 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:18.014909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe47cb0-bc74-48bf-a6e1-5a8376286934 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:18.016361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe47cb0-bc74-48bf-a6e1-5a8376286934 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:18.125964 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 581/1164] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:06:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:06:18.135039 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd3978-45cb-4cae-9d65-a189c18de205 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:18.155932 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:18.248629 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd3978-45cb-4cae-9d65-a189c18de205 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:18.284256 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e526f44-d59b-422c-b461-ded97213722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:18.285089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e526f44-d59b-422c-b461-ded97213722c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:18.285353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e526f44-d59b-422c-b461-ded97213722c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:18.285621 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e526f44-d59b-422c-b461-ded97213722c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:18.293312 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e526f44-d59b-422c-b461-ded97213722c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:18.294809 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e526f44-d59b-422c-b461-ded97213722c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-4e526f44-d59b-422c-b461-ded97213722c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:18.310235 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:18.311569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:18.312727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:18.312727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:06:18.312727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:18.312727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:18.312727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:18.312727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:18.312727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:18.312727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:18.312727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:18.312727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:18.312727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 20:06:18.312727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:06:18.312727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 20:06:18.312727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:06:18.312727 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 584/1165] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:06:18 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:18.318312 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe47cb0-bc74-48bf-a6e1-5a8376286934 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:18.323353 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:18.444286 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe47cb0-bc74-48bf-a6e1-5a8376286934 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:18.479792 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09168335-721f-401c-82ec-c3fc0509cd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:18.480457 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09168335-721f-401c-82ec-c3fc0509cd69 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:18.480733 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09168335-721f-401c-82ec-c3fc0509cd69 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:18.481019 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09168335-721f-401c-82ec-c3fc0509cd69 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:18.481739 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09168335-721f-401c-82ec-c3fc0509cd69 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:18.482044 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09168335-721f-401c-82ec-c3fc0509cd69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:18.483453 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09168335-721f-401c-82ec-c3fc0509cd69 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:18.490675 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 582/1166] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 20:06:18 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:20.710759 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c67da1-cb95-4291-aaa4-c731a05edfe6 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:20.711667 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c67da1-cb95-4291-aaa4-c731a05edfe6 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:20.711667 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c67da1-cb95-4291-aaa4-c731a05edfe6 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:20.713798 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 585/1167] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:06:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:20.719900 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013c4f7-4e20-4abe-8abc-fb5e1061b6fb None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:20.720317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013c4f7-4e20-4abe-8abc-fb5e1061b6fb None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:20.720503 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013c4f7-4e20-4abe-8abc-fb5e1061b6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:20.738620 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-e013c4f7-4e20-4abe-8abc-fb5e1061b6fb None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:20.739811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-e013c4f7-4e20-4abe-8abc-fb5e1061b6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:20.810595 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:20.821039 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 583/1168] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:06:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:06:20.851171 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c67da1-cb95-4291-aaa4-c731a05edfe6 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:20.937140 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:20.965894 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c67da1-cb95-4291-aaa4-c731a05edfe6 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:21.007620 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6c2c0-120b-4ec2-a402-a0c63a7f2a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:21.008382 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6c2c0-120b-4ec2-a402-a0c63a7f2a33 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:21.008729 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6c2c0-120b-4ec2-a402-a0c63a7f2a33 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:21.009003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6c2c0-120b-4ec2-a402-a0c63a7f2a33 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:21.009762 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6c2c0-120b-4ec2-a402-a0c63a7f2a33 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:21.011179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6c2c0-120b-4ec2-a402-a0c63a7f2a33 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:21.033757 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 586/1169] 10.222.0.84 () {60 vars in 1138 bytes} [Thu May 19 20:06:20 2022] POST /identity/v3/domains => generated 267 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:06:23.272385 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6768a49-5f62-4beb-8d4f-9d0f6f4be096 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:23.273300 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6768a49-5f62-4beb-8d4f-9d0f6f4be096 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:23.273410 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6768a49-5f62-4beb-8d4f-9d0f6f4be096 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:23.275513 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 584/1170] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:06:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:23.284689 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea03d7-5e84-4f85-9cde-a40eaeee6bf7 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:23.285449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea03d7-5e84-4f85-9cde-a40eaeee6bf7 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:23.285624 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea03d7-5e84-4f85-9cde-a40eaeee6bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:23.305970 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-b3ea03d7-5e84-4f85-9cde-a40eaeee6bf7 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:23.307223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-b3ea03d7-5e84-4f85-9cde-a40eaeee6bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:23.379575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 587/1171] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:06:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:06:23.406052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6768a49-5f62-4beb-8d4f-9d0f6f4be096 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:23.524650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6768a49-5f62-4beb-8d4f-9d0f6f4be096 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:23.563431 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d777ab-e9ec-4ce8-aa09-dea57094d38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:23.564326 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d777ab-e9ec-4ce8-aa09-dea57094d38a None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:23.564604 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d777ab-e9ec-4ce8-aa09-dea57094d38a None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:23.564872 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d777ab-e9ec-4ce8-aa09-dea57094d38a None admin] PATH_INFO: `/v3/domains/29a81afcdb004692b2b1cfe0a331d4ea` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:23.576670 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d777ab-e9ec-4ce8-aa09-dea57094d38a None admin] RBAC: Authorizing `identity:get_domain(domain_id=29a81afcdb004692b2b1cfe0a331d4ea)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:23.578120 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d777ab-e9ec-4ce8-aa09-dea57094d38a None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:23.579816 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 585/1172] 10.222.0.84 () {56 vars in 1183 bytes} [Thu May 19 20:06:23 2022] GET /identity/v3/domains/29a81afcdb004692b2b1cfe0a331d4ea => generated 267 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:23.587008 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea03d7-5e84-4f85-9cde-a40eaeee6bf7 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:23.665589 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:23.701535 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea03d7-5e84-4f85-9cde-a40eaeee6bf7 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:23.738850 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dee718-e2e2-40f7-a5b5-631d3a0adee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:23.739501 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dee718-e2e2-40f7-a5b5-631d3a0adee6 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:23.739778 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dee718-e2e2-40f7-a5b5-631d3a0adee6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:23.740048 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dee718-e2e2-40f7-a5b5-631d3a0adee6 None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:23.740783 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dee718-e2e2-40f7-a5b5-631d3a0adee6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:23.742304 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dee718-e2e2-40f7-a5b5-631d3a0adee6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:23.775089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 588/1173] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:06:23 2022] POST /identity/v3/projects => generated 361 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:06:26.063396 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb260e2-bd1d-45ce-b6ab-7256fef5d563 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.063989 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb260e2-bd1d-45ce-b6ab-7256fef5d563 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.063989 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb260e2-bd1d-45ce-b6ab-7256fef5d563 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.065362 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 586/1174] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:26.071289 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b1e13-f50a-4162-8e60-bbc894ae921e None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.071742 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b1e13-f50a-4162-8e60-bbc894ae921e None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.071847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b1e13-f50a-4162-8e60-bbc894ae921e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.090826 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-164b1e13-f50a-4162-8e60-bbc894ae921e None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:26.092017 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-164b1e13-f50a-4162-8e60-bbc894ae921e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:26.168218 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 589/1175] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:06:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:06:26.193740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb260e2-bd1d-45ce-b6ab-7256fef5d563 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:26.269017 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:26.325752 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb260e2-bd1d-45ce-b6ab-7256fef5d563 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:26.361249 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78061c39-52bf-44d3-a7b4-f3d4eb6d1936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:26.362122 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78061c39-52bf-44d3-a7b4-f3d4eb6d1936 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.362433 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78061c39-52bf-44d3-a7b4-f3d4eb6d1936 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.362709 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78061c39-52bf-44d3-a7b4-f3d4eb6d1936 None admin] PATH_INFO: `/v3/domains/29a81afcdb004692b2b1cfe0a331d4ea` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.364008 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78061c39-52bf-44d3-a7b4-f3d4eb6d1936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=29a81afcdb004692b2b1cfe0a331d4ea)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:26.365456 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78061c39-52bf-44d3-a7b4-f3d4eb6d1936 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:26.367478 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 587/1176] 10.222.0.84 () {56 vars in 1183 bytes} [Thu May 19 20:06:26 2022] GET /identity/v3/domains/29a81afcdb004692b2b1cfe0a331d4ea => generated 267 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:26.376270 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b1e13-f50a-4162-8e60-bbc894ae921e None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:26.429911 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:26.488273 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b1e13-f50a-4162-8e60-bbc894ae921e None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:26.523916 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d259b13-91ed-46c5-a6b5-2c774825407d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:26.524582 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d259b13-91ed-46c5-a6b5-2c774825407d None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.524858 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d259b13-91ed-46c5-a6b5-2c774825407d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.525127 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d259b13-91ed-46c5-a6b5-2c774825407d None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.525829 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d259b13-91ed-46c5-a6b5-2c774825407d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:26.527300 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d259b13-91ed-46c5-a6b5-2c774825407d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:26.549648 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 590/1177] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:06:26 2022] POST /identity/v3/users => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:06:28.783588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d8f3a-0f6e-4e03-b341-f166f08c6395 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:28.784486 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d8f3a-0f6e-4e03-b341-f166f08c6395 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:28.784486 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d8f3a-0f6e-4e03-b341-f166f08c6395 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:28.786742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 588/1178] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:06:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:28.792567 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff9fc6-ae37-4f4a-8a08-d664755424de None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:28.793015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff9fc6-ae37-4f4a-8a08-d664755424de None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:28.793112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff9fc6-ae37-4f4a-8a08-d664755424de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:28.811520 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-dfff9fc6-ae37-4f4a-8a08-d664755424de None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:28.812699 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-dfff9fc6-ae37-4f4a-8a08-d664755424de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:28.885031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 591/1179] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:06:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:06:28.908368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d8f3a-0f6e-4e03-b341-f166f08c6395 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:29.011925 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d8f3a-0f6e-4e03-b341-f166f08c6395 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:29.042538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:29.053958 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157afe7-e8a6-4c7e-8336-25ac13a98de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:29.054830 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157afe7-e8a6-4c7e-8336-25ac13a98de5 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:29.055275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157afe7-e8a6-4c7e-8336-25ac13a98de5 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:29.055558 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157afe7-e8a6-4c7e-8336-25ac13a98de5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:29.066150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157afe7-e8a6-4c7e-8336-25ac13a98de5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-a157afe7-e8a6-4c7e-8336-25ac13a98de5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:29.073643 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:29.074782 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:29.075897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:29.075897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:29.075897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:29.075897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:29.075897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:29.075897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:29.075897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:29.075897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:29.075897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:29.075897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:29.075897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:29.075897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:06:29.075897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 589/1180] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:06:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:29.081117 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff9fc6-ae37-4f4a-8a08-d664755424de None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:29.116317 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:29.201619 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff9fc6-ae37-4f4a-8a08-d664755424de None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:29.236473 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2af44-40b1-4ce7-b865-00735e0944db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:29.237166 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2af44-40b1-4ce7-b865-00735e0944db None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:29.237450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2af44-40b1-4ce7-b865-00735e0944db None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:29.237725 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2af44-40b1-4ce7-b865-00735e0944db None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:29.238503 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2af44-40b1-4ce7-b865-00735e0944db None admin] RBAC: Adding query filter params (name=admin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:29.238815 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2af44-40b1-4ce7-b865-00735e0944db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:29.240199 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2af44-40b1-4ce7-b865-00735e0944db None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:29.248569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 592/1181] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:06:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:29.255415 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157afe7-e8a6-4c7e-8336-25ac13a98de5 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:29.375400 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157afe7-e8a6-4c7e-8336-25ac13a98de5 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:29.411282 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabe685-095e-4a8b-99c7-1af62d1e69fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:29.412148 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabe685-095e-4a8b-99c7-1af62d1e69fa None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:29.412427 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabe685-095e-4a8b-99c7-1af62d1e69fa None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:29.412699 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabe685-095e-4a8b-99c7-1af62d1e69fa None admin] PATH_INFO: `/v3/users/32beb2ab514947f09fc3f458a4b4c679` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:29.427718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabe685-095e-4a8b-99c7-1af62d1e69fa None admin] RBAC: Authorizing `identity:get_user(user_id=32beb2ab514947f09fc3f458a4b4c679)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:29.429180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabe685-095e-4a8b-99c7-1af62d1e69fa None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:29.430886 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 590/1182] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:06:29 2022] GET /identity/v3/users/32beb2ab514947f09fc3f458a4b4c679 => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:29.437578 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2af44-40b1-4ce7-b865-00735e0944db None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:29.539512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:29.545417 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2af44-40b1-4ce7-b865-00735e0944db None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:29.579773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400af30-8777-4210-a492-f2ba48a2a7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:29.580435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400af30-8777-4210-a492-f2ba48a2a7ed None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:29.580715 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400af30-8777-4210-a492-f2ba48a2a7ed None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:29.580989 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400af30-8777-4210-a492-f2ba48a2a7ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:29.653046 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400af30-8777-4210-a492-f2ba48a2a7ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:29.654575 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400af30-8777-4210-a492-f2ba48a2a7ed None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:29.745481 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:29.752870 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 593/1183] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:06:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:06:29.761696 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabe685-095e-4a8b-99c7-1af62d1e69fa None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:29.808947 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:29.865796 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabe685-095e-4a8b-99c7-1af62d1e69fa None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:29.902219 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7a1fa-4747-4f46-b688-b085f1e0200f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:29.903112 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7a1fa-4747-4f46-b688-b085f1e0200f None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:29.903387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7a1fa-4747-4f46-b688-b085f1e0200f None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:29.903662 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7a1fa-4747-4f46-b688-b085f1e0200f None admin] PATH_INFO: `/v3/projects/103be881f8fd49deb4074a7d43932343` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:29.904961 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7a1fa-4747-4f46-b688-b085f1e0200f None admin] RBAC: Authorizing `identity:get_project(project_id=103be881f8fd49deb4074a7d43932343)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:29.906450 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7a1fa-4747-4f46-b688-b085f1e0200f None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:29.908146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 591/1184] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:06:29 2022] GET /identity/v3/projects/103be881f8fd49deb4074a7d43932343 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:29.916010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400af30-8777-4210-a492-f2ba48a2a7ed None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:30.034097 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400af30-8777-4210-a492-f2ba48a2a7ed None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:30.072986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f6ca6-bd27-4540-a609-78cca1204e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:30.073570 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f6ca6-bd27-4540-a609-78cca1204e1a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:30.073849 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f6ca6-bd27-4540-a609-78cca1204e1a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:30.074124 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f6ca6-bd27-4540-a609-78cca1204e1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:30.074996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f6ca6-bd27-4540-a609-78cca1204e1a None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=103be881f8fd49deb4074a7d43932343, user.id=32beb2ab514947f09fc3f458a4b4c679) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:30.075307 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f6ca6-bd27-4540-a609-78cca1204e1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:30.076756 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f6ca6-bd27-4540-a609-78cca1204e1a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:30.082708 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 594/1185] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:06:29 2022] GET /identity/v3/role_assignments?user.id=32beb2ab514947f09fc3f458a4b4c679&scope.project.id=103be881f8fd49deb4074a7d43932343&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:32.338573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42675645-ee2e-4788-afbd-715633ada06b None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.339444 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42675645-ee2e-4788-afbd-715633ada06b None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.339444 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42675645-ee2e-4788-afbd-715633ada06b None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.341705 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 592/1186] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:06:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:32.348543 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22661e8e-6bc7-4d09-af55-0874826218e9 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.349010 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22661e8e-6bc7-4d09-af55-0874826218e9 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.349119 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22661e8e-6bc7-4d09-af55-0874826218e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.369255 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-22661e8e-6bc7-4d09-af55-0874826218e9 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:32.370483 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-22661e8e-6bc7-4d09-af55-0874826218e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:32.433205 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:32.448094 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 595/1187] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:06:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:06:32.472305 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42675645-ee2e-4788-afbd-715633ada06b None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:32.583039 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42675645-ee2e-4788-afbd-715633ada06b None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:32.597057 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:32.620974 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f031c3c-edb2-4d6b-b356-c8dec9ee332b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:32.621764 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f031c3c-edb2-4d6b-b356-c8dec9ee332b None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.622038 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f031c3c-edb2-4d6b-b356-c8dec9ee332b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.622301 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f031c3c-edb2-4d6b-b356-c8dec9ee332b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.633285 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f031c3c-edb2-4d6b-b356-c8dec9ee332b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-7f031c3c-edb2-4d6b-b356-c8dec9ee332b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:32.640798 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:32.641934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:32.642926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:32.642926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:32.642926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:32.642926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:32.642926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:32.642926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:32.642926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:32.642926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:32.642926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:32.642926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:32.642926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:32.642926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:06:32.642926 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 593/1188] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:06:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:32.648534 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22661e8e-6bc7-4d09-af55-0874826218e9 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:32.759833 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22661e8e-6bc7-4d09-af55-0874826218e9 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:32.799504 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb13515-dacc-46a5-9b5d-802a24b4ba05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:32.800181 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb13515-dacc-46a5-9b5d-802a24b4ba05 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.800464 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb13515-dacc-46a5-9b5d-802a24b4ba05 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.800747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb13515-dacc-46a5-9b5d-802a24b4ba05 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.801505 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb13515-dacc-46a5-9b5d-802a24b4ba05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:32.801814 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb13515-dacc-46a5-9b5d-802a24b4ba05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:32.803240 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb13515-dacc-46a5-9b5d-802a24b4ba05 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:32.811827 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 596/1189] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:06:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:32.818801 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f031c3c-edb2-4d6b-b356-c8dec9ee332b None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:32.930659 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f031c3c-edb2-4d6b-b356-c8dec9ee332b None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:32.967519 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec5af8-3d4d-45e4-aaff-deb417fbacc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:32.968406 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec5af8-3d4d-45e4-aaff-deb417fbacc8 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.968690 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec5af8-3d4d-45e4-aaff-deb417fbacc8 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.969275 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec5af8-3d4d-45e4-aaff-deb417fbacc8 None admin] PATH_INFO: `/v3/users/32beb2ab514947f09fc3f458a4b4c679` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.970656 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec5af8-3d4d-45e4-aaff-deb417fbacc8 None admin] RBAC: Authorizing `identity:get_user(user_id=32beb2ab514947f09fc3f458a4b4c679)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:32.972100 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec5af8-3d4d-45e4-aaff-deb417fbacc8 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:32.973745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 594/1190] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:06:32 2022] GET /identity/v3/users/32beb2ab514947f09fc3f458a4b4c679 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:32.980056 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb13515-dacc-46a5-9b5d-802a24b4ba05 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:33.040415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:33.093159 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb13515-dacc-46a5-9b5d-802a24b4ba05 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:33.129502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ccb45-2ab8-48c8-948a-ff6712638464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:33.130394 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ccb45-2ab8-48c8-948a-ff6712638464 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:33.130675 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ccb45-2ab8-48c8-948a-ff6712638464 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:33.130955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ccb45-2ab8-48c8-948a-ff6712638464 None admin] PATH_INFO: `/v3/projects/103be881f8fd49deb4074a7d43932343` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:33.132272 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ccb45-2ab8-48c8-948a-ff6712638464 None admin] RBAC: Authorizing `identity:get_project(project_id=103be881f8fd49deb4074a7d43932343)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:33.133910 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ccb45-2ab8-48c8-948a-ff6712638464 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:33.135654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 597/1191] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:06:32 2022] GET /identity/v3/projects/103be881f8fd49deb4074a7d43932343 => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:33.142416 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec5af8-3d4d-45e4-aaff-deb417fbacc8 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:33.194187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:33.258219 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec5af8-3d4d-45e4-aaff-deb417fbacc8 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:33.296579 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a5b26-a41c-4508-b494-72b134bb961e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:33.297469 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a5b26-a41c-4508-b494-72b134bb961e None admin] REQUEST_METHOD: `PUT` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:33.297753 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a5b26-a41c-4508-b494-72b134bb961e None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:33.298037 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a5b26-a41c-4508-b494-72b134bb961e None admin] PATH_INFO: `/v3/projects/103be881f8fd49deb4074a7d43932343/users/32beb2ab514947f09fc3f458a4b4c679/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:33.300321 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a5b26-a41c-4508-b494-72b134bb961e None admin] RBAC: Authorizing `identity:create_grant(project_id=103be881f8fd49deb4074a7d43932343, user_id=32beb2ab514947f09fc3f458a4b4c679, role_id=b34e4ad4fe7b4a60bd2fc669e2b31a66)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:33.301762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a5b26-a41c-4508-b494-72b134bb961e None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:33.318630 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 595/1192] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:06:33 2022] PUT /identity/v3/projects/103be881f8fd49deb4074a7d43932343/users/32beb2ab514947f09fc3f458a4b4c679/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:06:35.532866 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4c1a0-d303-4d87-8f15-ef6d4f538611 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:35.533459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4c1a0-d303-4d87-8f15-ef6d4f538611 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:35.533459 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4c1a0-d303-4d87-8f15-ef6d4f538611 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:35.534881 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 598/1193] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:35.540695 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e396f-6893-46a4-a328-ac7c0a8d1981 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:35.541166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e396f-6893-46a4-a328-ac7c0a8d1981 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:35.541261 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e396f-6893-46a4-a328-ac7c0a8d1981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:35.564397 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-302e396f-6893-46a4-a328-ac7c0a8d1981 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:35.565593 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-302e396f-6893-46a4-a328-ac7c0a8d1981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:35.637852 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 596/1194] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:06:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:06:35.662014 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4c1a0-d303-4d87-8f15-ef6d4f538611 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:35.773527 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4c1a0-d303-4d87-8f15-ef6d4f538611 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:35.793665 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:35.811020 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a977777-2262-4184-bf2b-b024ee33af0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:35.811811 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a977777-2262-4184-bf2b-b024ee33af0e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:35.812092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a977777-2262-4184-bf2b-b024ee33af0e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:35.812365 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a977777-2262-4184-bf2b-b024ee33af0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:35.825025 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a977777-2262-4184-bf2b-b024ee33af0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-4a977777-2262-4184-bf2b-b024ee33af0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:35.832571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:35.833686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:35.834882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:35.834882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:35.834882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:35.834882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:35.834882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:35.834882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:35.834882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:35.834882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:35.834882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:35.834882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:35.834882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:35.834882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:06:35.834882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 599/1195] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:06:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:35.840722 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e396f-6893-46a4-a328-ac7c0a8d1981 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:35.882758 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:35.955555 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e396f-6893-46a4-a328-ac7c0a8d1981 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:35.991382 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db03d1-6ae3-4839-814d-d45004dbc980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:35.992086 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db03d1-6ae3-4839-814d-d45004dbc980 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:35.992375 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db03d1-6ae3-4839-814d-d45004dbc980 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:35.992633 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db03d1-6ae3-4839-814d-d45004dbc980 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:35.993376 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db03d1-6ae3-4839-814d-d45004dbc980 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:35.993683 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db03d1-6ae3-4839-814d-d45004dbc980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:35.995085 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db03d1-6ae3-4839-814d-d45004dbc980 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:36.003955 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 597/1196] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:06:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:36.010589 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a977777-2262-4184-bf2b-b024ee33af0e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:36.133040 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a977777-2262-4184-bf2b-b024ee33af0e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:36.175555 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618c6c0-fdff-43a5-8071-6ab4b8ac0c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:36.176463 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618c6c0-fdff-43a5-8071-6ab4b8ac0c3a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:36.176747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618c6c0-fdff-43a5-8071-6ab4b8ac0c3a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:36.177020 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618c6c0-fdff-43a5-8071-6ab4b8ac0c3a None admin] PATH_INFO: `/v3/users/32beb2ab514947f09fc3f458a4b4c679` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:36.178310 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618c6c0-fdff-43a5-8071-6ab4b8ac0c3a None admin] RBAC: Authorizing `identity:get_user(user_id=32beb2ab514947f09fc3f458a4b4c679)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:36.179780 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618c6c0-fdff-43a5-8071-6ab4b8ac0c3a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:36.181417 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 600/1197] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:06:36 2022] GET /identity/v3/users/32beb2ab514947f09fc3f458a4b4c679 => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:36.188222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db03d1-6ae3-4839-814d-d45004dbc980 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:36.290654 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:36.299988 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db03d1-6ae3-4839-814d-d45004dbc980 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:36.337484 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64221012-8fff-47b6-b74b-97a4bb1ffeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:36.338132 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64221012-8fff-47b6-b74b-97a4bb1ffeab None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:36.338444 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64221012-8fff-47b6-b74b-97a4bb1ffeab None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:36.338717 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64221012-8fff-47b6-b74b-97a4bb1ffeab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:36.408441 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64221012-8fff-47b6-b74b-97a4bb1ffeab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:36.409823 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64221012-8fff-47b6-b74b-97a4bb1ffeab None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:36.511620 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:36.522304 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 598/1198] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:06:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:06:36.528730 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618c6c0-fdff-43a5-8071-6ab4b8ac0c3a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:36.583579 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:36.637457 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618c6c0-fdff-43a5-8071-6ab4b8ac0c3a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:36.675811 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307eac4f-1205-4be8-b0ae-0231c383e407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:36.676610 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307eac4f-1205-4be8-b0ae-0231c383e407 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:36.676891 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307eac4f-1205-4be8-b0ae-0231c383e407 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:36.677168 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307eac4f-1205-4be8-b0ae-0231c383e407 None admin] PATH_INFO: `/v3/projects/103be881f8fd49deb4074a7d43932343` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:36.678504 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307eac4f-1205-4be8-b0ae-0231c383e407 None admin] RBAC: Authorizing `identity:get_project(project_id=103be881f8fd49deb4074a7d43932343)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:36.680616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307eac4f-1205-4be8-b0ae-0231c383e407 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:36.682368 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 601/1199] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:06:36 2022] GET /identity/v3/projects/103be881f8fd49deb4074a7d43932343 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:36.692222 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64221012-8fff-47b6-b74b-97a4bb1ffeab None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:36.795912 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64221012-8fff-47b6-b74b-97a4bb1ffeab None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:36.839207 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124441e-43b8-480f-8aba-ea25f05fa9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:36.839863 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124441e-43b8-480f-8aba-ea25f05fa9e4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:36.840144 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124441e-43b8-480f-8aba-ea25f05fa9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:36.840413 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124441e-43b8-480f-8aba-ea25f05fa9e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:36.841250 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124441e-43b8-480f-8aba-ea25f05fa9e4 None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=103be881f8fd49deb4074a7d43932343, user.id=32beb2ab514947f09fc3f458a4b4c679) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:36.841566 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124441e-43b8-480f-8aba-ea25f05fa9e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:36.843034 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124441e-43b8-480f-8aba-ea25f05fa9e4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:36.849390 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 599/1200] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:06:36 2022] GET /identity/v3/role_assignments?user.id=32beb2ab514947f09fc3f458a4b4c679&scope.project.id=103be881f8fd49deb4074a7d43932343&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:46.670461 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=42 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:46.846625 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=72 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:35.712885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318586f6-1857-4085-88f1-c9f010bc9446 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:35.713504 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318586f6-1857-4085-88f1-c9f010bc9446 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:35.713504 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318586f6-1857-4085-88f1-c9f010bc9446 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:35.715027 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 602/1201] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:35.720715 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33a249-ab31-4067-afcc-67e14bd6cbe1 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:35.721184 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33a249-ab31-4067-afcc-67e14bd6cbe1 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:35.721292 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33a249-ab31-4067-afcc-67e14bd6cbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:35.738720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-6d33a249-ab31-4067-afcc-67e14bd6cbe1 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:35.739896 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-6d33a249-ab31-4067-afcc-67e14bd6cbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:35.806930 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 600/1202] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:07:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:07:35.831460 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318586f6-1857-4085-88f1-c9f010bc9446 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:35.943566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318586f6-1857-4085-88f1-c9f010bc9446 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:35.985165 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825b9a9-d385-427a-ba54-0dec4f5daf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:35.986059 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825b9a9-d385-427a-ba54-0dec4f5daf15 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:35.986374 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825b9a9-d385-427a-ba54-0dec4f5daf15 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:35.986646 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825b9a9-d385-427a-ba54-0dec4f5daf15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:35.999977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825b9a9-d385-427a-ba54-0dec4f5daf15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:36.001794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825b9a9-d385-427a-ba54-0dec4f5daf15 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:36.021207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-1825b9a9-d385-427a-ba54-0dec4f5daf15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:36.021207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:36.021207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:36.021207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:36.021207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:36.021207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:36.021207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:36.021207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:36.021207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:36.021207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:36.021207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:36.021207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:36.021207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:36.021207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:36.021207 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:36.022512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:36.024029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:36.024029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:36.024029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:36.024029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:07:36.024029 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 603/1203] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:36.029334 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33a249-ab31-4067-afcc-67e14bd6cbe1 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:36.131546 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33a249-ab31-4067-afcc-67e14bd6cbe1 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:36.160128 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:36.169668 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5de25-0841-432b-b71d-4ab36d71b6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:36.170389 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5de25-0841-432b-b71d-4ab36d71b6b4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:36.170666 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5de25-0841-432b-b71d-4ab36d71b6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:36.170938 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5de25-0841-432b-b71d-4ab36d71b6b4 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:36.171659 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5de25-0841-432b-b71d-4ab36d71b6b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:36.171961 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5de25-0841-432b-b71d-4ab36d71b6b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:36.173335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5de25-0841-432b-b71d-4ab36d71b6b4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:36.187419 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 601/1204] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:36.193559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825b9a9-d385-427a-ba54-0dec4f5daf15 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:36.270364 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:36.335563 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825b9a9-d385-427a-ba54-0dec4f5daf15 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:36.377277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e5898-0686-4231-84b4-bfee894fe331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:36.377961 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e5898-0686-4231-84b4-bfee894fe331 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:36.378245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e5898-0686-4231-84b4-bfee894fe331 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:36.378559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e5898-0686-4231-84b4-bfee894fe331 None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:36.379388 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e5898-0686-4231-84b4-bfee894fe331 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:36.380807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e5898-0686-4231-84b4-bfee894fe331 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:36.414620 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 604/1205] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:07:36 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:07:38.846884 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4f3f5-95e2-4901-869e-bf86ed3321af None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:38.847416 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4f3f5-95e2-4901-869e-bf86ed3321af None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:38.847416 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4f3f5-95e2-4901-869e-bf86ed3321af None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:38.848870 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 602/1206] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:38.855385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f06b77-225a-498f-9991-0ff7c72712ed None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:38.855890 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f06b77-225a-498f-9991-0ff7c72712ed None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:38.855999 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f06b77-225a-498f-9991-0ff7c72712ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:38.874949 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-c0f06b77-225a-498f-9991-0ff7c72712ed None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:38.876540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-c0f06b77-225a-498f-9991-0ff7c72712ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:38.964228 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 605/1207] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:07:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:07:38.991827 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4f3f5-95e2-4901-869e-bf86ed3321af None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.115668 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4f3f5-95e2-4901-869e-bf86ed3321af None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:39.156218 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18022c9d-2383-4413-b955-c5dd0700c816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.157570 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18022c9d-2383-4413-b955-c5dd0700c816 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.157980 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18022c9d-2383-4413-b955-c5dd0700c816 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.158284 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18022c9d-2383-4413-b955-c5dd0700c816 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:39.173135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18022c9d-2383-4413-b955-c5dd0700c816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-18022c9d-2383-4413-b955-c5dd0700c816 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:39.181064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:39.182192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:39.183686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:39.183686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:39.183686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:39.183686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:39.183686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:39.183686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:39.183686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:39.183686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:39.183686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:39.183686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:39.183686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:39.183686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:07:39.183686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 603/1208] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:07:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:39.188673 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f06b77-225a-498f-9991-0ff7c72712ed None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.234449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:39.312694 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f06b77-225a-498f-9991-0ff7c72712ed None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:39.353089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5e1d2-202a-41eb-839e-b281b11099f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.353779 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5e1d2-202a-41eb-839e-b281b11099f1 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.354083 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5e1d2-202a-41eb-839e-b281b11099f1 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.354368 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5e1d2-202a-41eb-839e-b281b11099f1 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:39.355126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5e1d2-202a-41eb-839e-b281b11099f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:39.355430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5e1d2-202a-41eb-839e-b281b11099f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:39.356855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5e1d2-202a-41eb-839e-b281b11099f1 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:39.366126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 606/1209] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:07:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:39.373741 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18022c9d-2383-4413-b955-c5dd0700c816 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.430853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:39.499233 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18022c9d-2383-4413-b955-c5dd0700c816 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:39.538734 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90feff71-a847-40e0-93d0-e55ba7ee97a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.539614 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90feff71-a847-40e0-93d0-e55ba7ee97a6 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.539893 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90feff71-a847-40e0-93d0-e55ba7ee97a6 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.540162 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90feff71-a847-40e0-93d0-e55ba7ee97a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:39.554951 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90feff71-a847-40e0-93d0-e55ba7ee97a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:39.556759 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90feff71-a847-40e0-93d0-e55ba7ee97a6 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:39.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-90feff71-a847-40e0-93d0-e55ba7ee97a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:39.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:39.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:39.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:39.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:39.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:39.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:39.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:39.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:39.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:39.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:39.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:39.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:39.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:39.580655 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:39.581999 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:39.583030 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:39.583030 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:39.583030 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:39.583030 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:07:39.583030 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 604/1210] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:39.588740 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5e1d2-202a-41eb-839e-b281b11099f1 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.707493 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:39.710654 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5e1d2-202a-41eb-839e-b281b11099f1 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:39.752124 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd7a1d-1264-4292-a3d0-72521f48c46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.752886 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd7a1d-1264-4292-a3d0-72521f48c46d None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.753089 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd7a1d-1264-4292-a3d0-72521f48c46d None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.753377 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd7a1d-1264-4292-a3d0-72521f48c46d None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:39.754101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd7a1d-1264-4292-a3d0-72521f48c46d None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:39.754449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd7a1d-1264-4292-a3d0-72521f48c46d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:39.755854 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd7a1d-1264-4292-a3d0-72521f48c46d None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:39.769339 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 607/1211] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:39.775329 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90feff71-a847-40e0-93d0-e55ba7ee97a6 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.884746 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90feff71-a847-40e0-93d0-e55ba7ee97a6 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:39.886445 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:39.921273 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677728bb-d018-4b8c-87a6-00dd212be77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.922166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677728bb-d018-4b8c-87a6-00dd212be77d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.922486 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677728bb-d018-4b8c-87a6-00dd212be77d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.922765 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677728bb-d018-4b8c-87a6-00dd212be77d None admin] PATH_INFO: `/v3/users/placement` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:39.931265 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677728bb-d018-4b8c-87a6-00dd212be77d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:39.932766 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677728bb-d018-4b8c-87a6-00dd212be77d None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-677728bb-d018-4b8c-87a6-00dd212be77d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:39.948026 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:39.949135 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:39.950504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:39.950504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:39.950504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:39.950504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:39.950504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:39.950504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:39.950504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:39.950504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:39.950504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:07:39.950504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:07:39.950504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 605/1212] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:39.955512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd7a1d-1264-4292-a3d0-72521f48c46d None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:40.069380 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd7a1d-1264-4292-a3d0-72521f48c46d None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:40.110509 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71793596-4aa7-45fd-a344-53df6646ad8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:40.111561 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71793596-4aa7-45fd-a344-53df6646ad8b None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:40.112047 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71793596-4aa7-45fd-a344-53df6646ad8b None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:40.112536 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71793596-4aa7-45fd-a344-53df6646ad8b None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:40.114379 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71793596-4aa7-45fd-a344-53df6646ad8b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:40.114944 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71793596-4aa7-45fd-a344-53df6646ad8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:40.116643 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71793596-4aa7-45fd-a344-53df6646ad8b None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:40.137330 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 608/1213] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 20:07:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:40.143171 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677728bb-d018-4b8c-87a6-00dd212be77d None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:40.261960 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677728bb-d018-4b8c-87a6-00dd212be77d None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:40.303866 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bff0f-54f2-4f8e-917e-0b4d7c3b58f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:40.304674 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bff0f-54f2-4f8e-917e-0b4d7c3b58f5 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:40.304897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bff0f-54f2-4f8e-917e-0b4d7c3b58f5 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:40.305813 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bff0f-54f2-4f8e-917e-0b4d7c3b58f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:40.337558 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:40.383924 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bff0f-54f2-4f8e-917e-0b4d7c3b58f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:40.385386 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bff0f-54f2-4f8e-917e-0b4d7c3b58f5 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:40.506713 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 606/1214] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:07:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:07:40.513112 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71793596-4aa7-45fd-a344-53df6646ad8b None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:40.532841 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:40.646638 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71793596-4aa7-45fd-a344-53df6646ad8b None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:40.689634 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532ca0e-ab56-4c21-874e-e21b4b13af18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:40.690482 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532ca0e-ab56-4c21-874e-e21b4b13af18 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:40.690789 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532ca0e-ab56-4c21-874e-e21b4b13af18 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:40.691039 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532ca0e-ab56-4c21-874e-e21b4b13af18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:40.704406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6532ca0e-ab56-4c21-874e-e21b4b13af18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:40.705885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6532ca0e-ab56-4c21-874e-e21b4b13af18 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:40.724516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-6532ca0e-ab56-4c21-874e-e21b4b13af18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:40.724516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:40.724516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:40.724516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:40.724516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:40.724516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:40.724516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:40.724516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:40.724516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:40.724516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:40.724516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:40.724516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:40.724516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:40.724516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:40.724516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:40.725810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:40.726983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:40.726983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:40.726983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:40.726983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:07:40.726983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 609/1215] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:40.732347 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bff0f-54f2-4f8e-917e-0b4d7c3b58f5 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:40.805082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:40.853311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bff0f-54f2-4f8e-917e-0b4d7c3b58f5 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:40.891502 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1a744-c529-43a8-adf5-73af7fc2e683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:40.892180 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1a744-c529-43a8-adf5-73af7fc2e683 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:40.892574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1a744-c529-43a8-adf5-73af7fc2e683 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:40.892914 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1a744-c529-43a8-adf5-73af7fc2e683 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:40.893805 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1a744-c529-43a8-adf5-73af7fc2e683 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:40.894207 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1a744-c529-43a8-adf5-73af7fc2e683 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:40.895951 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1a744-c529-43a8-adf5-73af7fc2e683 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:40.909763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 607/1216] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:40.915831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532ca0e-ab56-4c21-874e-e21b4b13af18 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:41.006904 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:41.048309 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532ca0e-ab56-4c21-874e-e21b4b13af18 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:41.097905 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c27f5c-16cc-4382-a496-c57502f5d58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:41.098834 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c27f5c-16cc-4382-a496-c57502f5d58c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:41.099115 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c27f5c-16cc-4382-a496-c57502f5d58c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:41.099387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c27f5c-16cc-4382-a496-c57502f5d58c None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:41.107644 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c27f5c-16cc-4382-a496-c57502f5d58c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:41.109202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c27f5c-16cc-4382-a496-c57502f5d58c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:41.127977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-05c27f5c-16cc-4382-a496-c57502f5d58c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:41.127977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:41.127977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:41.127977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:41.127977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:41.127977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:41.127977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:41.127977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:41.127977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:41.127977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:41.127977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:41.127977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:41.127977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:41.127977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:41.127977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:41.129179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:41.130392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:41.130392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:41.130392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:41.130392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:41.130392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:41.130392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:41.130392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:41.130392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:07:41.130392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 610/1217] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:07:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:41.135299 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1a744-c529-43a8-adf5-73af7fc2e683 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:41.253150 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1a744-c529-43a8-adf5-73af7fc2e683 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:41.280118 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:41.293609 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca10e54-4cba-46b9-931f-dbbf02c44250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:41.294306 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca10e54-4cba-46b9-931f-dbbf02c44250 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:41.294594 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca10e54-4cba-46b9-931f-dbbf02c44250 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:41.294909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca10e54-4cba-46b9-931f-dbbf02c44250 None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:41.295667 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca10e54-4cba-46b9-931f-dbbf02c44250 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:41.295982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca10e54-4cba-46b9-931f-dbbf02c44250 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:41.297417 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca10e54-4cba-46b9-931f-dbbf02c44250 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:41.310523 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 608/1218] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:07:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:41.316539 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c27f5c-16cc-4382-a496-c57502f5d58c None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:41.426228 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c27f5c-16cc-4382-a496-c57502f5d58c None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:41.457165 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:41.463747 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6e208-3772-45df-8624-b63b3bfc71e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:41.464441 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6e208-3772-45df-8624-b63b3bfc71e7 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:41.464699 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6e208-3772-45df-8624-b63b3bfc71e7 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:41.465216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6e208-3772-45df-8624-b63b3bfc71e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:41.467724 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6e208-3772-45df-8624-b63b3bfc71e7 None admin] RBAC: Adding query filter params (role.id=bdaf8732b34c43a38465efde7e18a060, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=28953289ea704853b1c0c7049915633d) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:41.467724 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6e208-3772-45df-8624-b63b3bfc71e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:41.467945 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6e208-3772-45df-8624-b63b3bfc71e7 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:41.476709 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 611/1219] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:07:41 2022] GET /identity/v3/role_assignments?user.id=28953289ea704853b1c0c7049915633d&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=bdaf8732b34c43a38465efde7e18a060 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:43.739046 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b462d-83af-4d68-9bc4-d2be5640fd2f None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:43.739046 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b462d-83af-4d68-9bc4-d2be5640fd2f None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:43.739046 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b462d-83af-4d68-9bc4-d2be5640fd2f None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:43.740255 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 609/1220] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:43.747808 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40253b66-5f30-48b1-bdc1-a0f4c5df6068 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:43.747808 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40253b66-5f30-48b1-bdc1-a0f4c5df6068 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:43.747808 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40253b66-5f30-48b1-bdc1-a0f4c5df6068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:43.765577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-40253b66-5f30-48b1-bdc1-a0f4c5df6068 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:43.766941 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-40253b66-5f30-48b1-bdc1-a0f4c5df6068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:43.852274 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 612/1221] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:07:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:07:43.875515 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b462d-83af-4d68-9bc4-d2be5640fd2f None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:43.993328 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b462d-83af-4d68-9bc4-d2be5640fd2f None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:44.031074 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bf739-94c2-4236-b1c6-88d8a2a2f7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:44.031884 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bf739-94c2-4236-b1c6-88d8a2a2f7a0 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.032167 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bf739-94c2-4236-b1c6-88d8a2a2f7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.032465 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bf739-94c2-4236-b1c6-88d8a2a2f7a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.043220 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bf739-94c2-4236-b1c6-88d8a2a2f7a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-ad0bf739-94c2-4236-b1c6-88d8a2a2f7a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:44.052156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:44.053613 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:44.055103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:44.055103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:44.055103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:44.055103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:44.055103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:44.055103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:44.055103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:44.055103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:44.055103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:44.055103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:44.055103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:44.055103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:07:44.055103 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 610/1222] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:07:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:44.059563 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40253b66-5f30-48b1-bdc1-a0f4c5df6068 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:44.165015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40253b66-5f30-48b1-bdc1-a0f4c5df6068 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:44.190972 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:44.203701 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2737f1-468e-44b2-9efc-e894b9736b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:44.204775 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2737f1-468e-44b2-9efc-e894b9736b93 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.204965 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2737f1-468e-44b2-9efc-e894b9736b93 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.205201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2737f1-468e-44b2-9efc-e894b9736b93 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.206095 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2737f1-468e-44b2-9efc-e894b9736b93 None admin] RBAC: Adding query filter params (name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:44.206428 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2737f1-468e-44b2-9efc-e894b9736b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:44.207828 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2737f1-468e-44b2-9efc-e894b9736b93 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:44.217266 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 613/1223] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:07:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:44.224162 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bf739-94c2-4236-b1c6-88d8a2a2f7a0 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:44.273739 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:44.335146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bf739-94c2-4236-b1c6-88d8a2a2f7a0 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:44.373598 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de2ba3-e2b9-43bc-9922-248caf7ad616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:44.374841 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de2ba3-e2b9-43bc-9922-248caf7ad616 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.375122 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de2ba3-e2b9-43bc-9922-248caf7ad616 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.375394 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de2ba3-e2b9-43bc-9922-248caf7ad616 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.389299 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de2ba3-e2b9-43bc-9922-248caf7ad616 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:44.390974 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de2ba3-e2b9-43bc-9922-248caf7ad616 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:44.410221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-45de2ba3-e2b9-43bc-9922-248caf7ad616 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:44.410221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:44.410221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:44.410221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:44.410221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:44.410221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:44.410221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:44.410221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:44.410221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:44.410221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:44.410221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:44.410221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:44.410221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:44.410221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:44.410221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:44.411520 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:44.412738 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:44.412738 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:44.412738 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:44.412738 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:07:44.417601 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2737f1-468e-44b2-9efc-e894b9736b93 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:44.417990 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 611/1224] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:44.523133 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2737f1-468e-44b2-9efc-e894b9736b93 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:44.559839 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4cafb-362d-4c72-82c5-488a6df8ac9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:44.560514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4cafb-362d-4c72-82c5-488a6df8ac9e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.560794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4cafb-362d-4c72-82c5-488a6df8ac9e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.561068 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4cafb-362d-4c72-82c5-488a6df8ac9e None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.561888 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4cafb-362d-4c72-82c5-488a6df8ac9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:44.562202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4cafb-362d-4c72-82c5-488a6df8ac9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:44.563694 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4cafb-362d-4c72-82c5-488a6df8ac9e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:44.579592 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 614/1225] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:44.585602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de2ba3-e2b9-43bc-9922-248caf7ad616 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:44.693216 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:44.700748 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de2ba3-e2b9-43bc-9922-248caf7ad616 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:44.744718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe06b6-705a-45dc-9a22-73b593a36f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:44.745588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe06b6-705a-45dc-9a22-73b593a36f8a None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.745867 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe06b6-705a-45dc-9a22-73b593a36f8a None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.746136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe06b6-705a-45dc-9a22-73b593a36f8a None admin] PATH_INFO: `/v3/users/placement` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.754036 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe06b6-705a-45dc-9a22-73b593a36f8a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:44.755745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe06b6-705a-45dc-9a22-73b593a36f8a None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-f0fe06b6-705a-45dc-9a22-73b593a36f8a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:44.771585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:44.773586 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:44.774986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:44.774986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:44.774986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:44.774986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:44.774986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:44.774986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:44.774986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:44.774986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:44.774986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:07:44.774986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:07:44.774986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 612/1226] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:44.779034 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4cafb-362d-4c72-82c5-488a6df8ac9e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:44.834162 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:44.894549 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4cafb-362d-4c72-82c5-488a6df8ac9e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:44.933458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa404eb8-136c-4f63-957f-0c7e2483c61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:44.934142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa404eb8-136c-4f63-957f-0c7e2483c61a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.934454 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa404eb8-136c-4f63-957f-0c7e2483c61a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.934734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa404eb8-136c-4f63-957f-0c7e2483c61a None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.935581 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa404eb8-136c-4f63-957f-0c7e2483c61a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:44.935894 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa404eb8-136c-4f63-957f-0c7e2483c61a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:44.937341 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa404eb8-136c-4f63-957f-0c7e2483c61a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:44.956211 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 615/1227] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 20:07:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:44.962131 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe06b6-705a-45dc-9a22-73b593a36f8a None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:45.074729 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe06b6-705a-45dc-9a22-73b593a36f8a None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:45.113441 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47006d5d-0f7d-427c-b2c2-e166d1d7c3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:45.114278 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47006d5d-0f7d-427c-b2c2-e166d1d7c3db None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.114570 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47006d5d-0f7d-427c-b2c2-e166d1d7c3db None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.114850 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47006d5d-0f7d-427c-b2c2-e166d1d7c3db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.126957 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47006d5d-0f7d-427c-b2c2-e166d1d7c3db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:45.128663 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47006d5d-0f7d-427c-b2c2-e166d1d7c3db None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:45.147052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-47006d5d-0f7d-427c-b2c2-e166d1d7c3db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:45.147052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:45.147052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:45.147052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:45.147052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:45.147052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:45.147052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:45.147052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:45.147052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:45.147052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:45.147052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:45.147052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:45.147052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:45.147052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:45.147052 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:45.148365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:45.154233 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:45.154233 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:45.154233 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:45.154233 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:07:45.154233 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 613/1228] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:45.154652 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa404eb8-136c-4f63-957f-0c7e2483c61a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:45.262868 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa404eb8-136c-4f63-957f-0c7e2483c61a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:45.265183 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:45.299760 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84adbcb-cf5c-457a-8d98-1466632e0303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:45.300436 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84adbcb-cf5c-457a-8d98-1466632e0303 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.300741 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84adbcb-cf5c-457a-8d98-1466632e0303 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.300995 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84adbcb-cf5c-457a-8d98-1466632e0303 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.301718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84adbcb-cf5c-457a-8d98-1466632e0303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:45.302050 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84adbcb-cf5c-457a-8d98-1466632e0303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:45.303447 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84adbcb-cf5c-457a-8d98-1466632e0303 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:45.315124 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 616/1229] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:45.321141 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47006d5d-0f7d-427c-b2c2-e166d1d7c3db None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:45.330629 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:45.436087 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47006d5d-0f7d-427c-b2c2-e166d1d7c3db None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:45.471859 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682f2b0-b1e1-4ceb-ab16-ab644c39f1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:45.472747 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682f2b0-b1e1-4ceb-ab16-ab644c39f1c8 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.473030 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682f2b0-b1e1-4ceb-ab16-ab644c39f1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.473327 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682f2b0-b1e1-4ceb-ab16-ab644c39f1c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.481130 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682f2b0-b1e1-4ceb-ab16-ab644c39f1c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:45.482716 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682f2b0-b1e1-4ceb-ab16-ab644c39f1c8 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:45.497745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-0682f2b0-b1e1-4ceb-ab16-ab644c39f1c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:45.497745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:45.497745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:45.497745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:45.497745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:45.497745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:45.497745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:45.497745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:45.497745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:45.497745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:45.497745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:45.497745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:45.497745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:45.497745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:45.497745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:45.498982 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:45.500289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:45.500289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:45.500289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:45.500289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:45.500289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:45.500289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:45.500289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:45.500289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:07:45.500289 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 614/1230] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:07:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:45.504974 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84adbcb-cf5c-457a-8d98-1466632e0303 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:45.611985 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84adbcb-cf5c-457a-8d98-1466632e0303 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:45.647863 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c45d5-faa5-41bc-9e26-21e35ead68d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:45.648591 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c45d5-faa5-41bc-9e26-21e35ead68d6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.648898 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c45d5-faa5-41bc-9e26-21e35ead68d6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.649170 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c45d5-faa5-41bc-9e26-21e35ead68d6 None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.650030 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c45d5-faa5-41bc-9e26-21e35ead68d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:45.650371 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c45d5-faa5-41bc-9e26-21e35ead68d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:45.651963 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c45d5-faa5-41bc-9e26-21e35ead68d6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:45.665412 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 617/1231] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:07:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:45.672235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682f2b0-b1e1-4ceb-ab16-ab644c39f1c8 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:45.749925 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:45.790442 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682f2b0-b1e1-4ceb-ab16-ab644c39f1c8 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:45.825269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4fa54-8e07-4e85-9e0e-6c7999b16f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:45.826093 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4fa54-8e07-4e85-9e0e-6c7999b16f92 None admin] REQUEST_METHOD: `PUT` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.826408 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4fa54-8e07-4e85-9e0e-6c7999b16f92 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.826686 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4fa54-8e07-4e85-9e0e-6c7999b16f92 None admin] PATH_INFO: `/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/28953289ea704853b1c0c7049915633d/roles/bdaf8732b34c43a38465efde7e18a060` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.841761 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4fa54-8e07-4e85-9e0e-6c7999b16f92 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba22fdc274b74521847e5fceb2d307b1, user_id=28953289ea704853b1c0c7049915633d, role_id=bdaf8732b34c43a38465efde7e18a060)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:45.843240 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4fa54-8e07-4e85-9e0e-6c7999b16f92 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:45.856403 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 615/1232] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:07:45 2022] PUT /identity/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/28953289ea704853b1c0c7049915633d/roles/bdaf8732b34c43a38465efde7e18a060 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:07:48.214082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939b412-a393-487e-8002-fef35d01fbaf None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:48.214705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939b412-a393-487e-8002-fef35d01fbaf None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:48.214847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939b412-a393-487e-8002-fef35d01fbaf None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:48.216909 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 618/1233] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:07:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:07:48.223500 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084b4a9-34e8-400e-a366-f76348e47d73 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:48.224175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084b4a9-34e8-400e-a366-f76348e47d73 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:48.224175 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084b4a9-34e8-400e-a366-f76348e47d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:48.241587 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-5084b4a9-34e8-400e-a366-f76348e47d73 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:48.242964 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-5084b4a9-34e8-400e-a366-f76348e47d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:48.316446 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 616/1234] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:07:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:07:48.340708 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939b412-a393-487e-8002-fef35d01fbaf None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:48.367960 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:48.450874 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939b412-a393-487e-8002-fef35d01fbaf None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:48.488072 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cfae2-01d0-48c1-8b3d-bc27a17ac0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:48.488803 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cfae2-01d0-48c1-8b3d-bc27a17ac0d7 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:48.489080 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cfae2-01d0-48c1-8b3d-bc27a17ac0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:48.489352 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cfae2-01d0-48c1-8b3d-bc27a17ac0d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:48.500086 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24cfae2-01d0-48c1-8b3d-bc27a17ac0d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-c24cfae2-01d0-48c1-8b3d-bc27a17ac0d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:48.507943 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:48.509185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:48.510391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:48.510391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:48.510391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:48.510391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:48.510391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:48.510391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:48.510391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:48.510391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:48.510391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:48.510391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:48.510391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:48.510391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:07:48.510391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 619/1235] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:07:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:48.515292 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084b4a9-34e8-400e-a366-f76348e47d73 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:48.564551 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:48.622581 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084b4a9-34e8-400e-a366-f76348e47d73 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:48.658876 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100bf08-981e-4eba-8ad3-898a6ea359ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:48.659522 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100bf08-981e-4eba-8ad3-898a6ea359ad None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:48.659800 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100bf08-981e-4eba-8ad3-898a6ea359ad None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:48.660094 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100bf08-981e-4eba-8ad3-898a6ea359ad None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:48.660835 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8100bf08-981e-4eba-8ad3-898a6ea359ad None admin] RBAC: Adding query filter params (name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:48.661146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8100bf08-981e-4eba-8ad3-898a6ea359ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:48.664900 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8100bf08-981e-4eba-8ad3-898a6ea359ad None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:48.673475 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 617/1236] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:07:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:48.679773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cfae2-01d0-48c1-8b3d-bc27a17ac0d7 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:48.777408 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:48.794699 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cfae2-01d0-48c1-8b3d-bc27a17ac0d7 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:48.829723 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc679c-2a58-49d7-a00f-1b83ecf7a0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:48.830606 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc679c-2a58-49d7-a00f-1b83ecf7a0bd None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:48.830946 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc679c-2a58-49d7-a00f-1b83ecf7a0bd None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:48.831218 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc679c-2a58-49d7-a00f-1b83ecf7a0bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:48.842559 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc679c-2a58-49d7-a00f-1b83ecf7a0bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:48.844449 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc679c-2a58-49d7-a00f-1b83ecf7a0bd None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:48.862609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-b8cc679c-2a58-49d7-a00f-1b83ecf7a0bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:48.862609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:48.862609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:48.862609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:48.862609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:48.862609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:48.862609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:48.862609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:48.862609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:48.862609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:48.862609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:48.862609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:48.862609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:48.862609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:48.862609 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:48.863885 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:48.865665 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:48.865665 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:48.865665 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:48.865665 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:07:48.865665 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 620/1237] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:48.873217 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100bf08-981e-4eba-8ad3-898a6ea359ad None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:48.977304 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100bf08-981e-4eba-8ad3-898a6ea359ad None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:48.986718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:49.013337 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b2a40-bf5e-4d23-b47f-832f4dc5e5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:49.014060 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b2a40-bf5e-4d23-b47f-832f4dc5e5ce None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:49.014341 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b2a40-bf5e-4d23-b47f-832f4dc5e5ce None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:49.014620 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b2a40-bf5e-4d23-b47f-832f4dc5e5ce None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:49.015359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b2a40-bf5e-4d23-b47f-832f4dc5e5ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:49.015665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b2a40-bf5e-4d23-b47f-832f4dc5e5ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:49.017063 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b2a40-bf5e-4d23-b47f-832f4dc5e5ce None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:49.030053 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 618/1238] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:49.036335 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc679c-2a58-49d7-a00f-1b83ecf7a0bd None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:49.150279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc679c-2a58-49d7-a00f-1b83ecf7a0bd None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:49.186624 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b585cb-4bee-4c82-8f9e-a56854508620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:49.187435 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b585cb-4bee-4c82-8f9e-a56854508620 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:49.187710 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b585cb-4bee-4c82-8f9e-a56854508620 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:49.188003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b585cb-4bee-4c82-8f9e-a56854508620 None admin] PATH_INFO: `/v3/users/placement` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:49.191055 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:49.197788 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b585cb-4bee-4c82-8f9e-a56854508620 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:49.199743 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b585cb-4bee-4c82-8f9e-a56854508620 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-f1b585cb-4bee-4c82-8f9e-a56854508620 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:49.217201 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:49.219358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:49.220406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:49.220406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:49.220406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:49.220406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:49.220406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:49.220406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:49.220406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:49.220406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:49.220406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:07:49.220406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:07:49.220406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 621/1239] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:49.225642 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b2a40-bf5e-4d23-b47f-832f4dc5e5ce None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:49.329753 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b2a40-bf5e-4d23-b47f-832f4dc5e5ce None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:49.366894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03684094-2bc2-48d7-baf7-9cb42a964ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:49.367724 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03684094-2bc2-48d7-baf7-9cb42a964ba4 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:49.368008 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03684094-2bc2-48d7-baf7-9cb42a964ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:49.368283 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03684094-2bc2-48d7-baf7-9cb42a964ba4 None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:49.369119 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03684094-2bc2-48d7-baf7-9cb42a964ba4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:49.369429 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03684094-2bc2-48d7-baf7-9cb42a964ba4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:49.370891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03684094-2bc2-48d7-baf7-9cb42a964ba4 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:49.391080 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 619/1240] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 20:07:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:49.397126 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b585cb-4bee-4c82-8f9e-a56854508620 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:49.504216 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b585cb-4bee-4c82-8f9e-a56854508620 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:49.544599 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540a04a-77d9-443e-ac08-b03e0003ecd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:49.545261 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540a04a-77d9-443e-ac08-b03e0003ecd7 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:49.545537 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540a04a-77d9-443e-ac08-b03e0003ecd7 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:49.545807 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540a04a-77d9-443e-ac08-b03e0003ecd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:49.614719 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:49.623083 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540a04a-77d9-443e-ac08-b03e0003ecd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:49.624616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540a04a-77d9-443e-ac08-b03e0003ecd7 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:49.745471 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 622/1241] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:07:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:07:49.746869 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03684094-2bc2-48d7-baf7-9cb42a964ba4 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:49.776382 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:49.850770 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03684094-2bc2-48d7-baf7-9cb42a964ba4 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:49.885483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fa10c-5c53-46de-b50c-614d5aaf2d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:49.886196 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fa10c-5c53-46de-b50c-614d5aaf2d76 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:49.886504 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fa10c-5c53-46de-b50c-614d5aaf2d76 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:49.886777 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fa10c-5c53-46de-b50c-614d5aaf2d76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:49.901217 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fa10c-5c53-46de-b50c-614d5aaf2d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:49.902714 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fa10c-5c53-46de-b50c-614d5aaf2d76 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:49.921619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-6e2fa10c-5c53-46de-b50c-614d5aaf2d76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:49.921619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:49.921619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:49.921619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:49.921619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:49.921619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:49.921619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:49.921619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:49.921619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:49.921619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:49.921619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:49.921619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:49.921619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:49.921619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:49.921619 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:49.922913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:49.924071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:49.924071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:49.924071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:49.924071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:07:49.924071 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 620/1242] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:49.929042 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540a04a-77d9-443e-ac08-b03e0003ecd7 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:50.033941 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:50.040620 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540a04a-77d9-443e-ac08-b03e0003ecd7 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:50.075890 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0d857-690f-4650-84bc-06f3f978102a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:50.076564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0d857-690f-4650-84bc-06f3f978102a None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:50.076840 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0d857-690f-4650-84bc-06f3f978102a None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:50.077177 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0d857-690f-4650-84bc-06f3f978102a None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:50.077914 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0d857-690f-4650-84bc-06f3f978102a None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:50.078223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0d857-690f-4650-84bc-06f3f978102a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:50.080882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0d857-690f-4650-84bc-06f3f978102a None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:50.092370 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 623/1243] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:50.098546 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fa10c-5c53-46de-b50c-614d5aaf2d76 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:50.186953 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:50.217902 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fa10c-5c53-46de-b50c-614d5aaf2d76 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:50.264171 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c1e18-43c0-4ff9-88b9-f10082971d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:50.265185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c1e18-43c0-4ff9-88b9-f10082971d3a None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:50.265496 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c1e18-43c0-4ff9-88b9-f10082971d3a None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:50.265808 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c1e18-43c0-4ff9-88b9-f10082971d3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:50.273741 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c1e18-43c0-4ff9-88b9-f10082971d3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:50.275634 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c1e18-43c0-4ff9-88b9-f10082971d3a None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:50.291565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-ef1c1e18-43c0-4ff9-88b9-f10082971d3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:50.291565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:50.291565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:50.291565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:50.291565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:50.291565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:50.291565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:50.291565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:50.291565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:50.291565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:50.291565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:50.291565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:50.291565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:50.291565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:50.291565 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:50.292648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:50.293648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:50.293648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:50.293648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:50.293648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:50.293648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:50.293648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:50.293648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:50.293648 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:07:50.295516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 621/1244] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:07:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:50.301577 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0d857-690f-4650-84bc-06f3f978102a None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:50.414510 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0d857-690f-4650-84bc-06f3f978102a None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:50.456324 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f049fab6-bd4e-4087-b165-91da05934263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:50.457034 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049fab6-bd4e-4087-b165-91da05934263 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:50.457286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049fab6-bd4e-4087-b165-91da05934263 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:50.457564 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049fab6-bd4e-4087-b165-91da05934263 None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:50.458576 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049fab6-bd4e-4087-b165-91da05934263 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:50.458824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049fab6-bd4e-4087-b165-91da05934263 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:50.460644 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049fab6-bd4e-4087-b165-91da05934263 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:50.481424 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 624/1245] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:07:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:50.482802 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c1e18-43c0-4ff9-88b9-f10082971d3a None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:50.599149 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c1e18-43c0-4ff9-88b9-f10082971d3a None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:50.642192 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34de25-b24f-4435-9d72-faa605ffbd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:50.642992 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34de25-b24f-4435-9d72-faa605ffbd28 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:50.643351 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34de25-b24f-4435-9d72-faa605ffbd28 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:50.643580 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34de25-b24f-4435-9d72-faa605ffbd28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:50.644419 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34de25-b24f-4435-9d72-faa605ffbd28 None admin] RBAC: Adding query filter params (role.id=bdaf8732b34c43a38465efde7e18a060, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=28953289ea704853b1c0c7049915633d) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:50.644742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34de25-b24f-4435-9d72-faa605ffbd28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:50.646156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34de25-b24f-4435-9d72-faa605ffbd28 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:50.652635 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 622/1246] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:07:50 2022] GET /identity/v3/role_assignments?user.id=28953289ea704853b1c0c7049915633d&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=bdaf8732b34c43a38465efde7e18a060 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:53.064857 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab45bd-bf19-4068-b320-e38673dc2f44 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:53.065512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab45bd-bf19-4068-b320-e38673dc2f44 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:53.065512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab45bd-bf19-4068-b320-e38673dc2f44 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:53.066926 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 625/1247] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:53.072687 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907868a4-5bf5-4fc9-b2f1-b7e506235313 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:53.073146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907868a4-5bf5-4fc9-b2f1-b7e506235313 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:53.073260 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907868a4-5bf5-4fc9-b2f1-b7e506235313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:53.078779 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:53.093440 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-907868a4-5bf5-4fc9-b2f1-b7e506235313 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:53.095155 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-907868a4-5bf5-4fc9-b2f1-b7e506235313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:53.168646 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 623/1248] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:07:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:07:53.192987 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab45bd-bf19-4068-b320-e38673dc2f44 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:53.215838 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:53.316502 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab45bd-bf19-4068-b320-e38673dc2f44 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:53.353997 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2599f-95b3-471e-b335-655453051878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:53.355079 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2599f-95b3-471e-b335-655453051878 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:53.355466 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2599f-95b3-471e-b335-655453051878 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:53.355864 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2599f-95b3-471e-b335-655453051878 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:53.368202 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2599f-95b3-471e-b335-655453051878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-cde2599f-95b3-471e-b335-655453051878 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:53.377019 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:53.378450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:53.379633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:53.379633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:53.379633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:53.379633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:53.379633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:53.379633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:53.379633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:53.379633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:53.379633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:53.379633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:53.379633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:53.379633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:07:53.379633 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 626/1249] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:07:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:53.385294 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907868a4-5bf5-4fc9-b2f1-b7e506235313 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:53.500231 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907868a4-5bf5-4fc9-b2f1-b7e506235313 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:53.531296 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:53.538335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f416980-fa93-4fd5-8fd1-0ceb05416358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:53.538957 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f416980-fa93-4fd5-8fd1-0ceb05416358 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:53.539235 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f416980-fa93-4fd5-8fd1-0ceb05416358 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:53.539511 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f416980-fa93-4fd5-8fd1-0ceb05416358 None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:53.540269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f416980-fa93-4fd5-8fd1-0ceb05416358 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:53.540574 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f416980-fa93-4fd5-8fd1-0ceb05416358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:53.542431 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f416980-fa93-4fd5-8fd1-0ceb05416358 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:53.556907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 624/1250] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:07:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:53.562781 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2599f-95b3-471e-b335-655453051878 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:53.651676 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:53.676185 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2599f-95b3-471e-b335-655453051878 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:53.714021 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c0751-da33-4441-b206-d085721317a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:53.714851 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c0751-da33-4441-b206-d085721317a6 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:53.715130 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c0751-da33-4441-b206-d085721317a6 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:53.715407 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c0751-da33-4441-b206-d085721317a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:53.726954 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c0751-da33-4441-b206-d085721317a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:53.728427 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c0751-da33-4441-b206-d085721317a6 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:53.749251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-192c0751-da33-4441-b206-d085721317a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:53.749251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:53.749251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:53.749251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:53.749251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:53.749251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:53.749251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:53.749251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:53.749251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:53.749251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:53.749251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:53.749251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:53.749251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:53.749251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:53.749251 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:53.750583 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:53.751662 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:53.751662 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:53.751662 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:53.751662 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:07:53.751662 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 627/1251] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:53.756560 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f416980-fa93-4fd5-8fd1-0ceb05416358 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:53.873544 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f416980-fa93-4fd5-8fd1-0ceb05416358 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:53.909692 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c30e1-948f-4e0d-81e6-fe764d87fd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:53.910427 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c30e1-948f-4e0d-81e6-fe764d87fd0c None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:53.910712 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c30e1-948f-4e0d-81e6-fe764d87fd0c None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:53.910986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c30e1-948f-4e0d-81e6-fe764d87fd0c None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:53.911728 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c30e1-948f-4e0d-81e6-fe764d87fd0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:53.912036 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c30e1-948f-4e0d-81e6-fe764d87fd0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:53.913573 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c30e1-948f-4e0d-81e6-fe764d87fd0c None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:53.926605 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 625/1252] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:53.933101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c0751-da33-4441-b206-d085721317a6 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:54.043406 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c0751-da33-4441-b206-d085721317a6 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:54.072800 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:54.081663 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03795507-4e7d-4acf-814d-8e0628a95966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:54.082516 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03795507-4e7d-4acf-814d-8e0628a95966 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:54.082797 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03795507-4e7d-4acf-814d-8e0628a95966 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:54.083110 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03795507-4e7d-4acf-814d-8e0628a95966 None admin] PATH_INFO: `/v3/users/placement` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:54.091186 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03795507-4e7d-4acf-814d-8e0628a95966 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:54.092690 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03795507-4e7d-4acf-814d-8e0628a95966 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-03795507-4e7d-4acf-814d-8e0628a95966 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:54.108649 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:54.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:54.111031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:54.111031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:54.111031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:54.111031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:54.111031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:54.111031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:54.111031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:54.111031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:54.111031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:07:54.111031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:07:54.111031 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 628/1253] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:54.116440 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c30e1-948f-4e0d-81e6-fe764d87fd0c None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:54.177489 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:54.226031 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c30e1-948f-4e0d-81e6-fe764d87fd0c None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:54.263970 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ead6a-6eea-4fea-8e7c-bd77cfcf46b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:54.264652 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ead6a-6eea-4fea-8e7c-bd77cfcf46b0 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:54.264934 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ead6a-6eea-4fea-8e7c-bd77cfcf46b0 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:54.265210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ead6a-6eea-4fea-8e7c-bd77cfcf46b0 None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:54.266056 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ead6a-6eea-4fea-8e7c-bd77cfcf46b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:54.266398 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ead6a-6eea-4fea-8e7c-bd77cfcf46b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:54.267838 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ead6a-6eea-4fea-8e7c-bd77cfcf46b0 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:54.286151 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 626/1254] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 20:07:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:54.291790 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03795507-4e7d-4acf-814d-8e0628a95966 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:54.400563 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03795507-4e7d-4acf-814d-8e0628a95966 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:54.437060 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e7bdc-80cd-4872-bea6-7a7d3fb6744b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:54.437637 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e7bdc-80cd-4872-bea6-7a7d3fb6744b None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:54.437917 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e7bdc-80cd-4872-bea6-7a7d3fb6744b None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:54.438195 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e7bdc-80cd-4872-bea6-7a7d3fb6744b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:54.502547 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:54.518032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e7bdc-80cd-4872-bea6-7a7d3fb6744b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:54.519939 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e7bdc-80cd-4872-bea6-7a7d3fb6744b None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:54.633887 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 629/1255] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:07:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:07:54.639853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ead6a-6eea-4fea-8e7c-bd77cfcf46b0 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:54.755627 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ead6a-6eea-4fea-8e7c-bd77cfcf46b0 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:54.772496 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:54.800602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe22851-f3a0-40a0-8cd5-5e06c9f4f8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:54.801483 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe22851-f3a0-40a0-8cd5-5e06c9f4f8f8 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:54.801823 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe22851-f3a0-40a0-8cd5-5e06c9f4f8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:54.802244 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe22851-f3a0-40a0-8cd5-5e06c9f4f8f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:54.813854 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe22851-f3a0-40a0-8cd5-5e06c9f4f8f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:54.815646 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe22851-f3a0-40a0-8cd5-5e06c9f4f8f8 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:54.833812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-4fe22851-f3a0-40a0-8cd5-5e06c9f4f8f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:54.833812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:54.833812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:54.833812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:54.833812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:54.833812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:54.833812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:54.833812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:54.833812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:54.833812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:54.833812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:54.833812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:54.833812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:54.833812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:54.833812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:54.835157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:54.836365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:54.836365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:54.836365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:54.836365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:07:54.836365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 627/1256] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:54.842795 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e7bdc-80cd-4872-bea6-7a7d3fb6744b None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:54.951125 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:54.973772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e7bdc-80cd-4872-bea6-7a7d3fb6744b None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:55.015978 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef97ac3-ff24-420f-bd0d-cfabb7ca1a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:55.016570 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef97ac3-ff24-420f-bd0d-cfabb7ca1a5c None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.016847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef97ac3-ff24-420f-bd0d-cfabb7ca1a5c None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.017119 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef97ac3-ff24-420f-bd0d-cfabb7ca1a5c None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.017850 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef97ac3-ff24-420f-bd0d-cfabb7ca1a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:55.018155 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef97ac3-ff24-420f-bd0d-cfabb7ca1a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:55.019746 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef97ac3-ff24-420f-bd0d-cfabb7ca1a5c None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:55.031568 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 630/1257] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:55.037304 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe22851-f3a0-40a0-8cd5-5e06c9f4f8f8 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:55.143707 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe22851-f3a0-40a0-8cd5-5e06c9f4f8f8 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:55.186537 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3c86a-55c4-462b-910f-928e40653123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:55.187585 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3c86a-55c4-462b-910f-928e40653123 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.187896 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3c86a-55c4-462b-910f-928e40653123 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.188271 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3c86a-55c4-462b-910f-928e40653123 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.199212 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3c86a-55c4-462b-910f-928e40653123 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:55.202082 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3c86a-55c4-462b-910f-928e40653123 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:55.228187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-ace3c86a-55c4-462b-910f-928e40653123 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:55.228187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:55.228187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:55.228187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:55.228187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:55.228187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:55.228187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:55.228187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:55.228187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:55.228187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:55.228187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:55.228187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:55.228187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:55.228187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:55.228187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:55.229807 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:55.230851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:55.230851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:55.230851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:55.230851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:55.230851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:55.230851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:55.230851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:55.230851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:07:55.230851 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 628/1258] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:07:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:55.235735 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef97ac3-ff24-420f-bd0d-cfabb7ca1a5c None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:55.348340 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef97ac3-ff24-420f-bd0d-cfabb7ca1a5c None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:55.384627 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52817b-f4ff-4e50-a359-45bfde1a3ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:55.385332 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52817b-f4ff-4e50-a359-45bfde1a3ebb None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.385586 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52817b-f4ff-4e50-a359-45bfde1a3ebb None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.385880 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52817b-f4ff-4e50-a359-45bfde1a3ebb None admin] PATH_INFO: `/v3/projects` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.386760 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52817b-f4ff-4e50-a359-45bfde1a3ebb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:55.387075 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52817b-f4ff-4e50-a359-45bfde1a3ebb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:55.388585 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52817b-f4ff-4e50-a359-45bfde1a3ebb None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:55.402190 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 631/1259] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:07:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:55.408603 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3c86a-55c4-462b-910f-928e40653123 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:55.439179 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:55.524742 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3c86a-55c4-462b-910f-928e40653123 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:55.571913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53373600-b5f6-4674-ae99-edc63c21441f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:55.572617 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53373600-b5f6-4674-ae99-edc63c21441f None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.572884 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53373600-b5f6-4674-ae99-edc63c21441f None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.573145 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53373600-b5f6-4674-ae99-edc63c21441f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.573995 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53373600-b5f6-4674-ae99-edc63c21441f None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=28953289ea704853b1c0c7049915633d) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:55.574387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53373600-b5f6-4674-ae99-edc63c21441f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:55.575811 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53373600-b5f6-4674-ae99-edc63c21441f None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:55.581822 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 629/1260] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:07:55 2022] GET /identity/v3/role_assignments?user.id=28953289ea704853b1c0c7049915633d&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:57.807104 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857bf2c-065d-421f-8616-25c5cfcb4c4e None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:57.807690 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857bf2c-065d-421f-8616-25c5cfcb4c4e None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:57.807690 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857bf2c-065d-421f-8616-25c5cfcb4c4e None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:57.809064 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 632/1261] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:57.815507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8cd7a-1fa5-4cdd-b56f-26f40a0d48fb None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:57.816272 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8cd7a-1fa5-4cdd-b56f-26f40a0d48fb None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:57.816418 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8cd7a-1fa5-4cdd-b56f-26f40a0d48fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:57.838024 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-8ed8cd7a-1fa5-4cdd-b56f-26f40a0d48fb None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:57.839177 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-8ed8cd7a-1fa5-4cdd-b56f-26f40a0d48fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:57.920763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 630/1262] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:07:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:07:57.944023 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857bf2c-065d-421f-8616-25c5cfcb4c4e None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:57.956590 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:58.053986 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857bf2c-065d-421f-8616-25c5cfcb4c4e None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:58.090731 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ebe8e-7dd0-40ae-bdc7-44cacbb482a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:58.091450 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ebe8e-7dd0-40ae-bdc7-44cacbb482a3 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:58.091725 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ebe8e-7dd0-40ae-bdc7-44cacbb482a3 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:58.091997 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ebe8e-7dd0-40ae-bdc7-44cacbb482a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:58.103730 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ebe8e-7dd0-40ae-bdc7-44cacbb482a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-fb4ebe8e-7dd0-40ae-bdc7-44cacbb482a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:58.111306 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:58.112467 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:58.113718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:58.113718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:58.113718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:58.113718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:58.113718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:58.113718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:58.113718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:58.113718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:58.113718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:58.113718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:58.113718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:58.113718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:07:58.113718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 633/1263] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:07:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:58.118889 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8cd7a-1fa5-4cdd-b56f-26f40a0d48fb None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:58.137501 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:58.225901 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8cd7a-1fa5-4cdd-b56f-26f40a0d48fb None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:58.263827 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c02b49-4eb2-45bb-b10c-2a98a72a7f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:58.264647 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c02b49-4eb2-45bb-b10c-2a98a72a7f5a None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:58.265000 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c02b49-4eb2-45bb-b10c-2a98a72a7f5a None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:58.265317 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c02b49-4eb2-45bb-b10c-2a98a72a7f5a None admin] PATH_INFO: `/v3/roles` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:58.266293 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c02b49-4eb2-45bb-b10c-2a98a72a7f5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:58.266671 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c02b49-4eb2-45bb-b10c-2a98a72a7f5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:58.268331 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c02b49-4eb2-45bb-b10c-2a98a72a7f5a None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:58.276885 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 631/1264] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:07:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:58.282925 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ebe8e-7dd0-40ae-bdc7-44cacbb482a3 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:58.357977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:58.397391 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ebe8e-7dd0-40ae-bdc7-44cacbb482a3 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:58.433221 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f40cb-70ee-4f57-874e-08eaea842200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:58.434001 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f40cb-70ee-4f57-874e-08eaea842200 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:58.434304 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f40cb-70ee-4f57-874e-08eaea842200 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:58.434578 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f40cb-70ee-4f57-874e-08eaea842200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:58.445975 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f40cb-70ee-4f57-874e-08eaea842200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:58.447358 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f40cb-70ee-4f57-874e-08eaea842200 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:58.464411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-902f40cb-70ee-4f57-874e-08eaea842200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:58.464411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:58.464411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:58.464411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:58.464411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:58.464411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:58.464411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:58.464411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:58.464411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:58.464411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:58.464411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:58.464411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:58.464411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:58.464411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:58.464411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:58.465533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:58.466768 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:58.466768 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:58.466768 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:58.466768 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:07:58.466768 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 634/1265] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:58.471812 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c02b49-4eb2-45bb-b10c-2a98a72a7f5a None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:58.563762 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:58.588187 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c02b49-4eb2-45bb-b10c-2a98a72a7f5a None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:58.627299 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d4647-6ca0-424b-a323-69be9a993fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:58.628172 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d4647-6ca0-424b-a323-69be9a993fba None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:58.628529 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d4647-6ca0-424b-a323-69be9a993fba None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:58.628892 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d4647-6ca0-424b-a323-69be9a993fba None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:58.630002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d4647-6ca0-424b-a323-69be9a993fba None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:58.630412 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d4647-6ca0-424b-a323-69be9a993fba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:58.632361 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d4647-6ca0-424b-a323-69be9a993fba None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:58.644828 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 632/1266] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:58.652113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f40cb-70ee-4f57-874e-08eaea842200 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:58.765322 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f40cb-70ee-4f57-874e-08eaea842200 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:58.785390 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:58.802569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafa63b-4e84-4e68-bded-8706d28bca11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:58.803373 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafa63b-4e84-4e68-bded-8706d28bca11 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:58.803699 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafa63b-4e84-4e68-bded-8706d28bca11 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:58.803970 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafa63b-4e84-4e68-bded-8706d28bca11 None admin] PATH_INFO: `/v3/users/placement` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:58.812174 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafa63b-4e84-4e68-bded-8706d28bca11 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:58.813627 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafa63b-4e84-4e68-bded-8706d28bca11 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-9fafa63b-4e84-4e68-bded-8706d28bca11 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:58.828515 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:58.829666 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:58.831142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:58.831142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:58.831142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:58.831142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:58.831142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:58.831142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:58.831142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:58.831142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:58.831142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:07:58.831142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:07:58.831142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 635/1267] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:58.837008 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d4647-6ca0-424b-a323-69be9a993fba None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:58.943454 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d4647-6ca0-424b-a323-69be9a993fba None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:58.979280 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bd8be-2625-4180-8b13-9094b023370d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:58.979951 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bd8be-2625-4180-8b13-9094b023370d None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:58.980226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bd8be-2625-4180-8b13-9094b023370d None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:58.980501 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bd8be-2625-4180-8b13-9094b023370d None admin] PATH_INFO: `/v3/users` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:58.981342 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bd8be-2625-4180-8b13-9094b023370d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:58.981652 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bd8be-2625-4180-8b13-9094b023370d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:58.983106 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bd8be-2625-4180-8b13-9094b023370d None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:58.991913 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:59.005881 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 633/1268] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 20:07:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:59.012834 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafa63b-4e84-4e68-bded-8706d28bca11 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:59.121964 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafa63b-4e84-4e68-bded-8706d28bca11 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:59.156846 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40584a-e695-4f6b-8327-f1095236c60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:59.157616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40584a-e695-4f6b-8327-f1095236c60b None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:59.157889 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40584a-e695-4f6b-8327-f1095236c60b None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:59.158166 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40584a-e695-4f6b-8327-f1095236c60b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:59.170292 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40584a-e695-4f6b-8327-f1095236c60b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:59.171769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40584a-e695-4f6b-8327-f1095236c60b None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:59.190245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-aa40584a-e695-4f6b-8327-f1095236c60b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:59.190245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:59.190245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:59.190245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:59.190245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:59.190245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:59.190245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:59.190245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:59.190245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:59.190245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:59.190245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:59.190245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:59.190245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:59.190245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:59.190245 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:59.191415 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:59.192426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:59.192426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:59.192426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:59.192426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:07:59.192426 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 636/1269] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:59.198528 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bd8be-2625-4180-8b13-9094b023370d None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:59.305897 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bd8be-2625-4180-8b13-9094b023370d None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:59.342153 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79459c3-7b3e-4337-804d-339890996961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:59.342844 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79459c3-7b3e-4337-804d-339890996961 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:59.343125 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79459c3-7b3e-4337-804d-339890996961 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:59.343393 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79459c3-7b3e-4337-804d-339890996961 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:59.344448 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79459c3-7b3e-4337-804d-339890996961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:59.344751 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79459c3-7b3e-4337-804d-339890996961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:59.346121 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79459c3-7b3e-4337-804d-339890996961 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:59.357622 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 634/1270] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:59.367145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40584a-e695-4f6b-8327-f1095236c60b None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:59.402097 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:59.473830 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40584a-e695-4f6b-8327-f1095236c60b None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:59.508702 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666c2bb-3a04-4659-b496-a18a85f8a983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:59.509493 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666c2bb-3a04-4659-b496-a18a85f8a983 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:59.509769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666c2bb-3a04-4659-b496-a18a85f8a983 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:59.510043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666c2bb-3a04-4659-b496-a18a85f8a983 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:59.517157 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666c2bb-3a04-4659-b496-a18a85f8a983 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:59.518647 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666c2bb-3a04-4659-b496-a18a85f8a983 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:59.533385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-2666c2bb-3a04-4659-b496-a18a85f8a983 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:59.533385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:59.533385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:59.533385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:59.533385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:59.533385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:59.533385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:59.533385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:59.533385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:59.533385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:59.533385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:59.533385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:59.533385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:59.533385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:59.533385 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:59.534513 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:59.535514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:59.535514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:59.535514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:59.535514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:59.535514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:59.535514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:59.535514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:59.535514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:07:59.535514 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 637/1271] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:07:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:59.540744 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79459c3-7b3e-4337-804d-339890996961 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:59.603652 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:59.651424 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79459c3-7b3e-4337-804d-339890996961 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:59.689349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ddd35-d3f2-4b01-9f47-74ee02b66a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:59.690009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ddd35-d3f2-4b01-9f47-74ee02b66a48 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:59.690307 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ddd35-d3f2-4b01-9f47-74ee02b66a48 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:59.690590 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ddd35-d3f2-4b01-9f47-74ee02b66a48 None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:59.691370 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ddd35-d3f2-4b01-9f47-74ee02b66a48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:59.691678 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ddd35-d3f2-4b01-9f47-74ee02b66a48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:59.693107 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ddd35-d3f2-4b01-9f47-74ee02b66a48 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:59.706403 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 635/1272] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:07:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:59.713979 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666c2bb-3a04-4659-b496-a18a85f8a983 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:59.816057 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:59.827103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666c2bb-3a04-4659-b496-a18a85f8a983 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:59.867843 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea1ff6-e7bf-42cb-8fee-b1f41bb537f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:59.868728 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea1ff6-e7bf-42cb-8fee-b1f41bb537f5 None admin] REQUEST_METHOD: `PUT` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:59.869004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea1ff6-e7bf-42cb-8fee-b1f41bb537f5 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:59.869272 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea1ff6-e7bf-42cb-8fee-b1f41bb537f5 None admin] PATH_INFO: `/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/28953289ea704853b1c0c7049915633d/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:59.871347 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea1ff6-e7bf-42cb-8fee-b1f41bb537f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba22fdc274b74521847e5fceb2d307b1, user_id=28953289ea704853b1c0c7049915633d, role_id=b34e4ad4fe7b4a60bd2fc669e2b31a66)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:59.872786 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea1ff6-e7bf-42cb-8fee-b1f41bb537f5 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:59.889528 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 638/1273] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:07:59 2022] PUT /identity/v3/projects/ba22fdc274b74521847e5fceb2d307b1/users/28953289ea704853b1c0c7049915633d/roles/b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:08:02.072319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd82d3-d37b-4059-a15f-cf8673d612dd None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.073226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd82d3-d37b-4059-a15f-cf8673d612dd None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.073226 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd82d3-d37b-4059-a15f-cf8673d612dd None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.075511 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 636/1274] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:08:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:02.081856 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736a07f-f178-4adc-9b36-d6b95b153e45 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.082665 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736a07f-f178-4adc-9b36-d6b95b153e45 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.082853 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736a07f-f178-4adc-9b36-d6b95b153e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.103065 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-4736a07f-f178-4adc-9b36-d6b95b153e45 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:02.104286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-4736a07f-f178-4adc-9b36-d6b95b153e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:02.186865 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 639/1275] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:08:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:02.214791 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd82d3-d37b-4059-a15f-cf8673d612dd None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:02.336214 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd82d3-d37b-4059-a15f-cf8673d612dd None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:02.356602 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:02.375407 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebb23b-a28a-4c9c-9198-39a49c6163c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:02.376212 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebb23b-a28a-4c9c-9198-39a49c6163c0 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.376495 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebb23b-a28a-4c9c-9198-39a49c6163c0 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.376761 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebb23b-a28a-4c9c-9198-39a49c6163c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.388153 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebb23b-a28a-4c9c-9198-39a49c6163c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-72ebb23b-a28a-4c9c-9198-39a49c6163c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:08:02.395939 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:02.397095 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:02.398064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:02.398064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:02.398064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:02.398064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:02.398064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:08:02.398064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:08:02.398064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:08:02.398064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:08:02.398064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:08:02.398064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:08:02.398064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:08:02.398064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:08:02.398064 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 637/1276] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:08:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:02.403717 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736a07f-f178-4adc-9b36-d6b95b153e45 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:02.488651 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:02.516064 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736a07f-f178-4adc-9b36-d6b95b153e45 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:02.552687 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c4543-c101-4eea-a297-32ff69eaf984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:02.553395 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c4543-c101-4eea-a297-32ff69eaf984 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.553661 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c4543-c101-4eea-a297-32ff69eaf984 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.554024 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c4543-c101-4eea-a297-32ff69eaf984 None admin] PATH_INFO: `/v3/roles` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.554815 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c4543-c101-4eea-a297-32ff69eaf984 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:02.555119 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c4543-c101-4eea-a297-32ff69eaf984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:02.556500 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c4543-c101-4eea-a297-32ff69eaf984 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:02.565051 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 640/1277] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:08:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:02.571067 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebb23b-a28a-4c9c-9198-39a49c6163c0 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:02.681849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebb23b-a28a-4c9c-9198-39a49c6163c0 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:02.718166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad7fdc-346d-405e-841d-310fe1195fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:02.719301 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7fdc-346d-405e-841d-310fe1195fb1 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.719583 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7fdc-346d-405e-841d-310fe1195fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.719858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7fdc-346d-405e-841d-310fe1195fb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.732044 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad7fdc-346d-405e-841d-310fe1195fb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:02.733514 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad7fdc-346d-405e-841d-310fe1195fb1 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:02.752986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-d9ad7fdc-346d-405e-841d-310fe1195fb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:02.752986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:02.752986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:02.752986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:02.752986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:02.752986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:02.752986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:02.752986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:02.752986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:02.752986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:02.752986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:02.752986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:02.752986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:02.752986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:02.752986 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:02.754166 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:02.755605 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:02.755605 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:02.755605 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:02.755605 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:08:02.755605 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 638/1278] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:02.760336 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c4543-c101-4eea-a297-32ff69eaf984 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:02.868332 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c4543-c101-4eea-a297-32ff69eaf984 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:02.907853 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bac2d9-7339-467d-8ec7-2f572ca8e953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:02.908511 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bac2d9-7339-467d-8ec7-2f572ca8e953 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.908791 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bac2d9-7339-467d-8ec7-2f572ca8e953 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.909066 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bac2d9-7339-467d-8ec7-2f572ca8e953 None admin] PATH_INFO: `/v3/domains` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.909793 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bac2d9-7339-467d-8ec7-2f572ca8e953 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:02.910104 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bac2d9-7339-467d-8ec7-2f572ca8e953 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:02.911538 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bac2d9-7339-467d-8ec7-2f572ca8e953 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:02.924421 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 641/1279] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:02.931129 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad7fdc-346d-405e-841d-310fe1195fb1 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:02.967511 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:03.044208 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad7fdc-346d-405e-841d-310fe1195fb1 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:03.080104 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e221603-7b73-4dcb-96ee-0ab19d34c682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:03.080904 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e221603-7b73-4dcb-96ee-0ab19d34c682 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:03.081182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e221603-7b73-4dcb-96ee-0ab19d34c682 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:03.081451 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e221603-7b73-4dcb-96ee-0ab19d34c682 None admin] PATH_INFO: `/v3/users/placement` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:03.090312 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e221603-7b73-4dcb-96ee-0ab19d34c682 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:03.091761 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e221603-7b73-4dcb-96ee-0ab19d34c682 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-3e221603-7b73-4dcb-96ee-0ab19d34c682 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:08:03.107297 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:03.108432 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:03.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:03.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:03.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:08:03.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:08:03.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:08:03.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:08:03.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:08:03.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:08:03.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:08:03.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:08:03.109783 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 639/1280] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:08:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:03.114375 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bac2d9-7339-467d-8ec7-2f572ca8e953 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:03.118549 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:03.234194 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bac2d9-7339-467d-8ec7-2f572ca8e953 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:03.269721 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09f286-6ca4-43fa-9168-734bd4f5652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:03.270411 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09f286-6ca4-43fa-9168-734bd4f5652e None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:03.270690 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09f286-6ca4-43fa-9168-734bd4f5652e None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:03.270959 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09f286-6ca4-43fa-9168-734bd4f5652e None admin] PATH_INFO: `/v3/users` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:03.271785 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09f286-6ca4-43fa-9168-734bd4f5652e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:03.272141 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09f286-6ca4-43fa-9168-734bd4f5652e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:03.273571 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09f286-6ca4-43fa-9168-734bd4f5652e None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:03.290686 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 642/1281] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 20:08:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:03.300115 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e221603-7b73-4dcb-96ee-0ab19d34c682 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:03.409454 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:03.420145 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e221603-7b73-4dcb-96ee-0ab19d34c682 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:03.459970 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba8459-db64-4a90-be32-a89a1335996b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:03.460327 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba8459-db64-4a90-be32-a89a1335996b None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:03.460603 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba8459-db64-4a90-be32-a89a1335996b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:03.460875 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba8459-db64-4a90-be32-a89a1335996b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:03.531042 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba8459-db64-4a90-be32-a89a1335996b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:03.532439 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba8459-db64-4a90-be32-a89a1335996b None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:03.632755 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:03.643295 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 640/1282] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:08:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:08:03.649572 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09f286-6ca4-43fa-9168-734bd4f5652e None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:03.715968 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:03.763774 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09f286-6ca4-43fa-9168-734bd4f5652e None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:03.799920 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d70b3-4bb3-4443-97fe-f9b08ce1ab29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:03.800718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d70b3-4bb3-4443-97fe-f9b08ce1ab29 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:03.800996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d70b3-4bb3-4443-97fe-f9b08ce1ab29 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:03.801266 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d70b3-4bb3-4443-97fe-f9b08ce1ab29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:03.812083 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d70b3-4bb3-4443-97fe-f9b08ce1ab29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:03.813540 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d70b3-4bb3-4443-97fe-f9b08ce1ab29 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:03.831704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-8e0d70b3-4bb3-4443-97fe-f9b08ce1ab29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:03.831704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:03.831704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:03.831704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:03.831704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:03.831704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:03.831704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:03.831704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:03.831704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:03.831704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:03.831704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:03.831704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:03.831704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:03.831704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:03.831704 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:03.832824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:03.833802 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:03.833802 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:03.833802 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:03.833802 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:08:03.833802 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 643/1283] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:03.840969 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba8459-db64-4a90-be32-a89a1335996b None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:03.947185 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba8459-db64-4a90-be32-a89a1335996b None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:03.983706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766cbe4-be45-4bf2-8226-48de6c0240a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:03.984365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766cbe4-be45-4bf2-8226-48de6c0240a9 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:03.984641 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766cbe4-be45-4bf2-8226-48de6c0240a9 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:03.984912 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766cbe4-be45-4bf2-8226-48de6c0240a9 None admin] PATH_INFO: `/v3/domains` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:03.985627 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766cbe4-be45-4bf2-8226-48de6c0240a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:03.985927 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766cbe4-be45-4bf2-8226-48de6c0240a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:03.987336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766cbe4-be45-4bf2-8226-48de6c0240a9 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:03.998759 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 641/1284] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:04.005989 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d70b3-4bb3-4443-97fe-f9b08ce1ab29 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:04.106364 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d70b3-4bb3-4443-97fe-f9b08ce1ab29 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:04.121136 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:04.151843 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532af2f4-cc1d-4c73-baf1-a504dc3405f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:04.152639 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532af2f4-cc1d-4c73-baf1-a504dc3405f3 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:04.152925 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532af2f4-cc1d-4c73-baf1-a504dc3405f3 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:04.153198 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532af2f4-cc1d-4c73-baf1-a504dc3405f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:04.160292 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532af2f4-cc1d-4c73-baf1-a504dc3405f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:04.161765 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532af2f4-cc1d-4c73-baf1-a504dc3405f3 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:04.176032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application [None req-532af2f4-cc1d-4c73-baf1-a504dc3405f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:04.176032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:04.176032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:04.176032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:04.176032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:04.176032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:04.176032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:04.176032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:04.176032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:04.176032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:04.176032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:04.176032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:04.176032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:08:04.176032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:08:04.176032 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application with Lock( May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:04.177103 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:04.178414 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:08:04.178414 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:08:04.178414 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:08:04.178414 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:08:04.178414 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:08:04.178414 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:08:04.178414 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:04.178414 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: ERROR keystone.server.flask.application May 19 20:08:04.182442 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 644/1285] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:08:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:04.183885 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766cbe4-be45-4bf2-8226-48de6c0240a9 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:04.246083 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:04.299937 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766cbe4-be45-4bf2-8226-48de6c0240a9 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:04.336820 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef0d2e-1013-493d-b7c8-c94a764bb6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:04.337485 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef0d2e-1013-493d-b7c8-c94a764bb6b6 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:04.337769 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef0d2e-1013-493d-b7c8-c94a764bb6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:04.338039 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef0d2e-1013-493d-b7c8-c94a764bb6b6 None admin] PATH_INFO: `/v3/projects` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:04.338863 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef0d2e-1013-493d-b7c8-c94a764bb6b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:04.339171 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef0d2e-1013-493d-b7c8-c94a764bb6b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:04.340599 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef0d2e-1013-493d-b7c8-c94a764bb6b6 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:04.354589 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 642/1286] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:08:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:04.361113 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532af2f4-cc1d-4c73-baf1-a504dc3405f3 None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:04.466956 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532af2f4-cc1d-4c73-baf1-a504dc3405f3 None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:04.504314 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec838d-b07c-4994-b82e-7295b15b7205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:04.504966 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec838d-b07c-4994-b82e-7295b15b7205 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:04.505239 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec838d-b07c-4994-b82e-7295b15b7205 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:04.505512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec838d-b07c-4994-b82e-7295b15b7205 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:04.506493 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec838d-b07c-4994-b82e-7295b15b7205 None admin] RBAC: Adding query filter params (role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66, scope.project.id=ba22fdc274b74521847e5fceb2d307b1, user.id=28953289ea704853b1c0c7049915633d) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:04.506783 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec838d-b07c-4994-b82e-7295b15b7205 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:04.508230 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec838d-b07c-4994-b82e-7295b15b7205 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:04.514300 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 645/1287] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:08:04 2022] GET /identity/v3/role_assignments?user.id=28953289ea704853b1c0c7049915633d&scope.project.id=ba22fdc274b74521847e5fceb2d307b1&role.id=b34e4ad4fe7b4a60bd2fc669e2b31a66 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:06.804765 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a82fd5-a45c-4a70-875e-6015cc29792c None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:06.805660 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a82fd5-a45c-4a70-875e-6015cc29792c None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:06.805660 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a82fd5-a45c-4a70-875e-6015cc29792c None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:06.808039 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 643/1288] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:08:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:06.814359 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339b4e4-e50e-42ae-8232-6018c347117e None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:06.814809 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339b4e4-e50e-42ae-8232-6018c347117e None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:06.814916 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339b4e4-e50e-42ae-8232-6018c347117e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:06.834476 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-d339b4e4-e50e-42ae-8232-6018c347117e None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:06.835607 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-d339b4e4-e50e-42ae-8232-6018c347117e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:06.870094 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:06.910961 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 646/1289] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:08:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:06.934298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a82fd5-a45c-4a70-875e-6015cc29792c None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:07.047111 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a82fd5-a45c-4a70-875e-6015cc29792c None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:07.058937 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:07.086501 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290b8a2-bbcc-45df-b11a-f66fdce39ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:07.087239 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290b8a2-bbcc-45df-b11a-f66fdce39ca2 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:07.087526 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290b8a2-bbcc-45df-b11a-f66fdce39ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:07.087803 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290b8a2-bbcc-45df-b11a-f66fdce39ca2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:07.088463 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290b8a2-bbcc-45df-b11a-f66fdce39ca2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:07.089840 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290b8a2-bbcc-45df-b11a-f66fdce39ca2 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-5290b8a2-bbcc-45df-b11a-f66fdce39ca2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:07.102665 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:07.103949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:07.104883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:07.104883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:08:07.104883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:07.104883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:07.104883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:07.104883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:07.104883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:07.104883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:07.104883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:07.104883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:07.104883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:07.104883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:07.104883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:08:07.104883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:08:07.104883 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 644/1290] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:08:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:07.110327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339b4e4-e50e-42ae-8232-6018c347117e None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:07.221380 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339b4e4-e50e-42ae-8232-6018c347117e None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:07.262821 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea096782-d56f-41a3-900f-b3d550813094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:07.263504 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea096782-d56f-41a3-900f-b3d550813094 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:07.263771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea096782-d56f-41a3-900f-b3d550813094 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:07.264043 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea096782-d56f-41a3-900f-b3d550813094 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:07.264759 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea096782-d56f-41a3-900f-b3d550813094 None admin] RBAC: Adding query filter params (name=placement) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:07.265063 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea096782-d56f-41a3-900f-b3d550813094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:07.266479 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea096782-d56f-41a3-900f-b3d550813094 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:07.271671 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 647/1291] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 20:08:07 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:07.277858 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290b8a2-bbcc-45df-b11a-f66fdce39ca2 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:07.385710 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290b8a2-bbcc-45df-b11a-f66fdce39ca2 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:07.427909 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac93edb-03f8-4b2b-9dcd-9ef0b9af88f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:07.428672 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac93edb-03f8-4b2b-9dcd-9ef0b9af88f6 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:07.428977 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac93edb-03f8-4b2b-9dcd-9ef0b9af88f6 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:07.429261 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac93edb-03f8-4b2b-9dcd-9ef0b9af88f6 None admin] PATH_INFO: `/v3/services` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:07.429975 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac93edb-03f8-4b2b-9dcd-9ef0b9af88f6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:07.430292 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac93edb-03f8-4b2b-9dcd-9ef0b9af88f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:07.431688 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac93edb-03f8-4b2b-9dcd-9ef0b9af88f6 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:07.436269 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 645/1292] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 20:08:07 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:09.672497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acb627-49b5-46da-b52d-3d8fa0903556 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:09.673392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acb627-49b5-46da-b52d-3d8fa0903556 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:09.673392 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acb627-49b5-46da-b52d-3d8fa0903556 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:09.675716 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 648/1293] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:08:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:09.682229 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e66ba3-53e2-4947-af33-ff8bcd05013b None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:09.683002 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e66ba3-53e2-4947-af33-ff8bcd05013b None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:09.683157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e66ba3-53e2-4947-af33-ff8bcd05013b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:09.704744 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-f3e66ba3-53e2-4947-af33-ff8bcd05013b None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:09.705935 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-f3e66ba3-53e2-4947-af33-ff8bcd05013b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:09.738316 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:09.781182 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 646/1294] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:08:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:09.804598 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acb627-49b5-46da-b52d-3d8fa0903556 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:09.837874 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:09.911792 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acb627-49b5-46da-b52d-3d8fa0903556 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:09.947751 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b807bd-ea0a-4818-890c-004d6197d5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:09.948419 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b807bd-ea0a-4818-890c-004d6197d5d1 None admin] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:09.948705 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b807bd-ea0a-4818-890c-004d6197d5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:09.948977 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b807bd-ea0a-4818-890c-004d6197d5d1 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:09.949714 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b807bd-ea0a-4818-890c-004d6197d5d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:09.951140 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b807bd-ea0a-4818-890c-004d6197d5d1 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:09.972423 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 649/1295] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:08:09 2022] POST /identity/v3/services => generated 246 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:08:12.203555 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e504aa-a10a-42c0-8ffe-d270509d2b24 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:12.204320 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e504aa-a10a-42c0-8ffe-d270509d2b24 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:12.204320 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e504aa-a10a-42c0-8ffe-d270509d2b24 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:12.206581 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 647/1296] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:08:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:12.212445 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ad0e1-b27a-49e3-a12e-b499a154dbcb None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:12.212911 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ad0e1-b27a-49e3-a12e-b499a154dbcb None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:12.212911 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ad0e1-b27a-49e3-a12e-b499a154dbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:12.232510 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-a30ad0e1-b27a-49e3-a12e-b499a154dbcb None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:12.233796 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-a30ad0e1-b27a-49e3-a12e-b499a154dbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:12.310098 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 650/1297] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:08:12 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:12.333786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e504aa-a10a-42c0-8ffe-d270509d2b24 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:12.464165 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e504aa-a10a-42c0-8ffe-d270509d2b24 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:12.502092 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc48dd6-7ee7-403f-a855-f948e5008424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:12.502972 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc48dd6-7ee7-403f-a855-f948e5008424 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:12.503263 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc48dd6-7ee7-403f-a855-f948e5008424 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:12.503534 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc48dd6-7ee7-403f-a855-f948e5008424 None admin] PATH_INFO: `/v3/services/placement` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:12.504199 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc48dd6-7ee7-403f-a855-f948e5008424 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:12.505620 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc48dd6-7ee7-403f-a855-f948e5008424 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-adc48dd6-7ee7-403f-a855-f948e5008424 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:12.518314 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:12.519588 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:12.521009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:12.521009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:08:12.521009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:12.521009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:12.521009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:12.521009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:12.521009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:12.521009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:12.521009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:12.521009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:12.521009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:12.521009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:12.521009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:08:12.521009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:08:12.521009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 648/1298] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:08:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:12.526769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ad0e1-b27a-49e3-a12e-b499a154dbcb None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:12.547108 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:12.636393 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ad0e1-b27a-49e3-a12e-b499a154dbcb None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:12.672831 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920499e-9ca9-41b5-8c05-e0b232e285ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:12.673492 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920499e-9ca9-41b5-8c05-e0b232e285ef None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:12.673771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920499e-9ca9-41b5-8c05-e0b232e285ef None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:12.674042 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920499e-9ca9-41b5-8c05-e0b232e285ef None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:12.674773 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920499e-9ca9-41b5-8c05-e0b232e285ef None admin] RBAC: Adding query filter params (name=placement) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:12.675077 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920499e-9ca9-41b5-8c05-e0b232e285ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:12.676479 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920499e-9ca9-41b5-8c05-e0b232e285ef None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:12.683507 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 651/1299] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 20:08:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:12.690487 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc48dd6-7ee7-403f-a855-f948e5008424 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:12.729849 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:12.805471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc48dd6-7ee7-403f-a855-f948e5008424 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:12.841144 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828dd38e-8c93-4cbc-8ab2-18fc0d85a5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:12.841794 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dd38e-8c93-4cbc-8ab2-18fc0d85a5b3 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:12.842075 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dd38e-8c93-4cbc-8ab2-18fc0d85a5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:12.842387 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dd38e-8c93-4cbc-8ab2-18fc0d85a5b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:12.843176 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828dd38e-8c93-4cbc-8ab2-18fc0d85a5b3 None admin] RBAC: Adding query filter params (interface=public, service_id=24cda6cdf348445084612dc9927cfb68, region_id=RegionOne) {{(pid=214315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:12.843486 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828dd38e-8c93-4cbc-8ab2-18fc0d85a5b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:12.845183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828dd38e-8c93-4cbc-8ab2-18fc0d85a5b3 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:12.850442 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 649/1300] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:08:12 2022] GET /identity/v3/endpoints?service_id=24cda6cdf348445084612dc9927cfb68&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:12.856590 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920499e-9ca9-41b5-8c05-e0b232e285ef None admin] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:12.948460 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:12.971769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920499e-9ca9-41b5-8c05-e0b232e285ef None admin] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:13.007978 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47248342-a938-4620-a5bd-baf4b8459055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:13.008574 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47248342-a938-4620-a5bd-baf4b8459055 None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:13.008813 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47248342-a938-4620-a5bd-baf4b8459055 None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:13.009110 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47248342-a938-4620-a5bd-baf4b8459055 None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:13.009795 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47248342-a938-4620-a5bd-baf4b8459055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:13.011196 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47248342-a938-4620-a5bd-baf4b8459055 None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:13.018171 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 652/1301] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:08:12 2022] GET /identity/v3/services => generated 2191 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:08:15.216745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceafe55-76d6-47c9-90c0-375b04fb3292 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.217394 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceafe55-76d6-47c9-90c0-375b04fb3292 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.217394 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceafe55-76d6-47c9-90c0-375b04fb3292 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.219088 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 650/1302] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:15.225512 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0adbaa-76d1-4abf-8271-784a3596b262 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.226281 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0adbaa-76d1-4abf-8271-784a3596b262 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.226445 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0adbaa-76d1-4abf-8271-784a3596b262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.255522 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-7e0adbaa-76d1-4abf-8271-784a3596b262 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:15.257145 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-7e0adbaa-76d1-4abf-8271-784a3596b262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:15.330093 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 653/1303] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:08:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:15.355160 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceafe55-76d6-47c9-90c0-375b04fb3292 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:15.467983 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:15.481748 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceafe55-76d6-47c9-90c0-375b04fb3292 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:15.521455 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242730ea-6fac-4828-a6ac-00ca31591eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:15.522801 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242730ea-6fac-4828-a6ac-00ca31591eb0 None admin] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.523100 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242730ea-6fac-4828-a6ac-00ca31591eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.523379 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242730ea-6fac-4828-a6ac-00ca31591eb0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.524048 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242730ea-6fac-4828-a6ac-00ca31591eb0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:15.525435 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242730ea-6fac-4828-a6ac-00ca31591eb0 None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application [None req-242730ea-6fac-4828-a6ac-00ca31591eb0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:15.537878 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:15.539189 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:15.540366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:15.540366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application with Lock( May 19 20:08:15.540366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:15.540366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:15.540366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:15.540366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:15.540366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:15.540366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:15.540366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:15.540366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:15.540366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:15.540366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:15.540366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:08:15.540366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: ERROR keystone.server.flask.application May 19 20:08:15.540366 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 651/1304] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:08:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:15.546759 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0adbaa-76d1-4abf-8271-784a3596b262 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:15.624881 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:15.649625 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0adbaa-76d1-4abf-8271-784a3596b262 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:15.682333 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dc542-984c-412f-96e7-c419456a02fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:15.683003 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dc542-984c-412f-96e7-c419456a02fa None admin] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.683288 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dc542-984c-412f-96e7-c419456a02fa None admin] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.683566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dc542-984c-412f-96e7-c419456a02fa None admin] PATH_INFO: `/v3/services` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.684279 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dc542-984c-412f-96e7-c419456a02fa None admin] RBAC: Adding query filter params (name=placement) {{(pid=214314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:15.684588 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dc542-984c-412f-96e7-c419456a02fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:15.685974 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dc542-984c-412f-96e7-c419456a02fa None admin] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:15.691801 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 654/1305] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 20:08:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:15.697908 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242730ea-6fac-4828-a6ac-00ca31591eb0 None admin] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:15.826727 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242730ea-6fac-4828-a6ac-00ca31591eb0 None admin] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:15.862760 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe18c58-d478-4d78-8299-3fcd42a3fa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'def9ab1b20e24b65870b4a851d7650b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:15.863421 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe18c58-d478-4d78-8299-3fcd42a3fa5b None admin] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.863706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe18c58-d478-4d78-8299-3fcd42a3fa5b None admin] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.863978 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe18c58-d478-4d78-8299-3fcd42a3fa5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.864726 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe18c58-d478-4d78-8299-3fcd42a3fa5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:15.866121 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe18c58-d478-4d78-8299-3fcd42a3fa5b None admin] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:15.888786 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 652/1306] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:08:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:08:25.688313 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=30 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:25.884603 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:22.526796 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045a6a8-082e-423e-a999-fed06f546fd1 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:22.527950 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045a6a8-082e-423e-a999-fed06f546fd1 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:22.528471 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045a6a8-082e-423e-a999-fed06f546fd1 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:22.534548 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 655/1307] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:09:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:22.535949 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda16cd3-06d7-4190-8821-96d451c8fd19 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:22.536408 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda16cd3-06d7-4190-8821-96d451c8fd19 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:22.536501 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda16cd3-06d7-4190-8821-96d451c8fd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:22.680903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-bda16cd3-06d7-4190-8821-96d451c8fd19 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:22.682136 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-bda16cd3-06d7-4190-8821-96d451c8fd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:22.931100 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 653/1308] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:09:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:09:22.989143 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb3fc1-a51d-4466-af68-920154de9c44 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:22.989485 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb3fc1-a51d-4466-af68-920154de9c44 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:22.989847 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb3fc1-a51d-4466-af68-920154de9c44 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:22.991853 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 656/1309] 10.222.0.84 () {54 vars in 964 bytes} [Thu May 19 20:09:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:23.009079 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea096039-aeb4-43d5-a04d-64c16465942e None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:23.009359 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea096039-aeb4-43d5-a04d-64c16465942e None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:23.009794 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea096039-aeb4-43d5-a04d-64c16465942e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:23.168449 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-ea096039-aeb4-43d5-a04d-64c16465942e None None] MFA Rules not processed for user `e2c5489b02604fcbae8dd5eff816d39d`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:23.169810 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-ea096039-aeb4-43d5-a04d-64c16465942e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:23.206718 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:23.321879 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 654/1310] 10.222.0.84 () {58 vars in 1061 bytes} [Thu May 19 20:09:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:09:23.330829 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb3fc1-a51d-4466-af68-920154de9c44 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:23.332690 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-e3fb3fc1-a51d-4466-af68-920154de9c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:23.518079 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb3fc1-a51d-4466-af68-920154de9c44 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:23.603764 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34af999-a7fd-48ff-a612-4849b24f6b7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c5489b02604fcbae8dd5eff816d39d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:23.604417 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34af999-a7fd-48ff-a612-4849b24f6b7e service neutron] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:23.604687 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34af999-a7fd-48ff-a612-4849b24f6b7e service neutron] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:23.604955 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34af999-a7fd-48ff-a612-4849b24f6b7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:23.606632 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-b34af999-a7fd-48ff-a612-4849b24f6b7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:23.667557 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:23.730983 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34af999-a7fd-48ff-a612-4849b24f6b7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:23.732591 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34af999-a7fd-48ff-a612-4849b24f6b7e service neutron] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:23.753560 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 657/1311] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:09:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:09:26.631696 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19017ebe-0678-417b-b820-c2e68062f8a5 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:26.632243 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19017ebe-0678-417b-b820-c2e68062f8a5 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:26.632243 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19017ebe-0678-417b-b820-c2e68062f8a5 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:26.637679 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 655/1312] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:09:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:26.639293 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecdc50-08e8-4d95-a0e7-b91c92a13188 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:26.639752 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecdc50-08e8-4d95-a0e7-b91c92a13188 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:26.639855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecdc50-08e8-4d95-a0e7-b91c92a13188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:26.734004 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-b3ecdc50-08e8-4d95-a0e7-b91c92a13188 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:26.735300 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-b3ecdc50-08e8-4d95-a0e7-b91c92a13188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:26.846109 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 658/1313] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:09:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:09:26.885465 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19017ebe-0678-417b-b820-c2e68062f8a5 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:26.887053 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-19017ebe-0678-417b-b820-c2e68062f8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:27.009679 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19017ebe-0678-417b-b820-c2e68062f8a5 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:27.106223 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:27.109319 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf77ac1-1363-44d1-a84e-85728bbe6108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c5489b02604fcbae8dd5eff816d39d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:27.110019 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf77ac1-1363-44d1-a84e-85728bbe6108 service neutron] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:27.110333 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf77ac1-1363-44d1-a84e-85728bbe6108 service neutron] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:27.110594 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf77ac1-1363-44d1-a84e-85728bbe6108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:27.112899 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-5cf77ac1-1363-44d1-a84e-85728bbe6108 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:27.257311 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf77ac1-1363-44d1-a84e-85728bbe6108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:27.258838 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf77ac1-1363-44d1-a84e-85728bbe6108 service neutron] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:27.282656 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 656/1314] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:09:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:09:29.835702 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfe57c-569e-48a7-8e80-be5bf77ba599 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:29.836313 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfe57c-569e-48a7-8e80-be5bf77ba599 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:29.836313 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfe57c-569e-48a7-8e80-be5bf77ba599 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:29.837631 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 659/1315] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:09:29.844745 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb937e1-2e63-459b-897f-39702061e27f None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:29.845482 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb937e1-2e63-459b-897f-39702061e27f None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:29.845673 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb937e1-2e63-459b-897f-39702061e27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:29.915072 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-2cb937e1-2e63-459b-897f-39702061e27f None None] MFA Rules not processed for user `f1a2f68967994de49bab3d1e94aa44dc`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:29.916302 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-2cb937e1-2e63-459b-897f-39702061e27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:29.978487 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:30.024308 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 657/1316] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:09:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:09:30.067335 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfe57c-569e-48a7-8e80-be5bf77ba599 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:30.069879 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-0ddfe57c-569e-48a7-8e80-be5bf77ba599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:30.080082 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:30.187799 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfe57c-569e-48a7-8e80-be5bf77ba599 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:30.266558 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c259cbb-da2d-4d4b-a476-7d1103153b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c5489b02604fcbae8dd5eff816d39d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:30.267208 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c259cbb-da2d-4d4b-a476-7d1103153b0a service neutron] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:30.267492 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c259cbb-da2d-4d4b-a476-7d1103153b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:30.267769 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c259cbb-da2d-4d4b-a476-7d1103153b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:30.269432 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-6c259cbb-da2d-4d4b-a476-7d1103153b0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:30.345430 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:30.400234 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c259cbb-da2d-4d4b-a476-7d1103153b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:30.401715 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c259cbb-da2d-4d4b-a476-7d1103153b0a service neutron] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:30.423389 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 660/1317] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:09:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:09:33.207298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660b4f1-0836-4c1c-8a24-0faee120cf0b None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:33.207857 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660b4f1-0836-4c1c-8a24-0faee120cf0b None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:33.207857 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660b4f1-0836-4c1c-8a24-0faee120cf0b None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:33.209502 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 658/1318] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:09:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:33.215206 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcde8e3-487b-4b09-af04-355d7c01b18f None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:33.215694 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcde8e3-487b-4b09-af04-355d7c01b18f None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:33.215778 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcde8e3-487b-4b09-af04-355d7c01b18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:33.281736 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-ddcde8e3-487b-4b09-af04-355d7c01b18f None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:33.282973 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-ddcde8e3-487b-4b09-af04-355d7c01b18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:33.392096 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 661/1319] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:09:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:09:33.431368 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660b4f1-0836-4c1c-8a24-0faee120cf0b None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:33.432911 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-5660b4f1-0836-4c1c-8a24-0faee120cf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:33.546274 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660b4f1-0836-4c1c-8a24-0faee120cf0b None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:33.625866 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8da27b-5774-4e9a-a6ed-f77c9aa6ad57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c5489b02604fcbae8dd5eff816d39d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:33.626553 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8da27b-5774-4e9a-a6ed-f77c9aa6ad57 service neutron] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:33.626814 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8da27b-5774-4e9a-a6ed-f77c9aa6ad57 service neutron] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:33.627114 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8da27b-5774-4e9a-a6ed-f77c9aa6ad57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:33.628764 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-6e8da27b-5774-4e9a-a6ed-f77c9aa6ad57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:33.636787 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:33.748809 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8da27b-5774-4e9a-a6ed-f77c9aa6ad57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:33.750252 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8da27b-5774-4e9a-a6ed-f77c9aa6ad57 service neutron] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:33.772062 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 659/1320] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:09:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:09:36.741578 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4072c6-2749-4d8e-a40e-da6023337447 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:36.742142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4072c6-2749-4d8e-a40e-da6023337447 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:36.742142 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4072c6-2749-4d8e-a40e-da6023337447 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:36.743569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 662/1321] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:09:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:36.749019 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50725b-6874-40d7-87ea-ba003320f4c0 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:36.749445 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50725b-6874-40d7-87ea-ba003320f4c0 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:36.749538 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50725b-6874-40d7-87ea-ba003320f4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:36.821936 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-9a50725b-6874-40d7-87ea-ba003320f4c0 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:36.823159 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-9a50725b-6874-40d7-87ea-ba003320f4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:36.885529 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:36.935009 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 660/1322] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:09:36.986277 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c052d-a7ef-43d6-9778-768d1e33637d None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:36.986772 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c052d-a7ef-43d6-9778-768d1e33637d None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:36.987120 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c052d-a7ef-43d6-9778-768d1e33637d None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:36.988582 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 663/1323] 10.222.0.84 () {54 vars in 964 bytes} [Thu May 19 20:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:36.995429 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45944f-1ed9-4349-8fa2-50c99b454a15 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:36.995797 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45944f-1ed9-4349-8fa2-50c99b454a15 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:36.996152 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45944f-1ed9-4349-8fa2-50c99b454a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:37.073890 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-7b45944f-1ed9-4349-8fa2-50c99b454a15 None None] MFA Rules not processed for user `e2c5489b02604fcbae8dd5eff816d39d`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:37.075027 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-7b45944f-1ed9-4349-8fa2-50c99b454a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:37.152850 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 661/1324] 10.222.0.84 () {58 vars in 1061 bytes} [Thu May 19 20:09:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:09:37.162058 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c052d-a7ef-43d6-9778-768d1e33637d None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:37.163743 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-ca4c052d-a7ef-43d6-9778-768d1e33637d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:37.185785 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:37.277092 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c052d-a7ef-43d6-9778-768d1e33637d None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:37.354882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321508c-dc42-412b-b84d-01e1bed5896e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c5489b02604fcbae8dd5eff816d39d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:37.355445 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321508c-dc42-412b-b84d-01e1bed5896e service neutron] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:37.355718 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321508c-dc42-412b-b84d-01e1bed5896e service neutron] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:37.355981 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321508c-dc42-412b-b84d-01e1bed5896e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:37.357822 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-d321508c-dc42-412b-b84d-01e1bed5896e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:37.433679 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:37.474197 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321508c-dc42-412b-b84d-01e1bed5896e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:37.475702 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321508c-dc42-412b-b84d-01e1bed5896e service neutron] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:37.494518 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 664/1325] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:09:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:09:39.979780 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ae764-440d-4e2d-85a4-16c0d1ed22f0 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:39.980351 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ae764-440d-4e2d-85a4-16c0d1ed22f0 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:39.980452 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ae764-440d-4e2d-85a4-16c0d1ed22f0 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:39.981845 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 662/1326] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:39.987488 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba6230-19ee-4f68-b72d-6eb12c28f6bb None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:39.987969 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba6230-19ee-4f68-b72d-6eb12c28f6bb None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:39.988066 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba6230-19ee-4f68-b72d-6eb12c28f6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:40.055233 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-46ba6230-19ee-4f68-b72d-6eb12c28f6bb None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:40.056440 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-46ba6230-19ee-4f68-b72d-6eb12c28f6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:40.159355 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 665/1327] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:09:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:09:40.195221 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ae764-440d-4e2d-85a4-16c0d1ed22f0 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:40.196738 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-024ae764-440d-4e2d-85a4-16c0d1ed22f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:40.259264 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:40.307968 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ae764-440d-4e2d-85a4-16c0d1ed22f0 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:40.382047 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39fbea-f9ee-4768-9dfb-ee5c4ccab957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c5489b02604fcbae8dd5eff816d39d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:40.382738 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39fbea-f9ee-4768-9dfb-ee5c4ccab957 service neutron] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:40.383054 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39fbea-f9ee-4768-9dfb-ee5c4ccab957 service neutron] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:40.383298 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39fbea-f9ee-4768-9dfb-ee5c4ccab957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:40.384996 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-2c39fbea-f9ee-4768-9dfb-ee5c4ccab957 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:40.504635 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:40.508070 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c39fbea-f9ee-4768-9dfb-ee5c4ccab957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:40.510255 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c39fbea-f9ee-4768-9dfb-ee5c4ccab957 service neutron] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:40.529797 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 663/1328] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:09:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 20:09:42.768751 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc07317-d046-485a-9611-c0c4f4166f09 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:42.769665 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc07317-d046-485a-9611-c0c4f4166f09 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:42.769665 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc07317-d046-485a-9611-c0c4f4166f09 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:42.772378 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 666/1329] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:09:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:42.777833 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce04953-eab5-4117-8188-0835136e98a0 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:42.778287 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce04953-eab5-4117-8188-0835136e98a0 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:42.778420 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce04953-eab5-4117-8188-0835136e98a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:42.845876 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-5ce04953-eab5-4117-8188-0835136e98a0 None None] MFA Rules not processed for user `f1a2f68967994de49bab3d1e94aa44dc`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:42.847121 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-5ce04953-eab5-4117-8188-0835136e98a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:42.960320 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 664/1330] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:09:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:09:43.011533 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc07317-d046-485a-9611-c0c4f4166f09 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:43.014328 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-dbc07317-d046-485a-9611-c0c4f4166f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:43.052794 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:43.145286 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc07317-d046-485a-9611-c0c4f4166f09 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:43.224882 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba939387-3fa4-408b-b0fc-941197b98a5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c5489b02604fcbae8dd5eff816d39d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:43.225545 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba939387-3fa4-408b-b0fc-941197b98a5e service neutron] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:43.225832 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba939387-3fa4-408b-b0fc-941197b98a5e service neutron] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:43.226106 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba939387-3fa4-408b-b0fc-941197b98a5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:43.227797 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-ba939387-3fa4-408b-b0fc-941197b98a5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:43.317497 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:43.358534 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba939387-3fa4-408b-b0fc-941197b98a5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:43.359996 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba939387-3fa4-408b-b0fc-941197b98a5e service neutron] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:43.378590 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 667/1331] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:09:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:09:45.852304 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d7c67-fd83-4e2f-b73e-3aa83fe57a13 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:45.853365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d7c67-fd83-4e2f-b73e-3aa83fe57a13 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:45.853365 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d7c67-fd83-4e2f-b73e-3aa83fe57a13 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:45.855472 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 665/1332] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:09:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:45.862080 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d70cd5-0c12-4bbc-b3fa-c8f2d19d6b92 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:45.862566 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d70cd5-0c12-4bbc-b3fa-c8f2d19d6b92 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:45.862671 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d70cd5-0c12-4bbc-b3fa-c8f2d19d6b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:45.937754 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-73d70cd5-0c12-4bbc-b3fa-c8f2d19d6b92 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:45.939015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-73d70cd5-0c12-4bbc-b3fa-c8f2d19d6b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:46.043370 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 668/1333] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:09:46.080099 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d7c67-fd83-4e2f-b73e-3aa83fe57a13 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:46.081679 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-0e8d7c67-fd83-4e2f-b73e-3aa83fe57a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:46.160992 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:46.198884 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d7c67-fd83-4e2f-b73e-3aa83fe57a13 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:46.285349 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777aeae0-8e3f-48df-9e4f-940407219326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c5489b02604fcbae8dd5eff816d39d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:46.286013 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777aeae0-8e3f-48df-9e4f-940407219326 service neutron] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:46.286300 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777aeae0-8e3f-48df-9e4f-940407219326 service neutron] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:46.286607 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777aeae0-8e3f-48df-9e4f-940407219326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:46.288388 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-777aeae0-8e3f-48df-9e4f-940407219326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:46.418592 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777aeae0-8e3f-48df-9e4f-940407219326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:46.419971 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777aeae0-8e3f-48df-9e4f-940407219326 service neutron] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:46.431376 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:46.441616 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 666/1334] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:09:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:09:48.751824 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699f6c6-b450-4020-9f15-c36390fe3120 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:48.752408 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699f6c6-b450-4020-9f15-c36390fe3120 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:48.752408 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699f6c6-b450-4020-9f15-c36390fe3120 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:48.754535 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 669/1335] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:09:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:48.763894 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef26c7-d9f4-4a2f-a8e7-68230e2ffbe6 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:48.764584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef26c7-d9f4-4a2f-a8e7-68230e2ffbe6 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:48.764740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef26c7-d9f4-4a2f-a8e7-68230e2ffbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:48.839744 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-63ef26c7-d9f4-4a2f-a8e7-68230e2ffbe6 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:48.840890 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-63ef26c7-d9f4-4a2f-a8e7-68230e2ffbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:48.943650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 667/1336] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:09:48.987798 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699f6c6-b450-4020-9f15-c36390fe3120 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:48.989413 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-6699f6c6-b450-4020-9f15-c36390fe3120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:49.033327 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:49.102063 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699f6c6-b450-4020-9f15-c36390fe3120 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:49.193375 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383b785-99b4-4bed-a4d3-e093032158e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c5489b02604fcbae8dd5eff816d39d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:49.194163 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383b785-99b4-4bed-a4d3-e093032158e2 service neutron] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:49.194475 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383b785-99b4-4bed-a4d3-e093032158e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:49.194754 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383b785-99b4-4bed-a4d3-e093032158e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:49.196457 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-1383b785-99b4-4bed-a4d3-e093032158e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:49.282891 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:49.311183 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383b785-99b4-4bed-a4d3-e093032158e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:49.312662 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383b785-99b4-4bed-a4d3-e093032158e2 service neutron] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:49.332125 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 670/1337] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:09:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:09:51.884548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d4a16-6652-4bca-b067-2909a8170cb3 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:51.885158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d4a16-6652-4bca-b067-2909a8170cb3 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:51.885158 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d4a16-6652-4bca-b067-2909a8170cb3 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:51.886521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 668/1338] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:51.892800 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5cd85-76cd-45cc-bad5-29fa59eb5f0b None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:51.893584 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5cd85-76cd-45cc-bad5-29fa59eb5f0b None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:51.893771 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5cd85-76cd-45cc-bad5-29fa59eb5f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:51.980734 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-2ac5cd85-76cd-45cc-bad5-29fa59eb5f0b None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:51.982569 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-2ac5cd85-76cd-45cc-bad5-29fa59eb5f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:52.094021 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 671/1339] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:09:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:09:52.131157 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d4a16-6652-4bca-b067-2909a8170cb3 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:52.133484 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-3e5d4a16-6652-4bca-b067-2909a8170cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:52.222744 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:52.252584 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d4a16-6652-4bca-b067-2909a8170cb3 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:52.331507 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8924d-b936-4845-bf0d-e56d279921d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c5489b02604fcbae8dd5eff816d39d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:52.332160 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8924d-b936-4845-bf0d-e56d279921d1 service neutron] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:52.332440 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8924d-b936-4845-bf0d-e56d279921d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:52.332721 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8924d-b936-4845-bf0d-e56d279921d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:52.334390 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-cbb8924d-b936-4845-bf0d-e56d279921d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:52.468882 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8924d-b936-4845-bf0d-e56d279921d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:52.470372 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8924d-b936-4845-bf0d-e56d279921d1 service neutron] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:52.489201 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 669/1340] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:09:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:09:56.730596 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbe2e7-a714-4c6b-be91-8d68cd69c5ef None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:56.731225 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbe2e7-a714-4c6b-be91-8d68cd69c5ef None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:56.731225 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbe2e7-a714-4c6b-be91-8d68cd69c5ef None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:56.732525 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 672/1341] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:56.737952 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4b123-9a2d-4a5b-9a02-c4aa9dd5cee0 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:56.738444 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4b123-9a2d-4a5b-9a02-c4aa9dd5cee0 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:56.738557 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4b123-9a2d-4a5b-9a02-c4aa9dd5cee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:56.745410 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=100 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:56.816484 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-01b4b123-9a2d-4a5b-9a02-c4aa9dd5cee0 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:56.817637 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-01b4b123-9a2d-4a5b-9a02-c4aa9dd5cee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:56.924805 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 670/1342] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:09:56.963016 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbe2e7-a714-4c6b-be91-8d68cd69c5ef None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:56.965713 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-5ecbe2e7-a714-4c6b-be91-8d68cd69c5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:57.018991 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:57.067906 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbe2e7-a714-4c6b-be91-8d68cd69c5ef None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:57.071939 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8ff02-5c2f-4407-93d3-f091a0d836b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c5489b02604fcbae8dd5eff816d39d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:57.072620 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8ff02-5c2f-4407-93d3-f091a0d836b1 service neutron] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:57.072905 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8ff02-5c2f-4407-93d3-f091a0d836b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:57.073181 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8ff02-5c2f-4407-93d3-f091a0d836b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:57.084599 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8ff02-5c2f-4407-93d3-f091a0d836b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:57.088645 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8ff02-5c2f-4407-93d3-f091a0d836b1 service neutron] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:57.101129 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 673/1343] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:09:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:09:59.974068 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae5276-b2eb-4b34-a0dd-3d9ea8a9cdc5 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:59.974706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae5276-b2eb-4b34-a0dd-3d9ea8a9cdc5 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:59.974706 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae5276-b2eb-4b34-a0dd-3d9ea8a9cdc5 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:59.975993 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 671/1344] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:59.981323 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95032313-073f-418d-9abc-67ca63b7dca4 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:59.981765 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95032313-073f-418d-9abc-67ca63b7dca4 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:59.981883 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95032313-073f-418d-9abc-67ca63b7dca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:00.011421 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-95032313-073f-418d-9abc-67ca63b7dca4 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:00.012650 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-95032313-073f-418d-9abc-67ca63b7dca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:00.096739 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 674/1345] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:09:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:10:00.136596 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae5276-b2eb-4b34-a0dd-3d9ea8a9cdc5 None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:00.149260 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae5276-b2eb-4b34-a0dd-3d9ea8a9cdc5 None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:00.153901 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0921e-ccf0-4d85-bea7-52d7d52bb2d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c5489b02604fcbae8dd5eff816d39d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:00.154591 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0921e-ccf0-4d85-bea7-52d7d52bb2d6 service neutron] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:00.154876 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0921e-ccf0-4d85-bea7-52d7d52bb2d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:00.155156 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0921e-ccf0-4d85-bea7-52d7d52bb2d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:00.165067 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0921e-ccf0-4d85-bea7-52d7d52bb2d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:00.166548 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0921e-ccf0-4d85-bea7-52d7d52bb2d6 service neutron] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:00.176336 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 672/1346] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:10:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:10:04.372723 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e381921-0b1b-4ee4-aefd-c3934b62b87b None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:04.373224 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e381921-0b1b-4ee4-aefd-c3934b62b87b None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:04.373330 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e381921-0b1b-4ee4-aefd-c3934b62b87b None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:04.374721 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 675/1347] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:04.380148 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee6a8a-3dcf-41c9-8759-02d70f0a4b69 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:04.380757 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee6a8a-3dcf-41c9-8759-02d70f0a4b69 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:04.380757 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee6a8a-3dcf-41c9-8759-02d70f0a4b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:04.398958 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-e8ee6a8a-3dcf-41c9-8759-02d70f0a4b69 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:04.400160 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-e8ee6a8a-3dcf-41c9-8759-02d70f0a4b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:04.408578 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 673/1348] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:10:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:10:04.446220 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e381921-0b1b-4ee4-aefd-c3934b62b87b None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:04.448212 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-7e381921-0b1b-4ee4-aefd-c3934b62b87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:04.463662 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e381921-0b1b-4ee4-aefd-c3934b62b87b None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:04.467694 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f9a6d-134c-4ae7-9299-1b837507cc33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c5489b02604fcbae8dd5eff816d39d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:04.468667 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f9a6d-134c-4ae7-9299-1b837507cc33 service neutron] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:04.468963 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f9a6d-134c-4ae7-9299-1b837507cc33 service neutron] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:04.469179 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f9a6d-134c-4ae7-9299-1b837507cc33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:04.479007 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f9a6d-134c-4ae7-9299-1b837507cc33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:04.480455 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f9a6d-134c-4ae7-9299-1b837507cc33 service neutron] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:04.490301 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 676/1349] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:10:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:10:10.072426 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdce705-fece-4c74-9d6b-ab9dfd6d0158 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:10.073202 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdce705-fece-4c74-9d6b-ab9dfd6d0158 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:10.073653 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdce705-fece-4c74-9d6b-ab9dfd6d0158 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:10.075227 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 674/1350] 10.222.0.84 () {54 vars in 921 bytes} [Thu May 19 20:10:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:10.080387 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4461d-c5b4-4b45-b831-b124556ecc3f None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:10.080916 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4461d-c5b4-4b45-b831-b124556ecc3f None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:10.081015 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4461d-c5b4-4b45-b831-b124556ecc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:10.193790 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-9be4461d-c5b4-4b45-b831-b124556ecc3f None None] MFA Rules not processed for user `28953289ea704853b1c0c7049915633d`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:10.202703 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-9be4461d-c5b4-4b45-b831-b124556ecc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:10.315121 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=100 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:10.366371 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 677/1351] 10.222.0.84 () {58 vars in 1018 bytes} [Thu May 19 20:10:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:10:14.393437 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=72 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:14.811891 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172af4f0-df1c-4130-8931-20d77e00b11e None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:14.812323 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172af4f0-df1c-4130-8931-20d77e00b11e None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:14.812450 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172af4f0-df1c-4130-8931-20d77e00b11e None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:14.814047 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 675/1352] 10.222.0.84 () {54 vars in 921 bytes} [Thu May 19 20:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:14.819490 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b38e4-f4ca-4027-8cf1-6aec6e8013cd None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:14.819971 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b38e4-f4ca-4027-8cf1-6aec6e8013cd None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:14.820101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b38e4-f4ca-4027-8cf1-6aec6e8013cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:14.838960 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-2a7b38e4-f4ca-4027-8cf1-6aec6e8013cd None None] MFA Rules not processed for user `28953289ea704853b1c0c7049915633d`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:14.840158 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-2a7b38e4-f4ca-4027-8cf1-6aec6e8013cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:14.850393 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 678/1353] 10.222.0.84 () {58 vars in 1018 bytes} [Thu May 19 20:10:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:10:16.489246 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9ebc8-1e4c-4ed2-9185-7549c65e69d5 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:16.490231 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9ebc8-1e4c-4ed2-9185-7549c65e69d5 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:16.490231 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9ebc8-1e4c-4ed2-9185-7549c65e69d5 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:16.491304 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 676/1354] 10.222.0.84 () {54 vars in 921 bytes} [Thu May 19 20:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:16.496403 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3544b-a6a0-448f-9cdd-89cd5b9db9f7 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:16.496853 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3544b-a6a0-448f-9cdd-89cd5b9db9f7 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:16.497156 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3544b-a6a0-448f-9cdd-89cd5b9db9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:16.516366 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-cba3544b-a6a0-448f-9cdd-89cd5b9db9f7 None None] MFA Rules not processed for user `28953289ea704853b1c0c7049915633d`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:16.517616 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-cba3544b-a6a0-448f-9cdd-89cd5b9db9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:16.527692 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 679/1355] 10.222.0.84 () {58 vars in 1018 bytes} [Thu May 19 20:10:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:10:20.444741 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0311d1f-6836-49f0-a1e3-33601c32ab48 None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:20.445650 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0311d1f-6836-49f0-a1e3-33601c32ab48 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:20.446090 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0311d1f-6836-49f0-a1e3-33601c32ab48 None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:20.447720 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 677/1356] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:10:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:20.455920 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d8766-ce8c-4e4a-b996-15f835ec1dce None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:20.456458 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d8766-ce8c-4e4a-b996-15f835ec1dce None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:20.456697 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d8766-ce8c-4e4a-b996-15f835ec1dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:20.484060 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-997d8766-ce8c-4e4a-b996-15f835ec1dce None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:20.485966 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-997d8766-ce8c-4e4a-b996-15f835ec1dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:20.495520 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 680/1357] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:10:20.554900 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb2ad1-c679-4b8e-be36-1339682e20cd None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:20.555326 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb2ad1-c679-4b8e-be36-1339682e20cd None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:20.555763 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb2ad1-c679-4b8e-be36-1339682e20cd None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:20.557533 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 678/1358] 10.222.0.84 () {54 vars in 961 bytes} [Thu May 19 20:10:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:20.565156 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f843b09-4b06-4ee3-a96b-ad67bfef01c4 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:20.566214 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f843b09-4b06-4ee3-a96b-ad67bfef01c4 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:20.567101 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f843b09-4b06-4ee3-a96b-ad67bfef01c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:20.648776 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-7f843b09-4b06-4ee3-a96b-ad67bfef01c4 None None] MFA Rules not processed for user `378b046698c04b7b88194a9ffe5b045a`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:20.650315 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-7f843b09-4b06-4ee3-a96b-ad67bfef01c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:20.747556 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 681/1359] 10.222.0.84 () {58 vars in 1058 bytes} [Thu May 19 20:10:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:10:20.755740 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb2ad1-c679-4b8e-be36-1339682e20cd None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:20.774165 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb2ad1-c679-4b8e-be36-1339682e20cd None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:20.780034 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f2a31-4f91-4758-847e-5684f45885b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '378b046698c04b7b88194a9ffe5b045a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:20.780802 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f2a31-4f91-4758-847e-5684f45885b0 service nova] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:20.781194 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f2a31-4f91-4758-847e-5684f45885b0 service nova] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:20.781553 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f2a31-4f91-4758-847e-5684f45885b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:20.796563 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f2a31-4f91-4758-847e-5684f45885b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:20.798272 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f2a31-4f91-4758-847e-5684f45885b0 service nova] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:20.808853 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 679/1360] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:10:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:10:23.282758 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d925605-e926-41df-82a4-6758955c7eb7 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:23.283318 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d925605-e926-41df-82a4-6758955c7eb7 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:23.283318 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d925605-e926-41df-82a4-6758955c7eb7 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:23.284810 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 682/1361] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:23.290353 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881a733-4931-4bc8-a7c8-6ff7c586477e None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:23.290788 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881a733-4931-4bc8-a7c8-6ff7c586477e None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:23.290903 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881a733-4931-4bc8-a7c8-6ff7c586477e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:23.309777 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-0881a733-4931-4bc8-a7c8-6ff7c586477e None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:23.311036 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-0881a733-4931-4bc8-a7c8-6ff7c586477e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:23.319747 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 680/1362] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:10:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:10:23.363227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d925605-e926-41df-82a4-6758955c7eb7 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:23.378252 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d925605-e926-41df-82a4-6758955c7eb7 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:23.384191 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e509d5-76bf-4ac6-a70e-0dc7529d6a7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '378b046698c04b7b88194a9ffe5b045a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:23.384866 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e509d5-76bf-4ac6-a70e-0dc7529d6a7a service nova] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:23.385151 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e509d5-76bf-4ac6-a70e-0dc7529d6a7a service nova] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:23.385428 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e509d5-76bf-4ac6-a70e-0dc7529d6a7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:23.394466 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e509d5-76bf-4ac6-a70e-0dc7529d6a7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:23.396247 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e509d5-76bf-4ac6-a70e-0dc7529d6a7a service nova] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:23.406295 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 683/1363] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:10:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:10:25.987675 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b037089-0f60-4b63-8292-44f211874b8a None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:25.988241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b037089-0f60-4b63-8292-44f211874b8a None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:25.988241 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b037089-0f60-4b63-8292-44f211874b8a None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:25.989652 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 681/1364] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:25.995684 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded629ef-f0fe-4df4-9c9b-1e9c4983bc4a None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:25.996136 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded629ef-f0fe-4df4-9c9b-1e9c4983bc4a None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:25.996223 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded629ef-f0fe-4df4-9c9b-1e9c4983bc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:26.017353 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-ded629ef-f0fe-4df4-9c9b-1e9c4983bc4a None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:26.019033 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-ded629ef-f0fe-4df4-9c9b-1e9c4983bc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:26.028227 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 684/1365] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:10:26.079234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec80074-327a-4d45-b438-2f320e5c864f None None] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:26.079907 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec80074-327a-4d45-b438-2f320e5c864f None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:26.080600 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec80074-327a-4d45-b438-2f320e5c864f None None] PATH_INFO: `/` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:26.082210 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 682/1366] 10.222.0.84 () {54 vars in 961 bytes} [Thu May 19 20:10:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:26.088701 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990b916-73c1-46d4-80e8-d8ecebaec3c3 None None] REQUEST_METHOD: `POST` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:26.088991 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990b916-73c1-46d4-80e8-d8ecebaec3c3 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:26.089263 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990b916-73c1-46d4-80e8-d8ecebaec3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:26.109343 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.auth.core [None req-a990b916-73c1-46d4-80e8-d8ecebaec3c3 None None] MFA Rules not processed for user `378b046698c04b7b88194a9ffe5b045a`. Rule list: `[]` (Enabled: `True`). {{(pid=214314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:26.110658 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.fernet_utils [None req-a990b916-73c1-46d4-80e8-d8ecebaec3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:26.120786 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 685/1367] 10.222.0.84 () {58 vars in 1058 bytes} [Thu May 19 20:10:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:10:26.130069 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec80074-327a-4d45-b438-2f320e5c864f None None] Authenticating user token {{(pid=214315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:26.145940 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec80074-327a-4d45-b438-2f320e5c864f None None] Validating token access rules against request {{(pid=214315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:26.151521 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e843e09-02a0-49c4-9617-6291d879cf68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '378b046698c04b7b88194a9ffe5b045a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:26.152183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e843e09-02a0-49c4-9617-6291d879cf68 service nova] REQUEST_METHOD: `GET` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:26.152471 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e843e09-02a0-49c4-9617-6291d879cf68 service nova] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:26.152748 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e843e09-02a0-49c4-9617-6291d879cf68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:26.162977 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e843e09-02a0-49c4-9617-6291d879cf68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:26.164516 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e843e09-02a0-49c4-9617-6291d879cf68 service nova] RBAC: Authorization granted {{(pid=214315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:26.175146 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 683/1368] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:10:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:10:27.121365 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c96986-91e7-4059-9320-856d40a6af4b None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:27.121956 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c96986-91e7-4059-9320-856d40a6af4b None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:27.121956 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c96986-91e7-4059-9320-856d40a6af4b None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:27.123257 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 686/1369] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:27.129437 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed06aa-aaec-4726-a831-2099a55cfb2a None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:27.130084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed06aa-aaec-4726-a831-2099a55cfb2a None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:27.130084 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed06aa-aaec-4726-a831-2099a55cfb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:27.149423 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-57ed06aa-aaec-4726-a831-2099a55cfb2a None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:27.150734 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-57ed06aa-aaec-4726-a831-2099a55cfb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:27.159183 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 684/1370] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:10:28.480549 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc17419-a897-4323-8836-8ccdc82fde04 None None] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:28.481110 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc17419-a897-4323-8836-8ccdc82fde04 None None] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:28.481110 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc17419-a897-4323-8836-8ccdc82fde04 None None] PATH_INFO: `/` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:28.482506 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 687/1371] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:28.487790 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786903d-4df7-41eb-95ec-e1bc4bd06632 None None] REQUEST_METHOD: `POST` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:28.488234 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786903d-4df7-41eb-95ec-e1bc4bd06632 None None] SCRIPT_NAME: `/identity` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:28.488335 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786903d-4df7-41eb-95ec-e1bc4bd06632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:28.506555 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.auth.core [None req-f786903d-4df7-41eb-95ec-e1bc4bd06632 None None] MFA Rules not processed for user `def9ab1b20e24b65870b4a851d7650b7`. Rule list: `[]` (Enabled: `True`). {{(pid=214315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:28.507722 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG keystone.common.fernet_utils [None req-f786903d-4df7-41eb-95ec-e1bc4bd06632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:28.516072 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: [pid: 214315|app: 0|req: 685/1372] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:10:28.559299 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc17419-a897-4323-8836-8ccdc82fde04 None None] Authenticating user token {{(pid=214314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:28.574855 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc17419-a897-4323-8836-8ccdc82fde04 None None] Validating token access rules against request {{(pid=214314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:28.580670 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e25cb2-35c7-4dec-9028-98313d42d5be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '378b046698c04b7b88194a9ffe5b045a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba22fdc274b74521847e5fceb2d307b1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:28.581297 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e25cb2-35c7-4dec-9028-98313d42d5be service nova] REQUEST_METHOD: `GET` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:28.581572 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e25cb2-35c7-4dec-9028-98313d42d5be service nova] SCRIPT_NAME: `/identity` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:28.581819 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e25cb2-35c7-4dec-9028-98313d42d5be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=214314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:28.590541 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e25cb2-35c7-4dec-9028-98313d42d5be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:28.592139 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e25cb2-35c7-4dec-9028-98313d42d5be service nova] RBAC: Authorization granted {{(pid=214314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:28.602524 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: [pid: 214314|app: 0|req: 688/1373] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:10:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:10:38.501171 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: DEBUG dbcounter [-] [214315] Writing DB stats keystone:SELECT=21 {{(pid=214315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:38.597770 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: DEBUG dbcounter [-] [214314] Writing DB stats keystone:SELECT=95 {{(pid=214314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:50.815375 dv-cinder-835881-13-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 20:10:50.816149 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: Thu May 19 20:10:50 2022 - graceful shutdown triggered... May 19 20:10:50.816764 dv-cinder-835881-13-iscsi devstack@keystone.service[214314]: Gracefully killing worker 1 (pid: 214314)... May 19 20:10:50.817303 dv-cinder-835881-13-iscsi devstack@keystone.service[214315]: Gracefully killing worker 2 (pid: 214315)... May 19 20:10:51.817472 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: worker 1 buried after 1 seconds May 19 20:10:51.817472 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: worker 2 buried after 1 seconds May 19 20:10:51.817472 dv-cinder-835881-13-iscsi devstack@keystone.service[214300]: goodbye to uWSGI. May 19 20:10:51.819001 dv-cinder-835881-13-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 20:10:51.819562 dv-cinder-835881-13-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 19 20:16:50.264409 dv-cinder-835881-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 20:16:50.271916 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 20:16:50.273905 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 20:16:50 2022] *** May 19 20:16:50.273905 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 20:16:50.273905 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 20:16:50.273905 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: nodename: dv-cinder-835881-13-iscsi May 19 20:16:50.273905 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: machine: x86_64 May 19 20:16:50.273905 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: clock source: unix May 19 20:16:50.273905 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: pcre jit disabled May 19 20:16:50.273905 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: detected number of CPU cores: 4 May 19 20:16:50.273905 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: current working directory: / May 19 20:16:50.273905 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: detected binary path: /usr/bin/uwsgi-core May 19 20:16:50.273905 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: your processes number limit is 47680 May 19 20:16:50.273905 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: your memory page size is 4096 bytes May 19 20:16:50.273905 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: detected max file descriptor number: 2048 May 19 20:16:50.273905 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: lock engine: pthread robust mutexes May 19 20:16:50.274988 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: thunder lock: enabled May 19 20:16:50.275065 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 20:16:50.275139 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 20:16:50.320164 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: Python main interpreter initialized at 0x5579f3b439c0 May 19 20:16:50.320164 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: python threads support enabled May 19 20:16:50.320164 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: your server socket listen backlog is limited to 100 connections May 19 20:16:50.320164 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: your mercy for graceful operations on workers is 90 seconds May 19 20:16:50.320629 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: mapped 403077 bytes (393 KB) for 2 cores May 19 20:16:50.320629 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: *** Operational MODE: preforking *** May 19 20:16:50.320882 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: *** uWSGI is running in multiple interpreter mode *** May 19 20:16:50.320882 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: spawned uWSGI master process (pid: 277667) May 19 20:16:50.321066 dv-cinder-835881-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 20:16:50.323717 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: spawned uWSGI worker 1 (pid: 277681, cores: 1) May 19 20:16:50.323717 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: spawned uWSGI worker 2 (pid: 277682, cores: 1) May 19 20:16:50.323717 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 20:16:56.002161 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:16:56.002161 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: from cryptography.utils import int_from_bytes May 19 20:16:56.003189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:16:56.003189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: from cryptography.utils import int_from_bytes May 19 20:16:56.052090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 20:16:56.052288 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 20:16:56.052437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] command line args: [] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 20:16:56.052736 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 20:16:56.052884 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ================================================================================ {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 20:16:56.053164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] admin_token = **** {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.053415 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] config_dir = [] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.053604 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.053750 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] config_source = [] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.053959 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] debug = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.054139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.054534 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.054748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] insecure_debug = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.055005 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.055259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.055443 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] list_limit = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.055655 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] log_config_append = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.055839 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.056019 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] log_dir = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.056145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] log_file = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.056264 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:16:56.056264 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: from cryptography.utils import int_from_bytes May 19 20:16:56.057300 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.057449 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:16:56.057449 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: from cryptography.utils import int_from_bytes May 19 20:16:56.057697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.057697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.057697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.057697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.057697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.057697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.058602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.058602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.058602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.058983 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.059107 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.059328 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.059556 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] notification_format = cadf {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.059868 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.060370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.84/identity {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.060564 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] publish_errors = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.060760 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.061025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.061219 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.061444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.061637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.061889 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] standard_threads = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.062123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] strict_password_check = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.062405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.062555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] use_eventlog = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.062784 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] use_journal = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.062928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] use_json = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.063060 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] use_stderr = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.063247 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] use_syslog = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.063484 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] watch_log_file = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.063744 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.063985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.064250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.064494 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.064745 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.064990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.065243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.065482 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.065732 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.065983 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.066237 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.066500 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.066750 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.066972 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.067223 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.067472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.067710 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.067967 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.068223 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.068472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.068710 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.068967 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.069231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.069487 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.069742 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.069975 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.070208 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.070468 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.070706 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.070955 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.071204 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.071455 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.071673 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.071923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.072173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.072412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.072664 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.072900 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.073138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.073375 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.073710 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.074084 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.074387 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.074629 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.074877 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.075132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.075367 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.075609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.075843 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.076082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.076321 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.076572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.076805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.077045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.077283 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.077523 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.077763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.078001 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.078232 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.078512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.078750 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.078990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.079224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.079486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.079724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.079957 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.080197 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.080436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.080709 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.080951 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.081202 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.081449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.081687 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.081926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.082164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.082440 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.enabled = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.082681 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.082926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.083171 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.083411 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.083651 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.083888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.084125 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.084364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.084632 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.084884 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.085129 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.085370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.085615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.085856 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.086095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.086357 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.086593 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.086826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.087061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.087297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.087528 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.087770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.088006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.088256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.088494 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.088868 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.089133 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.089465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.090025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] auth.external = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.090280 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] auth.mapped = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.090576 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.090813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.091050 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] auth.password = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.091285 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] auth.token = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.091531 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.091771 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] catalog.caching = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.092023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.092260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.092513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.092755 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.092983 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.093222 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] credential.caching = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.093473 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] credential.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.093726 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.093977 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.094217 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.094492 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.094730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.095005 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.095261 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.095501 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.095755 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.096045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.096294 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.096561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.096805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.097065 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.097303 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] federation.caching = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.097541 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.097789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] federation.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.098037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.098283 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.098540 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.098780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.099036 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.099274 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.099531 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.099770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.100019 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.100257 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity.caching = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.100505 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.100755 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.100992 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.101229 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.101478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.101712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.101950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.102202 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.102491 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.102725 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.102958 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.103192 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.103440 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.103690 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.103940 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.104198 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.104448 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.104723 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.105034 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.105277 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.105513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.105943 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.106187 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.106454 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.106630 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 20:16:56.106786 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 20:16:56.106935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.107088 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] command line args: [] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 20:16:56.107245 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.107397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 20:16:56.107547 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.107705 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ================================================================================ {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 20:16:56.107850 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.108007 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] admin_token = **** {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.108159 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.108310 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] config_dir = [] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.108460 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.108630 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.108799 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.108953 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] config_source = [] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.108953 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] debug = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.109249 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.109393 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.109574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.109726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.109883 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.110036 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] insecure_debug = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.110179 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.110359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.110520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.password = **** {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.110670 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.110827 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.110987 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] list_limit = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.111138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.111299 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] log_config_append = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.111452 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.111610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.111610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] log_dir = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.111896 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.112048 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] log_file = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.112204 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.112356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.112523 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.112674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.112822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.112966 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.113108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.113108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.113399 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.113399 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.113713 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.113866 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.114026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.114192 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.114383 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.114588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.114745 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.114910 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.115069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.115213 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.115375 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.115527 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.115677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.115840 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.115990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.116141 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.116293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.116456 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] notification_format = cadf {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.116620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.116772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.116924 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.116924 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.117219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.84/identity {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.117370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.117521 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] publish_errors = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.117672 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.117828 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.117980 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.118130 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.118334 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.118496 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.118660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.118828 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.118983 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.119134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.119277 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.119420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] standard_threads = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.119564 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.119721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] strict_password_check = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.119866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.120016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.120016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] use_eventlog = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.120304 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.120465 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] use_journal = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.120622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.120784 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] use_json = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.120784 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] use_stderr = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.121063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.121221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] use_syslog = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.121378 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.121528 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] watch_log_file = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:16:56.121686 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.121844 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.121989 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.122147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.122350 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.122518 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.122678 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.122834 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.122985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.123130 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.123274 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.123419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.123576 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.123734 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.123895 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] policy.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.124053 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.124197 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.124350 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.124599 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.124753 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.124898 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.125061 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.125212 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] receipt.caching = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.125365 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.125519 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.125671 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.125821 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.125972 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.126122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.126295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.126466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.126466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.126764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.126915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] resource.caching = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.127072 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.127222 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.127376 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.127526 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] resource.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.127673 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.127825 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.127976 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.128126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.128284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.128443 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.128593 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.128744 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] revoke.caching = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.128894 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.129054 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.129233 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.129384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.129534 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.129685 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] role.cache_time = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.129829 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.129979 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] role.caching = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.130133 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.130307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] role.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.130467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.130617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] role.list_limit = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.130770 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.130920 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.131077 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.131252 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.131404 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.131564 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.131715 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.131858 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.132012 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.132162 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.132314 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.132467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.132617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.132771 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.132928 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.133071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.133235 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.133381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.133531 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.133680 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.133838 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.133981 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.134132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.134329 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.134501 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.134659 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.134804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.134804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.135089 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.135250 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.135423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.135577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.135729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.135880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.136023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.136203 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.136356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.136515 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.136659 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.136990 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.137140 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.137291 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.137436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.137605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.137755 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.137906 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.138056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.138200 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.138382 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.138546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.138698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.138849 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.138994 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.139145 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.139295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.139448 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.139611 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.139762 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.139906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.140056 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.140205 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.140361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.140522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.140667 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.140817 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] token.cache_time = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.140968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.141117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] token.caching = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.141270 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.141414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.141564 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.141740 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] token.provider = fernet {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.141890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.142041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.142198 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.142198 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.142512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.142687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.142839 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.142839 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.143134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.143285 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.143445 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.143599 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.143765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.143909 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] trust.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.144058 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.144215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.144365 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.144520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.144664 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.144807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.144957 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.145112 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.145271 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.enabled = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.145425 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.145568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.145711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.145872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.146022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.146166 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.146166 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.146499 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.146651 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.146805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.connection = **** {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.146805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.147100 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.147252 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.147395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.147545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.147699 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.147865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.148010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.148154 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.148305 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.148448 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.148601 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.148745 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.148745 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.149035 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.149179 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.149322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.149475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.149642 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.149793 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.149962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.150108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.150252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.150252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.150623 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.150774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.150938 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.151097 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.151248 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.151406 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.151549 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.151734 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.151893 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.152112 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.152281 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.152281 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 20:16:56.152743 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.152743 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.152743 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.152743 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.152743 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.152743 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.152743 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] auth.external = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.152743 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] auth.mapped = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.152743 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.153968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.153968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] auth.password = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.153968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] auth.token = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.153968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.153968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] catalog.caching = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.153968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.153968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.153968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.153968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.155237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.155237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] credential.caching = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.155237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] credential.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.155237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.155237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.155237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.155237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.155237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.155237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.156501 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.156501 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.156501 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.156501 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.156501 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.156501 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.156501 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.156501 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.156501 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] federation.caching = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.157728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.157728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] federation.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.157728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.157728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.157728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.157728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.157728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.157728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.157728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.159002 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.159002 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.159002 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity.caching = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.159002 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.159002 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.159002 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.159002 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.159002 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.159002 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.160246 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.160246 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.160246 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.160246 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.160246 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.160246 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.160246 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.160246 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.160246 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.161458 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.161458 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.161458 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.161458 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.161458 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.161458 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.161458 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.161458 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.161458 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.162763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.162763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.162763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.162763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.162763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.162763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.162763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.162763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.162763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.164016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.164016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.164016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.password = **** {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.164016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.164016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.164016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.164016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.164016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.164016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.165267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.165267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.165267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.165267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.165267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.165267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.165267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.165267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.165267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.166547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.166547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.166547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.166547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.166547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.166547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.166547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.166547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.166547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.168017 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.168017 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.168017 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.168017 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.168017 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.168697 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.168697 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.168980 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.168980 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.169259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.169408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.169635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.169880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.170131 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.170418 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.170673 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.170915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.171177 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] policy.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.171412 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.171660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.171815 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.171976 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.172121 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.172290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] receipt.caching = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.172452 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.172607 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.172607 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.172899 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.173042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.173271 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.173516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] resource.caching = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.173770 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.174022 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] resource.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.174285 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.174542 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.174796 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.175037 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] revoke.caching = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.175286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.175526 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.175769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] role.cache_time = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.176009 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] role.caching = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.176259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] role.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.176491 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] role.list_limit = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.176744 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.176998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.177250 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.177500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.177751 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.178004 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.178254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.178528 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.178769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.179021 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.179273 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.179523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.179772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.180183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.180425 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.180678 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.180931 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.181086 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.181246 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.181409 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.181646 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.181904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.182160 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.182429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.182665 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.182910 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.183403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.183692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.183970 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.184227 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.184472 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.184716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.184970 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] token.cache_time = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.185212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] token.caching = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.185457 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.185712 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] token.provider = fernet {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.185955 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.186200 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.186485 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.186729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.186979 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.187226 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.187477 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] trust.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.187718 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.187961 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.188198 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.188448 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.188700 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.188939 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.189185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.189495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.189760 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.connection = **** {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.190010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.190307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.190524 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.190769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.191009 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.191253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.191493 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.191732 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.191982 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.192222 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.192468 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.192748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.192929 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.193174 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.193422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.193656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.193901 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.194141 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:16:56.194335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277682) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 20:16:56.209520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.209756 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.220880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.221148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.221385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.232049 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.251452 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.251687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.263324 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.291087 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.314027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.314255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.329780 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.330021 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.379463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.379663 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.379900 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.380119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.380347 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.380505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.380608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.380706 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.380961 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.381192 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.381409 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.381599 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.403730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.403939 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.404199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.404433 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.404571 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.404684 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.404776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.404855 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.405028 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.405198 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.405306 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.405488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.405674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.405917 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.416354 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.416611 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.430286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.430286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.430286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.430286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.430857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.430857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.431099 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277682) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.432715 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.432939 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.433262 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.433622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.433817 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.434240 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.434457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:16:56.448891 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.449095 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.449454 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:16:56.449625 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:16:56.449726 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.449726 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:16:56.450061 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.450061 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:16:56.450317 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.450465 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.450663 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.450820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.450959 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.450959 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.451222 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.451222 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.461623 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.461880 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.462182 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.462495 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.463836 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.464093 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.464978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.465433 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.471091 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.473569 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.474402 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.474687 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.474975 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.475378 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.475664 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.475927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.476206 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.476478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.476754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.476892 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.477006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.477104 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.477281 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.477422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.477792 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.478075 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.478363 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.478634 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.478910 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.479181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.479451 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.479718 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.504165 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.505790 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.506068 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.506997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.508617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.508902 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.511392 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.511674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.511956 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.514604 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.514905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.515194 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.519277 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.519563 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.522507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.522796 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.524200 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.524487 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.524770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.525060 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.525342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.525619 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.525893 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.526163 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.527740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.528029 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.528315 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.528601 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.528870 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.529144 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.529416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.529685 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.544919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.545194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.548876 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.549158 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.549737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.553385 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.553684 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.553841 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.557335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.557635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.558945 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.562559 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.563095 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.566350 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.566653 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.566790 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.566888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.567206 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.567485 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.567772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.570454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.570753 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.571042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.571331 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.571612 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.571888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.586880 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.587257 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.587531 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.587807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.592104 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.592490 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.592764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.593042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.596195 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.597939 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.598218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.600998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.601277 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.601568 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.601861 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.602145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.602454 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.602771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.603054 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.606479 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.606759 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.607038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.607338 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.607623 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.615882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.616166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.616444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.616727 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.617006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.617286 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.617558 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.621393 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.621672 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.621954 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.622225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.622540 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.622821 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.623097 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.636308 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.639422 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.641701 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.642773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.644379 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.645003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.646033 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.646346 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.646653 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.648364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.649987 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.651665 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.652081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.652561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.654116 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.655772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.659045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.659340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.659625 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.659902 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.660039 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.661664 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.664935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.665224 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.665514 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.665792 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.669199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.669532 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.669855 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.675361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.675653 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.675936 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.677828 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=277681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.678110 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.678425 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.678701 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.678974 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.679266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.679545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.679812 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.680087 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.680357 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.680638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.680885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.681159 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.681420 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=277681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.684347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=277682) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:16:56.684631 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.684910 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.685189 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.685479 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.685759 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.686043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.686324 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.686598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.686868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.687138 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.687409 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.687678 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.687954 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=277682) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:16:56.721297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5579f3b439c0 pid: 277681 (default app) May 19 20:16:56.724753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21aeda-cb94-4ac4-b704-7332321e9378 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:56.725055 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21aeda-cb94-4ac4-b704-7332321e9378 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:56.725342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21aeda-cb94-4ac4-b704-7332321e9378 None None] PATH_INFO: `/v3` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:56.727845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5579f3b439c0 pid: 277682 (default app) May 19 20:16:56.729596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 1/1] 10.222.0.84 () {50 vars in 791 bytes} [Thu May 19 20:16:56 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:04.805774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6cbff-8f53-4f9e-a2a3-0cf703a13839 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:04.806683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6cbff-8f53-4f9e-a2a3-0cf703a13839 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:04.806683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6cbff-8f53-4f9e-a2a3-0cf703a13839 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:04.811898 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 1/2] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:04 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:04.818045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078e5b4-7759-4729-9c78-172cb6f10fc4 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:04.818509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078e5b4-7759-4729-9c78-172cb6f10fc4 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:04.818628 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078e5b4-7759-4729-9c78-172cb6f10fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:04.889503 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: INFO dbcounter [None req-3078e5b4-7759-4729-9c78-172cb6f10fc4 None None] Registered counter for database keystone May 19 20:17:04.915578 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG oslo_db.sqlalchemy.engines [None req-3078e5b4-7759-4729-9c78-172cb6f10fc4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=277681) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 20:17:04.924509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writer thread running {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 20:17:05.119088 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG passlib.handlers.bcrypt [None req-3078e5b4-7759-4729-9c78-172cb6f10fc4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=277681) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 20:17:05.119431 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG passlib.handlers.bcrypt [None req-3078e5b4-7759-4729-9c78-172cb6f10fc4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=277681) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 20:17:05.189178 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-3078e5b4-7759-4729-9c78-172cb6f10fc4 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:05.190863 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-3078e5b4-7759-4729-9c78-172cb6f10fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:05.323668 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 2/3] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 507 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 19 20:17:05.360217 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6cbff-8f53-4f9e-a2a3-0cf703a13839 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:05.439987 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: INFO dbcounter [None req-81b6cbff-8f53-4f9e-a2a3-0cf703a13839 None None] Registered counter for database keystone May 19 20:17:05.460617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG oslo_db.sqlalchemy.engines [None req-81b6cbff-8f53-4f9e-a2a3-0cf703a13839 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=277682) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 20:17:05.468930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writer thread running {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 20:17:05.591857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6cbff-8f53-4f9e-a2a3-0cf703a13839 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:05.597442 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fbc44-44c3-47e0-97b5-3b902fcb9ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fd7be0b818c4867be2daed29f53ab77', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:05.598105 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fbc44-44c3-47e0-97b5-3b902fcb9ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:05.598497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fbc44-44c3-47e0-97b5-3b902fcb9ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:05.598773 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fbc44-44c3-47e0-97b5-3b902fcb9ef1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:05.608301 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194fbc44-44c3-47e0-97b5-3b902fcb9ef1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:05.659567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: WARNING keystone.common.rbac_enforcer.enforcer [None req-194fbc44-44c3-47e0-97b5-3b902fcb9ef1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 20:17:05.659914 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194fbc44-44c3-47e0-97b5-3b902fcb9ef1 admin admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:05.670401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 2/4] 10.222.0.84 () {58 vars in 1324 bytes} [Thu May 19 20:17:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 20:17:05.677591 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078e5b4-7759-4729-9c78-172cb6f10fc4 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:05.695589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078e5b4-7759-4729-9c78-172cb6f10fc4 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:05.702084 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d427a8-0b15-4921-ad8f-87d6706701d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fd7be0b818c4867be2daed29f53ab77', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:05.703248 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d427a8-0b15-4921-ad8f-87d6706701d6 admin admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:05.703536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d427a8-0b15-4921-ad8f-87d6706701d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:05.703816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d427a8-0b15-4921-ad8f-87d6706701d6 admin admin] PATH_INFO: `/v3/projects/8fd7be0b818c4867be2daed29f53ab77` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:05.705581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d427a8-0b15-4921-ad8f-87d6706701d6 admin admin] RBAC: Authorizing `identity:get_project(project_id=8fd7be0b818c4867be2daed29f53ab77)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:05.878322 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c7d427a8-0b15-4921-ad8f-87d6706701d6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 20:17:05.879055 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d427a8-0b15-4921-ad8f-87d6706701d6 admin admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:05.883519 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 3/5] 10.222.0.84 () {56 vars in 1207 bytes} [Thu May 19 20:17:05 2022] GET /identity/v3/projects/8fd7be0b818c4867be2daed29f53ab77 => generated 344 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:08.128888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d57f91-6f60-457c-9a96-47ec7bdd8261 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:08.129743 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d57f91-6f60-457c-9a96-47ec7bdd8261 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:08.129743 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d57f91-6f60-457c-9a96-47ec7bdd8261 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:08.136593 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 3/6] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:08.144882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858340e-285e-4925-b5a5-e7373f1708f5 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:08.145349 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858340e-285e-4925-b5a5-e7373f1708f5 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:08.145447 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858340e-285e-4925-b5a5-e7373f1708f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:08.173187 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-7858340e-285e-4925-b5a5-e7373f1708f5 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:08.174773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-7858340e-285e-4925-b5a5-e7373f1708f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:08.184770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 4/7] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 19 20:17:08.207040 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d57f91-6f60-457c-9a96-47ec7bdd8261 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:08.230119 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d57f91-6f60-457c-9a96-47ec7bdd8261 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:08.235829 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7028dc-a39f-4ed3-bc63-4b999aebe426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fd7be0b818c4867be2daed29f53ab77', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:08.236480 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7028dc-a39f-4ed3-bc63-4b999aebe426 admin admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:08.236750 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7028dc-a39f-4ed3-bc63-4b999aebe426 admin admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:08.237035 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7028dc-a39f-4ed3-bc63-4b999aebe426 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:08.245254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7028dc-a39f-4ed3-bc63-4b999aebe426 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:08.247460 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7028dc-a39f-4ed3-bc63-4b999aebe426 admin admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:08.256627 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 4/8] 10.222.0.84 () {58 vars in 1324 bytes} [Thu May 19 20:17:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 20:17:08.264255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858340e-285e-4925-b5a5-e7373f1708f5 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:08.273994 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858340e-285e-4925-b5a5-e7373f1708f5 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:08.282638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d8274-f095-47ab-95d1-2dab4fef50f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fd7be0b818c4867be2daed29f53ab77', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:08.283919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d8274-f095-47ab-95d1-2dab4fef50f5 admin admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:08.284299 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d8274-f095-47ab-95d1-2dab4fef50f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:08.284667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d8274-f095-47ab-95d1-2dab4fef50f5 admin admin] PATH_INFO: `/v3/users/470d21b48683452bbca4f78884ff1182` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:08.287158 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d8274-f095-47ab-95d1-2dab4fef50f5 admin admin] RBAC: Authorizing `identity:get_user(user_id=470d21b48683452bbca4f78884ff1182)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:08.290035 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d8274-f095-47ab-95d1-2dab4fef50f5 admin admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:08.292445 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 5/9] 10.222.0.84 () {56 vars in 1198 bytes} [Thu May 19 20:17:08 2022] GET /identity/v3/users/470d21b48683452bbca4f78884ff1182 => generated 247 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:10.580038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a9986-3df4-44a9-8fd2-e333006b5897 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:10.580648 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a9986-3df4-44a9-8fd2-e333006b5897 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:10.580648 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a9986-3df4-44a9-8fd2-e333006b5897 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:10.581976 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 5/10] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:17:10.587422 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68498e-8bd4-4908-a05f-0fe21620e0f0 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:10.587854 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68498e-8bd4-4908-a05f-0fe21620e0f0 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:10.588116 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68498e-8bd4-4908-a05f-0fe21620e0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:10.607704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-7c68498e-8bd4-4908-a05f-0fe21620e0f0 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:10.608920 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-7c68498e-8bd4-4908-a05f-0fe21620e0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:10.649554 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a24e6-9153-4ba9-837d-01f3b6b5a0c6 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:10.649933 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a24e6-9153-4ba9-837d-01f3b6b5a0c6 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:10.650197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a24e6-9153-4ba9-837d-01f3b6b5a0c6 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:10.652339 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 6/11] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:10.657602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24af912-d2a3-4727-b9d3-cd6920386321 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:10.657882 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24af912-d2a3-4727-b9d3-cd6920386321 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:10.658167 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24af912-d2a3-4727-b9d3-cd6920386321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:10.694368 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 6/12] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:10.698642 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG passlib.handlers.bcrypt [None req-c24af912-d2a3-4727-b9d3-cd6920386321 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=277682) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 20:17:10.699115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG passlib.handlers.bcrypt [None req-c24af912-d2a3-4727-b9d3-cd6920386321 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=277682) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 20:17:10.718234 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68498e-8bd4-4908-a05f-0fe21620e0f0 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:10.742665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:10.745516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-c24af912-d2a3-4727-b9d3-cd6920386321 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:10.747325 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-c24af912-d2a3-4727-b9d3-cd6920386321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:10.824063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68498e-8bd4-4908-a05f-0fe21620e0f0 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:10.849410 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 7/13] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:10.863028 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b405c0-a7c1-477b-9019-4cedfd226dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:10.863842 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b405c0-a7c1-477b-9019-4cedfd226dac None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:10.864124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b405c0-a7c1-477b-9019-4cedfd226dac None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:10.864404 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b405c0-a7c1-477b-9019-4cedfd226dac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:10.871835 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24af912-d2a3-4727-b9d3-cd6920386321 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:10.876625 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b405c0-a7c1-477b-9019-4cedfd226dac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-d9b405c0-a7c1-477b-9019-4cedfd226dac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:10.888151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:10.889572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:10.890643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:10.890643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:10.890643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:10.890643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:10.890643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:10.890643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:10.890643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:10.890643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:10.890643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:10.890643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:10.890643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:10.890643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:10.890643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 7/14] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:17:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:17:10.895310 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b405c0-a7c1-477b-9019-4cedfd226dac None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:10.977577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:10.990543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24af912-d2a3-4727-b9d3-cd6920386321 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:10.993657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:10.999545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b405c0-a7c1-477b-9019-4cedfd226dac None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:11.032199 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc73ce-896a-4120-9a61-427ebd9f4ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:11.033118 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc73ce-896a-4120-9a61-427ebd9f4ced None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:11.033379 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc73ce-896a-4120-9a61-427ebd9f4ced None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:11.033685 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc73ce-896a-4120-9a61-427ebd9f4ced None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:11.036971 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c84ae7-d607-4d68-82fc-bff3b011e2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:11.037660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c84ae7-d607-4d68-82fc-bff3b011e2d6 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:11.037942 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c84ae7-d607-4d68-82fc-bff3b011e2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:11.038212 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c84ae7-d607-4d68-82fc-bff3b011e2d6 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:11.039001 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c84ae7-d607-4d68-82fc-bff3b011e2d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:11.039314 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c84ae7-d607-4d68-82fc-bff3b011e2d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:11.040727 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c84ae7-d607-4d68-82fc-bff3b011e2d6 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:11.053274 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 8/15] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:11.059068 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c84ae7-d607-4d68-82fc-bff3b011e2d6 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:11.103541 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc73ce-896a-4120-9a61-427ebd9f4ced None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:11.105153 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc73ce-896a-4120-9a61-427ebd9f4ced None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:11.162077 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c84ae7-d607-4d68-82fc-bff3b011e2d6 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:11.204547 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11939487-ceab-4b37-bf32-6fe9a2e65b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:11.205608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11939487-ceab-4b37-bf32-6fe9a2e65b59 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:11.205953 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11939487-ceab-4b37-bf32-6fe9a2e65b59 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:11.206438 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11939487-ceab-4b37-bf32-6fe9a2e65b59 None admin] PATH_INFO: `/v3/users/470d21b48683452bbca4f78884ff1182` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:11.207717 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11939487-ceab-4b37-bf32-6fe9a2e65b59 None admin] RBAC: Authorizing `identity:get_user(user_id=470d21b48683452bbca4f78884ff1182)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:11.208892 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 8/16] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:17:11.209214 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11939487-ceab-4b37-bf32-6fe9a2e65b59 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:11.211001 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 9/17] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:11 2022] GET /identity/v3/users/470d21b48683452bbca4f78884ff1182 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:17:11.215479 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc73ce-896a-4120-9a61-427ebd9f4ced None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:11.217146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11939487-ceab-4b37-bf32-6fe9a2e65b59 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:11.219785 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:11.232847 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:11.314518 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc73ce-896a-4120-9a61-427ebd9f4ced None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:11.319406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11939487-ceab-4b37-bf32-6fe9a2e65b59 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:11.350563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca16ff-bcb2-46fc-820c-49af21d43a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:11.351369 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca16ff-bcb2-46fc-820c-49af21d43a57 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:11.351645 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca16ff-bcb2-46fc-820c-49af21d43a57 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:11.351918 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca16ff-bcb2-46fc-820c-49af21d43a57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:11.354211 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1e008-552c-4a9b-a0f4-78d2c1d4c2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:11.355625 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1e008-552c-4a9b-a0f4-78d2c1d4c2c2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:11.355918 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1e008-552c-4a9b-a0f4-78d2c1d4c2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:11.356200 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1e008-552c-4a9b-a0f4-78d2c1d4c2c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:11.357449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1e008-552c-4a9b-a0f4-78d2c1d4c2c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:11.358972 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1e008-552c-4a9b-a0f4-78d2c1d4c2c2 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:11.360602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 10/18] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:11.366342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1e008-552c-4a9b-a0f4-78d2c1d4c2c2 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:11.370636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca16ff-bcb2-46fc-820c-49af21d43a57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:11.372114 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca16ff-bcb2-46fc-820c-49af21d43a57 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:11.393506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-6dca16ff-bcb2-46fc-820c-49af21d43a57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:17:11.393506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:11.393506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:11.393506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:11.393506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:11.393506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:11.393506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:11.393506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:11.393506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:11.393506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:11.393506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:11.393506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:11.393506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:17:11.393506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:17:11.393506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:11.394584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:11.396161 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:17:11.396161 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:17:11.396161 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:17:11.396161 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:11.396161 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 9/19] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:11.400619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca16ff-bcb2-46fc-820c-49af21d43a57 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:11.453657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:11.456050 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:11.475079 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1e008-552c-4a9b-a0f4-78d2c1d4c2c2 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:11.501966 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca16ff-bcb2-46fc-820c-49af21d43a57 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:11.512496 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ec97f-a0db-4f17-a428-2ec05efdebbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:11.513172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ec97f-a0db-4f17-a428-2ec05efdebbf None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:11.513459 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ec97f-a0db-4f17-a428-2ec05efdebbf None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:11.513740 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ec97f-a0db-4f17-a428-2ec05efdebbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:11.514618 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ec97f-a0db-4f17-a428-2ec05efdebbf None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.domain.id=default, user.id=470d21b48683452bbca4f78884ff1182) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:11.514933 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ec97f-a0db-4f17-a428-2ec05efdebbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:11.516425 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ec97f-a0db-4f17-a428-2ec05efdebbf None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:11.523354 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 11/20] 10.222.0.84 () {56 vars in 1322 bytes} [Thu May 19 20:17:11 2022] GET /identity/v3/role_assignments?user.id=470d21b48683452bbca4f78884ff1182&scope.domain.id=default&role.id=b239f941cb724331bf28dc489b70ad1d => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:11.538035 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254676c-12da-47df-a00a-ffa2f50eeafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:11.538776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254676c-12da-47df-a00a-ffa2f50eeafb None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:11.539070 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254676c-12da-47df-a00a-ffa2f50eeafb None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:11.539335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254676c-12da-47df-a00a-ffa2f50eeafb None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:11.540078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254676c-12da-47df-a00a-ffa2f50eeafb None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:11.540382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254676c-12da-47df-a00a-ffa2f50eeafb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:11.541835 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254676c-12da-47df-a00a-ffa2f50eeafb None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:11.564196 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 10/21] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:17:14.119057 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7ef48-9204-4bf6-8b4b-49657ccf5caa None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:14.120420 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7ef48-9204-4bf6-8b4b-49657ccf5caa None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:14.120933 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7ef48-9204-4bf6-8b4b-49657ccf5caa None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:14.122643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 12/22] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:14.128439 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a84e30-6ee6-492f-b0d4-5387e943461c None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:14.129378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a84e30-6ee6-492f-b0d4-5387e943461c None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:14.129860 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a84e30-6ee6-492f-b0d4-5387e943461c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:14.286173 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-76a84e30-6ee6-492f-b0d4-5387e943461c None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:14.287643 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-76a84e30-6ee6-492f-b0d4-5387e943461c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:14.352735 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af0677-50c2-41ef-aed8-4685ac767130 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:14.353369 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af0677-50c2-41ef-aed8-4685ac767130 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:14.353797 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af0677-50c2-41ef-aed8-4685ac767130 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:14.356200 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 13/23] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:14.369401 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98031a5-562d-4ae2-ac8f-5cac31fb9295 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:14.370025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98031a5-562d-4ae2-ac8f-5cac31fb9295 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:14.370803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98031a5-562d-4ae2-ac8f-5cac31fb9295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:14.384281 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 11/24] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:14.389728 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-e98031a5-562d-4ae2-ac8f-5cac31fb9295 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:14.391179 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-e98031a5-562d-4ae2-ac8f-5cac31fb9295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:14.421071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ccd67-6263-45a0-b924-5659f3e63e42 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:14.421841 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ccd67-6263-45a0-b924-5659f3e63e42 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:14.422561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ccd67-6263-45a0-b924-5659f3e63e42 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:14.424572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 12/25] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:17:14.425497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ccd67-6263-45a0-b924-5659f3e63e42 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:14.477865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 14/26] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:17:14.480467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ab2b4-fb09-4325-a0b1-e221a57dc62d None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:14.481035 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ab2b4-fb09-4325-a0b1-e221a57dc62d None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:14.481517 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ab2b4-fb09-4325-a0b1-e221a57dc62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:14.500123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-5a4ab2b4-fb09-4325-a0b1-e221a57dc62d None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:14.501637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-5a4ab2b4-fb09-4325-a0b1-e221a57dc62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:14.531047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:14.605690 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 15/27] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:14.609933 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ab2b4-fb09-4325-a0b1-e221a57dc62d None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:14.618653 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:14.648240 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ccd67-6263-45a0-b924-5659f3e63e42 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:14.714064 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bc91c-dcc6-43a1-96c9-6b2f59805b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:14.715625 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bc91c-dcc6-43a1-96c9-6b2f59805b9a None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:14.718351 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bc91c-dcc6-43a1-96c9-6b2f59805b9a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:14.718851 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bc91c-dcc6-43a1-96c9-6b2f59805b9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:14.745299 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bc91c-dcc6-43a1-96c9-6b2f59805b9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-c58bc91c-dcc6-43a1-96c9-6b2f59805b9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:14.761656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:14.762774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:14.763679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:14.763679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:14.763679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:14.763679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:14.763679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:14.763679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:14.763679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:14.763679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:14.763679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:14.763679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:14.763679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:14.763679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:14.773435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 13/28] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:17:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:14.775614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bc91c-dcc6-43a1-96c9-6b2f59805b9a None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:14.841154 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ab2b4-fb09-4325-a0b1-e221a57dc62d None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:14.894137 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd5361-2b6f-43ac-b75d-432b51e17252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:14.895503 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd5361-2b6f-43ac-b75d-432b51e17252 None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:14.895997 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd5361-2b6f-43ac-b75d-432b51e17252 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:14.896479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd5361-2b6f-43ac-b75d-432b51e17252 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:14.897441 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadd5361-2b6f-43ac-b75d-432b51e17252 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:14.899454 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadd5361-2b6f-43ac-b75d-432b51e17252 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:15.028861 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bc91c-dcc6-43a1-96c9-6b2f59805b9a None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:15.041477 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:15.054555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 16/29] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 20:17:14 2022] POST /identity/v3/roles => generated 219 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:17:15.056103 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd5361-2b6f-43ac-b75d-432b51e17252 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:15.111672 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60bb18-375c-4b53-8317-980e997c59fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:15.112415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60bb18-375c-4b53-8317-980e997c59fd None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:15.112702 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60bb18-375c-4b53-8317-980e997c59fd None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:15.112983 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60bb18-375c-4b53-8317-980e997c59fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:15.133858 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e60bb18-375c-4b53-8317-980e997c59fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:15.140068 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e60bb18-375c-4b53-8317-980e997c59fd None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:15.178382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-6e60bb18-375c-4b53-8317-980e997c59fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:17:15.178382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:15.178382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:15.178382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:15.178382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:15.178382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:15.178382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:15.178382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:15.178382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:15.178382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:15.178382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:15.178382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:15.178382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:17:15.178382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:17:15.178382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:15.179402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:15.180368 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:17:15.180368 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:17:15.180368 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:17:15.180368 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:15.188089 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 14/30] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:15.195333 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf1a66-d425-40db-ab5e-53ef8add0557 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:15.195623 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf1a66-d425-40db-ab5e-53ef8add0557 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:15.195898 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:15.196345 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf1a66-d425-40db-ab5e-53ef8add0557 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:15.197552 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 15/31] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:15.198744 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf1a66-d425-40db-ab5e-53ef8add0557 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:15.288441 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd5361-2b6f-43ac-b75d-432b51e17252 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:15.354480 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df4334-fbe3-499d-8d0a-e9f39912c4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:15.355603 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df4334-fbe3-499d-8d0a-e9f39912c4d6 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:15.356163 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df4334-fbe3-499d-8d0a-e9f39912c4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:15.356660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df4334-fbe3-499d-8d0a-e9f39912c4d6 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:15.357886 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df4334-fbe3-499d-8d0a-e9f39912c4d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:15.358486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df4334-fbe3-499d-8d0a-e9f39912c4d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:15.360376 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df4334-fbe3-499d-8d0a-e9f39912c4d6 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:15.371798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 17/32] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:15.373600 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8ed8e-9a06-4174-a7d6-05c2dda9f055 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:15.374163 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8ed8e-9a06-4174-a7d6-05c2dda9f055 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:15.374704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8ed8e-9a06-4174-a7d6-05c2dda9f055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:15.391604 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf1a66-d425-40db-ab5e-53ef8add0557 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:15.398346 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-54b8ed8e-9a06-4174-a7d6-05c2dda9f055 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:15.399930 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-54b8ed8e-9a06-4174-a7d6-05c2dda9f055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:15.443851 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece6994-ec66-44f1-bf9d-b00224ac2573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:15.444600 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece6994-ec66-44f1-bf9d-b00224ac2573 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:15.444891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece6994-ec66-44f1-bf9d-b00224ac2573 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:15.445174 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece6994-ec66-44f1-bf9d-b00224ac2573 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:15.446125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece6994-ec66-44f1-bf9d-b00224ac2573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:15.446253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece6994-ec66-44f1-bf9d-b00224ac2573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:15.447655 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece6994-ec66-44f1-bf9d-b00224ac2573 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:15.463335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 16/33] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:15.464294 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece6994-ec66-44f1-bf9d-b00224ac2573 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:15.487011 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:15.499195 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 18/34] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:15.500441 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8ed8e-9a06-4174-a7d6-05c2dda9f055 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:15.560130 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:15.592224 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece6994-ec66-44f1-bf9d-b00224ac2573 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:15.615301 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8ed8e-9a06-4174-a7d6-05c2dda9f055 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:15.637667 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e717c-e9a1-47db-b4b6-41ebe1a0e559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:15.639045 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e717c-e9a1-47db-b4b6-41ebe1a0e559 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:15.639332 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e717c-e9a1-47db-b4b6-41ebe1a0e559 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:15.639764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e717c-e9a1-47db-b4b6-41ebe1a0e559 None admin] PATH_INFO: `/v3/users/470d21b48683452bbca4f78884ff1182` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:15.641369 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e717c-e9a1-47db-b4b6-41ebe1a0e559 None admin] RBAC: Authorizing `identity:get_user(user_id=470d21b48683452bbca4f78884ff1182)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:15.642866 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e717c-e9a1-47db-b4b6-41ebe1a0e559 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:15.648737 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 17/35] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:15 2022] GET /identity/v3/users/470d21b48683452bbca4f78884ff1182 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:15.648737 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e717c-e9a1-47db-b4b6-41ebe1a0e559 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:15.658340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6fb3e-cfc0-4aea-9c26-15e1766a8a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:15.659319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6fb3e-cfc0-4aea-9c26-15e1766a8a41 None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:15.659630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6fb3e-cfc0-4aea-9c26-15e1766a8a41 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:15.659891 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6fb3e-cfc0-4aea-9c26-15e1766a8a41 None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:15.660590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6fb3e-cfc0-4aea-9c26-15e1766a8a41 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:15.662064 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6fb3e-cfc0-4aea-9c26-15e1766a8a41 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:15.692114 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 19/36] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:17:15 2022] POST /identity/v3/projects => generated 301 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:17:15.695741 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c68ca-81aa-4eb8-9eb6-b900853335b0 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:15.696159 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c68ca-81aa-4eb8-9eb6-b900853335b0 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:15.696518 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c68ca-81aa-4eb8-9eb6-b900853335b0 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:15.698380 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 20/37] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:15.699218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c68ca-81aa-4eb8-9eb6-b900853335b0 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:15.769504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:15.788433 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e717c-e9a1-47db-b4b6-41ebe1a0e559 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:15.828330 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f25e8-eb31-47ad-aa25-71f5fa6b9e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:15.829141 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f25e8-eb31-47ad-aa25-71f5fa6b9e57 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:15.829428 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f25e8-eb31-47ad-aa25-71f5fa6b9e57 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:15.829715 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f25e8-eb31-47ad-aa25-71f5fa6b9e57 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:15.830515 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f25e8-eb31-47ad-aa25-71f5fa6b9e57 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:15.832394 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f25e8-eb31-47ad-aa25-71f5fa6b9e57 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:15.835700 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c68ca-81aa-4eb8-9eb6-b900853335b0 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:15.846617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:15.875421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0410e-431b-42c6-930f-a73526606bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:15.876248 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0410e-431b-42c6-930f-a73526606bb9 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:15.876533 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0410e-431b-42c6-930f-a73526606bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:15.877052 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0410e-431b-42c6-930f-a73526606bb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:15.878375 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0410e-431b-42c6-930f-a73526606bb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:15.879878 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0410e-431b-42c6-930f-a73526606bb9 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:15.881561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 21/38] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:15.883653 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a581225-83c4-4bcb-8ff0-86b99c39071d None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:15.884216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a581225-83c4-4bcb-8ff0-86b99c39071d None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:15.884727 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a581225-83c4-4bcb-8ff0-86b99c39071d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:15.903883 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-4a581225-83c4-4bcb-8ff0-86b99c39071d None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:15.905502 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-4a581225-83c4-4bcb-8ff0-86b99c39071d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:15.980191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 18/39] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 20:17:15 2022] POST /identity/v3/roles => generated 225 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:17:15.981177 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 22/40] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:15.981643 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f16ae-5784-48bb-a808-074d3beb2b1d None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:15.981927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a581225-83c4-4bcb-8ff0-86b99c39071d None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:15.982016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f16ae-5784-48bb-a808-074d3beb2b1d None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:15.982211 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f16ae-5784-48bb-a808-074d3beb2b1d None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:15.983642 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 19/41] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:15.989173 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b40a5-acf1-4d78-bcba-d41ea457e56d None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:15.989464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b40a5-acf1-4d78-bcba-d41ea457e56d None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:15.989747 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b40a5-acf1-4d78-bcba-d41ea457e56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:16.007974 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-751b40a5-acf1-4d78-bcba-d41ea457e56d None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:16.009548 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-751b40a5-acf1-4d78-bcba-d41ea457e56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:16.085340 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 20/42] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:17:16.086233 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b40a5-acf1-4d78-bcba-d41ea457e56d None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:16.126311 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a581225-83c4-4bcb-8ff0-86b99c39071d None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:16.135386 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:16.168463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd56bb2-a911-4be4-97ec-b60b6cc7cb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:16.169477 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd56bb2-a911-4be4-97ec-b60b6cc7cb67 None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:16.169766 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd56bb2-a911-4be4-97ec-b60b6cc7cb67 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:16.170297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd56bb2-a911-4be4-97ec-b60b6cc7cb67 None admin] PATH_INFO: `/v3/domains/default/users/470d21b48683452bbca4f78884ff1182/roles/b239f941cb724331bf28dc489b70ad1d` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:16.172257 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd56bb2-a911-4be4-97ec-b60b6cc7cb67 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=470d21b48683452bbca4f78884ff1182, role_id=b239f941cb724331bf28dc489b70ad1d)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:16.173745 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd56bb2-a911-4be4-97ec-b60b6cc7cb67 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:16.188199 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:16.190871 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 23/43] 10.222.0.84 () {58 vars in 1361 bytes} [Thu May 19 20:17:15 2022] PUT /identity/v3/domains/default/users/470d21b48683452bbca4f78884ff1182/roles/b239f941cb724331bf28dc489b70ad1d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:17:16.191825 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd56bb2-a911-4be4-97ec-b60b6cc7cb67 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:16.219289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b40a5-acf1-4d78-bcba-d41ea457e56d None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:16.273956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b3d51-6943-432a-80cb-968417408d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:16.274615 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b3d51-6943-432a-80cb-968417408d9a None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:16.274906 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b3d51-6943-432a-80cb-968417408d9a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:16.275190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b3d51-6943-432a-80cb-968417408d9a None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:16.277402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b3d51-6943-432a-80cb-968417408d9a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:16.279094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b3d51-6943-432a-80cb-968417408d9a None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:16.300994 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 21/44] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 20:17:16 2022] POST /identity/v3/roles => generated 223 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:17:16.359341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd56bb2-a911-4be4-97ec-b60b6cc7cb67 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:16.401003 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee42a50-7d52-44a9-b9d9-d75077813fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:16.402600 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee42a50-7d52-44a9-b9d9-d75077813fe2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:16.402600 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee42a50-7d52-44a9-b9d9-d75077813fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:16.402600 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee42a50-7d52-44a9-b9d9-d75077813fe2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:16.403897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee42a50-7d52-44a9-b9d9-d75077813fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:16.405150 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee42a50-7d52-44a9-b9d9-d75077813fe2 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:16.407611 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 24/45] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:16.412789 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b3d51-6943-432a-80cb-968417408d9a None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:16.534620 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b3d51-6943-432a-80cb-968417408d9a None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:16.577922 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa73f51-ef3f-4703-bd9a-ab79247fb03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:16.578660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa73f51-ef3f-4703-bd9a-ab79247fb03a None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:16.578947 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa73f51-ef3f-4703-bd9a-ab79247fb03a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:16.579224 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa73f51-ef3f-4703-bd9a-ab79247fb03a None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:16.579963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa73f51-ef3f-4703-bd9a-ab79247fb03a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:16.581439 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa73f51-ef3f-4703-bd9a-ab79247fb03a None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:16.586621 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:16.613195 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 22/46] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:17:16 2022] POST /identity/v3/projects => generated 312 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:17:18.523844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca16d18-195d-4591-b6cd-88bbc0d96e89 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:18.524390 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca16d18-195d-4591-b6cd-88bbc0d96e89 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:18.524390 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca16d18-195d-4591-b6cd-88bbc0d96e89 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:18.525967 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 25/47] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:18.532012 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcff2ad-fb29-4d94-9ef0-f46b64f868f4 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:18.532431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcff2ad-fb29-4d94-9ef0-f46b64f868f4 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:18.532611 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcff2ad-fb29-4d94-9ef0-f46b64f868f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:18.551200 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-8fcff2ad-fb29-4d94-9ef0-f46b64f868f4 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:18.552416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-8fcff2ad-fb29-4d94-9ef0-f46b64f868f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:18.639878 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 23/48] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:18.670122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca16d18-195d-4591-b6cd-88bbc0d96e89 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:18.713260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:18.783443 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca16d18-195d-4591-b6cd-88bbc0d96e89 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:18.818917 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8ef4-81c0-4385-b9ba-59fe921b832c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:18.819734 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8ef4-81c0-4385-b9ba-59fe921b832c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:18.820020 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8ef4-81c0-4385-b9ba-59fe921b832c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:18.820352 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8ef4-81c0-4385-b9ba-59fe921b832c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:18.831389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea8ef4-81c0-4385-b9ba-59fe921b832c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-8aea8ef4-81c0-4385-b9ba-59fe921b832c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:18.839175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:18.840581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:18.841513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:18.841513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:18.841513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:18.841513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:18.841513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:18.841513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:18.841513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:18.841513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:18.841513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:18.841513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:18.841513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:18.841513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:18.841513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 26/49] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:17:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:18.843051 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47a6d0-10f2-459b-a29c-eb4923943ed8 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:18.843051 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47a6d0-10f2-459b-a29c-eb4923943ed8 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:18.843051 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47a6d0-10f2-459b-a29c-eb4923943ed8 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:18.844296 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 24/50] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:18.846703 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8ef4-81c0-4385-b9ba-59fe921b832c None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:18.849181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89540c7-ba0a-4ddf-ac1d-1fdb506d5572 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:18.849471 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89540c7-ba0a-4ddf-ac1d-1fdb506d5572 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:18.849755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89540c7-ba0a-4ddf-ac1d-1fdb506d5572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:18.866016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-f89540c7-ba0a-4ddf-ac1d-1fdb506d5572 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:18.867213 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-f89540c7-ba0a-4ddf-ac1d-1fdb506d5572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:18.935548 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 25/51] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:17:18.952390 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:18.959047 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8ef4-81c0-4385-b9ba-59fe921b832c None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:18.968388 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89540c7-ba0a-4ddf-ac1d-1fdb506d5572 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:18.993957 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae94493-3f0c-4f35-90dc-0652f0f25988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:18.994664 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae94493-3f0c-4f35-90dc-0652f0f25988 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:18.995047 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae94493-3f0c-4f35-90dc-0652f0f25988 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:18.995608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae94493-3f0c-4f35-90dc-0652f0f25988 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:18.996117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae94493-3f0c-4f35-90dc-0652f0f25988 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:18.996424 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae94493-3f0c-4f35-90dc-0652f0f25988 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:18.997826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae94493-3f0c-4f35-90dc-0652f0f25988 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:19.006568 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 27/52] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:19.012177 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae94493-3f0c-4f35-90dc-0652f0f25988 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:19.019874 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:19.072586 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89540c7-ba0a-4ddf-ac1d-1fdb506d5572 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:19.108264 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30927085-dd5f-45d6-b9da-98bd17077548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:19.109075 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30927085-dd5f-45d6-b9da-98bd17077548 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:19.109364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30927085-dd5f-45d6-b9da-98bd17077548 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:19.109646 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30927085-dd5f-45d6-b9da-98bd17077548 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:19.110875 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30927085-dd5f-45d6-b9da-98bd17077548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:19.111892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae94493-3f0c-4f35-90dc-0652f0f25988 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:19.112341 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30927085-dd5f-45d6-b9da-98bd17077548 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:19.113978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 26/53] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:19.119689 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30927085-dd5f-45d6-b9da-98bd17077548 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:19.145937 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a24060-74bf-43c3-8225-532a24baa9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:19.146852 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a24060-74bf-43c3-8225-532a24baa9b4 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:19.147135 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a24060-74bf-43c3-8225-532a24baa9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:19.147411 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a24060-74bf-43c3-8225-532a24baa9b4 None admin] PATH_INFO: `/v3/users/470d21b48683452bbca4f78884ff1182` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:19.148613 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a24060-74bf-43c3-8225-532a24baa9b4 None admin] RBAC: Authorizing `identity:get_user(user_id=470d21b48683452bbca4f78884ff1182)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:19.150076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a24060-74bf-43c3-8225-532a24baa9b4 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:19.156101 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 28/54] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:19 2022] GET /identity/v3/users/470d21b48683452bbca4f78884ff1182 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:17:19.157066 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a24060-74bf-43c3-8225-532a24baa9b4 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:19.182157 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:19.223598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30927085-dd5f-45d6-b9da-98bd17077548 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:19.241375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:19.258287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a24060-74bf-43c3-8225-532a24baa9b4 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:19.260351 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f8f63-9c0f-4f1e-b953-a5d850511703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:19.261458 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f8f63-9c0f-4f1e-b953-a5d850511703 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:19.261747 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f8f63-9c0f-4f1e-b953-a5d850511703 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:19.262028 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f8f63-9c0f-4f1e-b953-a5d850511703 None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:19.262789 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f8f63-9c0f-4f1e-b953-a5d850511703 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:19.264265 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f8f63-9c0f-4f1e-b953-a5d850511703 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:19.289860 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 27/55] 10.222.0.84 () {60 vars in 1140 bytes} [Thu May 19 20:17:19 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:17:19.292682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b0cc1-63e6-4213-9931-7e6ea3a15fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:19.293707 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b0cc1-63e6-4213-9931-7e6ea3a15fe3 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:19.293994 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b0cc1-63e6-4213-9931-7e6ea3a15fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:19.294284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b0cc1-63e6-4213-9931-7e6ea3a15fe3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:19.295564 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b0cc1-63e6-4213-9931-7e6ea3a15fe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:19.297063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b0cc1-63e6-4213-9931-7e6ea3a15fe3 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:19.298804 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 29/56] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:19.304870 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f8f63-9c0f-4f1e-b953-a5d850511703 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:19.415395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f8f63-9c0f-4f1e-b953-a5d850511703 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:19.454624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062f587-8b41-4e11-aba1-c776da472961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:19.455297 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062f587-8b41-4e11-aba1-c776da472961 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:19.455582 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062f587-8b41-4e11-aba1-c776da472961 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:19.455867 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062f587-8b41-4e11-aba1-c776da472961 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:19.456722 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062f587-8b41-4e11-aba1-c776da472961 None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.domain.id=default, user.id=470d21b48683452bbca4f78884ff1182) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:19.457040 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062f587-8b41-4e11-aba1-c776da472961 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:19.458558 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062f587-8b41-4e11-aba1-c776da472961 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:19.468697 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 28/57] 10.222.0.84 () {56 vars in 1322 bytes} [Thu May 19 20:17:19 2022] GET /identity/v3/role_assignments?user.id=470d21b48683452bbca4f78884ff1182&scope.domain.id=default&role.id=b239f941cb724331bf28dc489b70ad1d => generated 530 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:21.587748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3d730-43e5-4325-ae11-bfc41acddb6d None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:21.588367 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3d730-43e5-4325-ae11-bfc41acddb6d None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:21.588367 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3d730-43e5-4325-ae11-bfc41acddb6d None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:21.590374 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 30/58] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:21.596806 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f92eaf-e328-422b-8fdf-b55c355d7fa2 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:21.597551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f92eaf-e328-422b-8fdf-b55c355d7fa2 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:21.597751 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f92eaf-e328-422b-8fdf-b55c355d7fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:21.619151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-d0f92eaf-e328-422b-8fdf-b55c355d7fa2 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:21.620750 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-d0f92eaf-e328-422b-8fdf-b55c355d7fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:21.650998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:21.695426 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 29/59] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:21.721801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3d730-43e5-4325-ae11-bfc41acddb6d None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:21.826441 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:21.843087 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3d730-43e5-4325-ae11-bfc41acddb6d None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:21.896260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19774703-3f1a-47be-9b89-e5a9ff2c4bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:21.897073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19774703-3f1a-47be-9b89-e5a9ff2c4bf9 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:21.897363 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19774703-3f1a-47be-9b89-e5a9ff2c4bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:21.897646 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19774703-3f1a-47be-9b89-e5a9ff2c4bf9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:21.898998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19774703-3f1a-47be-9b89-e5a9ff2c4bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:21.900511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19774703-3f1a-47be-9b89-e5a9ff2c4bf9 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:21.902647 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 31/60] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:21.908674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f92eaf-e328-422b-8fdf-b55c355d7fa2 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:22.018377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f92eaf-e328-422b-8fdf-b55c355d7fa2 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:22.054579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d18d34-35f0-4005-a76a-3fa90017d888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:22.055264 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d18d34-35f0-4005-a76a-3fa90017d888 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:22.055554 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d18d34-35f0-4005-a76a-3fa90017d888 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:22.055837 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d18d34-35f0-4005-a76a-3fa90017d888 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:22.056581 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d18d34-35f0-4005-a76a-3fa90017d888 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:22.058496 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d18d34-35f0-4005-a76a-3fa90017d888 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:22.089305 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 30/61] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:17:21 2022] POST /identity/v3/users => generated 275 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:17:25.547126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff95317-b302-45b8-9ec0-a60ba5045e01 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:25.548096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff95317-b302-45b8-9ec0-a60ba5045e01 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:25.548568 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff95317-b302-45b8-9ec0-a60ba5045e01 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:25.550188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 32/62] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:25.555847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bd061-bea7-42b6-b2c2-64936839bf4e None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:25.556692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bd061-bea7-42b6-b2c2-64936839bf4e None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:25.557327 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bd061-bea7-42b6-b2c2-64936839bf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:25.570277 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a905b11-a2aa-4c5d-b162-f4b0dd2df4d1 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:25.570570 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a905b11-a2aa-4c5d-b162-f4b0dd2df4d1 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:25.570850 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a905b11-a2aa-4c5d-b162-f4b0dd2df4d1 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:25.572264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 33/63] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:17:25.579760 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-726bd061-bea7-42b6-b2c2-64936839bf4e None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:25.581116 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-726bd061-bea7-42b6-b2c2-64936839bf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:25.586065 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362d41b-b235-409c-aab6-3a664be525ba None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:25.586663 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362d41b-b235-409c-aab6-3a664be525ba None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:25.587094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362d41b-b235-409c-aab6-3a664be525ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:25.603083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:25.627701 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-f362d41b-b235-409c-aab6-3a664be525ba None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:25.628903 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-f362d41b-b235-409c-aab6-3a664be525ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:25.690167 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 31/64] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:25.714090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bd061-bea7-42b6-b2c2-64936839bf4e None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:25.768684 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 34/65] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:17:25.808215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362d41b-b235-409c-aab6-3a664be525ba None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:25.857757 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bd061-bea7-42b6-b2c2-64936839bf4e None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:25.909254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e0d37-3909-4a95-99dc-6fb11663506b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:25.910108 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e0d37-3909-4a95-99dc-6fb11663506b None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:25.910428 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e0d37-3909-4a95-99dc-6fb11663506b None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:25.910716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e0d37-3909-4a95-99dc-6fb11663506b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:25.916500 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:25.929103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e0d37-3909-4a95-99dc-6fb11663506b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:25.929968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362d41b-b235-409c-aab6-3a664be525ba None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-674e0d37-3909-4a95-99dc-6fb11663506b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:25.939486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:25.940605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:25.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:25.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:25.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:25.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:25.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:25.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:25.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:25.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:25.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:25.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:25.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:17:25.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:25.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 32/66] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:17:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:17:25.943164 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a64e3-8217-4b1e-8313-16bad6e7ede6 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:25.943164 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a64e3-8217-4b1e-8313-16bad6e7ede6 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:25.943164 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a64e3-8217-4b1e-8313-16bad6e7ede6 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:25.948030 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 33/67] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:25.949335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671ea9b-7963-4dab-9f5e-5005d5b510c8 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:25.949626 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671ea9b-7963-4dab-9f5e-5005d5b510c8 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:25.949908 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671ea9b-7963-4dab-9f5e-5005d5b510c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:25.970094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-6671ea9b-7963-4dab-9f5e-5005d5b510c8 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:25.971623 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-6671ea9b-7963-4dab-9f5e-5005d5b510c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:25.975524 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:25.986930 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca982f-47ee-4917-859d-bb2b49f826b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:25.987748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca982f-47ee-4917-859d-bb2b49f826b4 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:25.988035 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca982f-47ee-4917-859d-bb2b49f826b4 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:25.988319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca982f-47ee-4917-859d-bb2b49f826b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:26.003311 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca982f-47ee-4917-859d-bb2b49f826b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-ffca982f-47ee-4917-859d-bb2b49f826b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:26.014075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:26.015173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:26.016119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:26.016119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:26.016119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:26.016119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:26.016119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:26.016119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:26.016119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:26.016119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:26.016119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:26.016119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:26.016119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:26.016119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:26.016119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 35/68] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:17:26.016898 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca982f-47ee-4917-859d-bb2b49f826b4 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:26.056223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 34/69] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:26.057586 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671ea9b-7963-4dab-9f5e-5005d5b510c8 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:26.149045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca982f-47ee-4917-859d-bb2b49f826b4 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:26.186603 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671ea9b-7963-4dab-9f5e-5005d5b510c8 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:26.188839 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289197b-967e-4184-a94a-1bc4dc717fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:26.190376 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289197b-967e-4184-a94a-1bc4dc717fa0 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:26.190734 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289197b-967e-4184-a94a-1bc4dc717fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:26.190967 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289197b-967e-4184-a94a-1bc4dc717fa0 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:26.191740 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289197b-967e-4184-a94a-1bc4dc717fa0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:26.192057 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289197b-967e-4184-a94a-1bc4dc717fa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:26.193478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289197b-967e-4184-a94a-1bc4dc717fa0 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:26.202734 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 36/70] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:17:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:26.208564 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01113e5e-924d-4f4c-82c5-c9d479f39862 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:26.208857 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01113e5e-924d-4f4c-82c5-c9d479f39862 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:26.209139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01113e5e-924d-4f4c-82c5-c9d479f39862 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:26.215175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 37/71] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:26.215175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01113e5e-924d-4f4c-82c5-c9d479f39862 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:26.230624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecf6de-f59f-4460-bff8-d44e45b1ed59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:26.231339 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecf6de-f59f-4460-bff8-d44e45b1ed59 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:26.231624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecf6de-f59f-4460-bff8-d44e45b1ed59 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:26.231900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecf6de-f59f-4460-bff8-d44e45b1ed59 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:26.232654 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecf6de-f59f-4460-bff8-d44e45b1ed59 None admin] RBAC: Adding query filter params (name=member) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:26.233001 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecf6de-f59f-4460-bff8-d44e45b1ed59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:26.233486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:26.234972 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecf6de-f59f-4460-bff8-d44e45b1ed59 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:26.244166 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:26.253213 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 35/72] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:26.253213 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecf6de-f59f-4460-bff8-d44e45b1ed59 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:26.326536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01113e5e-924d-4f4c-82c5-c9d479f39862 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:26.352931 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecf6de-f59f-4460-bff8-d44e45b1ed59 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:26.363177 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763238b-0059-456e-9345-bf27d4dcbb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:26.363972 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763238b-0059-456e-9345-bf27d4dcbb0d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:26.364242 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763238b-0059-456e-9345-bf27d4dcbb0d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:26.364520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763238b-0059-456e-9345-bf27d4dcbb0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:26.377740 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763238b-0059-456e-9345-bf27d4dcbb0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-c763238b-0059-456e-9345-bf27d4dcbb0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:26.385562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:26.386866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:26.388522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:26.388522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:26.388522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:26.388522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:26.388522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:26.388522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:26.388522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:26.388522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:26.388522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:26.388522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:26.388522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:26.388522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:26.388522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 38/73] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:17:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:26.388522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af33029-f96a-4028-9cc3-e149bc5be830 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:26.389613 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af33029-f96a-4028-9cc3-e149bc5be830 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:26.389613 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af33029-f96a-4028-9cc3-e149bc5be830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:26.402060 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90442f36-44c1-465e-8e88-4f3061d69252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:26.402984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90442f36-44c1-465e-8e88-4f3061d69252 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:26.403267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90442f36-44c1-465e-8e88-4f3061d69252 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:26.403522 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90442f36-44c1-465e-8e88-4f3061d69252 None admin] PATH_INFO: `/v3/users/d5972d97f8c144e99fb7fb22399f0d94` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:26.405286 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-8af33029-f96a-4028-9cc3-e149bc5be830 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:26.406523 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-8af33029-f96a-4028-9cc3-e149bc5be830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:26.418871 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90442f36-44c1-465e-8e88-4f3061d69252 None admin] RBAC: Authorizing `identity:get_user(user_id=d5972d97f8c144e99fb7fb22399f0d94)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:26.420325 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90442f36-44c1-465e-8e88-4f3061d69252 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:26.422051 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 36/74] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:26 2022] GET /identity/v3/users/d5972d97f8c144e99fb7fb22399f0d94 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:26.423318 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed023b-e2c4-4325-997d-254234733b87 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:26.423600 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed023b-e2c4-4325-997d-254234733b87 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:26.423878 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed023b-e2c4-4325-997d-254234733b87 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:26.425107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 37/75] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:26.430477 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed023b-e2c4-4325-997d-254234733b87 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:26.502318 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 39/76] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:26.502544 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:26.502657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af33029-f96a-4028-9cc3-e149bc5be830 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:26.502657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:26.536431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed023b-e2c4-4325-997d-254234733b87 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:26.573454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7f105-f215-4bca-9237-b2006efa32dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:26.574397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7f105-f215-4bca-9237-b2006efa32dc None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:26.574688 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7f105-f215-4bca-9237-b2006efa32dc None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:26.574970 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7f105-f215-4bca-9237-b2006efa32dc None admin] PATH_INFO: `/v3/users/d5972d97f8c144e99fb7fb22399f0d94` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:26.576203 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7f105-f215-4bca-9237-b2006efa32dc None admin] RBAC: Authorizing `identity:get_user(user_id=d5972d97f8c144e99fb7fb22399f0d94)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:26.577684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7f105-f215-4bca-9237-b2006efa32dc None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:26.579389 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 38/77] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:26 2022] GET /identity/v3/users/d5972d97f8c144e99fb7fb22399f0d94 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:26.580168 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7f105-f215-4bca-9237-b2006efa32dc None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:26.601076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af33029-f96a-4028-9cc3-e149bc5be830 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:26.636222 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a956a7-ef67-429f-98b9-c1cb63278575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:26.636894 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a956a7-ef67-429f-98b9-c1cb63278575 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:26.637181 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a956a7-ef67-429f-98b9-c1cb63278575 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:26.637461 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a956a7-ef67-429f-98b9-c1cb63278575 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:26.638221 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a956a7-ef67-429f-98b9-c1cb63278575 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:26.638569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a956a7-ef67-429f-98b9-c1cb63278575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:26.639980 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a956a7-ef67-429f-98b9-c1cb63278575 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:26.648203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 40/78] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:26.649416 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c0d0c-c999-4a99-af25-c55bfa1f579a None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:26.649706 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c0d0c-c999-4a99-af25-c55bfa1f579a None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:26.649988 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c0d0c-c999-4a99-af25-c55bfa1f579a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:26.667469 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-eb5c0d0c-c999-4a99-af25-c55bfa1f579a None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:26.668667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-eb5c0d0c-c999-4a99-af25-c55bfa1f579a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:26.687138 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7f105-f215-4bca-9237-b2006efa32dc None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:26.712570 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:26.721367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93deea10-a40a-443d-a4e0-9f4d32e47e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:26.722022 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deea10-a40a-443d-a4e0-9f4d32e47e04 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:26.722326 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deea10-a40a-443d-a4e0-9f4d32e47e04 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:26.722611 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deea10-a40a-443d-a4e0-9f4d32e47e04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:26.728322 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:26.737321 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 41/79] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:26.738147 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c0d0c-c999-4a99-af25-c55bfa1f579a None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:26.790104 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93deea10-a40a-443d-a4e0-9f4d32e47e04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:26.791825 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93deea10-a40a-443d-a4e0-9f4d32e47e04 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:26.845198 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c0d0c-c999-4a99-af25-c55bfa1f579a None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:26.880141 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5a0b5-7182-40c9-94fa-807baa0e74e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:26.880946 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5a0b5-7182-40c9-94fa-807baa0e74e1 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:26.881237 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5a0b5-7182-40c9-94fa-807baa0e74e1 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:26.881486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5a0b5-7182-40c9-94fa-807baa0e74e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:26.892657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5a0b5-7182-40c9-94fa-807baa0e74e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:26.896420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 39/80] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:26.897327 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93deea10-a40a-443d-a4e0-9f4d32e47e04 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-78a5a0b5-7182-40c9-94fa-807baa0e74e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:26.900606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:26.901751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:26.902705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:26.902705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:26.902705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:26.902705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:26.902705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:26.902705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:26.902705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:26.902705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:26.902705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:26.902705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:26.902705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:26.902705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:26.902705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 42/81] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:26.906513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5a0b5-7182-40c9-94fa-807baa0e74e1 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:26.924861 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:26.961710 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:26.998980 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93deea10-a40a-443d-a4e0-9f4d32e47e04 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:27.007630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5a0b5-7182-40c9-94fa-807baa0e74e1 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:27.033453 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188a77a-afe9-41b8-9eac-544526a91b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:27.034136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188a77a-afe9-41b8-9eac-544526a91b7c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:27.034446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188a77a-afe9-41b8-9eac-544526a91b7c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:27.034727 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188a77a-afe9-41b8-9eac-544526a91b7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:27.041846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb049224-8483-4e26-a2ec-1ea3de320e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:27.042693 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb049224-8483-4e26-a2ec-1ea3de320e14 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:27.042978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb049224-8483-4e26-a2ec-1ea3de320e14 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:27.043260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb049224-8483-4e26-a2ec-1ea3de320e14 None admin] PATH_INFO: `/v3/users/470d21b48683452bbca4f78884ff1182` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:27.044476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb049224-8483-4e26-a2ec-1ea3de320e14 None admin] RBAC: Authorizing `identity:get_user(user_id=470d21b48683452bbca4f78884ff1182)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:27.045968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb049224-8483-4e26-a2ec-1ea3de320e14 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:27.047650 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 43/82] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:26 2022] GET /identity/v3/users/470d21b48683452bbca4f78884ff1182 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:27.052179 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb049224-8483-4e26-a2ec-1ea3de320e14 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:27.103724 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188a77a-afe9-41b8-9eac-544526a91b7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:27.105265 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188a77a-afe9-41b8-9eac-544526a91b7c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:27.135396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:27.149994 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb049224-8483-4e26-a2ec-1ea3de320e14 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:27.173901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:27.187230 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94df93-e43e-40d5-9073-a65df31a0e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:27.188042 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94df93-e43e-40d5-9073-a65df31a0e6a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:27.188330 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94df93-e43e-40d5-9073-a65df31a0e6a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:27.188621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94df93-e43e-40d5-9073-a65df31a0e6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:27.189813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94df93-e43e-40d5-9073-a65df31a0e6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:27.191317 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94df93-e43e-40d5-9073-a65df31a0e6a None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:27.192965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 44/83] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:27.193739 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94df93-e43e-40d5-9073-a65df31a0e6a None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:27.220244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 40/84] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:27.221109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188a77a-afe9-41b8-9eac-544526a91b7c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:27.297600 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94df93-e43e-40d5-9073-a65df31a0e6a None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:27.326182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188a77a-afe9-41b8-9eac-544526a91b7c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:27.335944 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51019890-9c7c-46b1-a822-63d143d5b95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:27.336845 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51019890-9c7c-46b1-a822-63d143d5b95f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:27.337136 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51019890-9c7c-46b1-a822-63d143d5b95f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:27.337420 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51019890-9c7c-46b1-a822-63d143d5b95f None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:27.339061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51019890-9c7c-46b1-a822-63d143d5b95f None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:27.340908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51019890-9c7c-46b1-a822-63d143d5b95f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:27.342679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 45/85] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:27 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:27.348186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51019890-9c7c-46b1-a822-63d143d5b95f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:27.353713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:27.364460 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98ca94-9c85-4901-8254-ff2fb712ddf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:27.365150 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98ca94-9c85-4901-8254-ff2fb712ddf5 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:27.365438 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98ca94-9c85-4901-8254-ff2fb712ddf5 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:27.365720 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98ca94-9c85-4901-8254-ff2fb712ddf5 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:27.366523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98ca94-9c85-4901-8254-ff2fb712ddf5 None admin] RBAC: Adding query filter params (name=member) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:27.366837 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98ca94-9c85-4901-8254-ff2fb712ddf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:27.368219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98ca94-9c85-4901-8254-ff2fb712ddf5 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:27.376996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 41/86] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:27.377780 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98ca94-9c85-4901-8254-ff2fb712ddf5 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:27.408733 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:27.455184 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51019890-9c7c-46b1-a822-63d143d5b95f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:27.484995 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98ca94-9c85-4901-8254-ff2fb712ddf5 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:27.490233 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2cba63-c771-4611-942d-0caba48be45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:27.490943 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cba63-c771-4611-942d-0caba48be45b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:27.491234 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cba63-c771-4611-942d-0caba48be45b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:27.491523 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cba63-c771-4611-942d-0caba48be45b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:27.519813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164d3e8-a20f-4c52-88a4-b2e689fa7d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:27.520515 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164d3e8-a20f-4c52-88a4-b2e689fa7d53 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:27.520814 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164d3e8-a20f-4c52-88a4-b2e689fa7d53 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:27.521099 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164d3e8-a20f-4c52-88a4-b2e689fa7d53 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:27.521836 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164d3e8-a20f-4c52-88a4-b2e689fa7d53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:27.523364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164d3e8-a20f-4c52-88a4-b2e689fa7d53 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:27.544471 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 42/87] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:17:27 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:17:27.551565 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164d3e8-a20f-4c52-88a4-b2e689fa7d53 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:27.562962 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2cba63-c771-4611-942d-0caba48be45b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:27.564523 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2cba63-c771-4611-942d-0caba48be45b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:27.612630 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:27.630452 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:27.679759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 46/88] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:27.680954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2cba63-c771-4611-942d-0caba48be45b None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:27.688461 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164d3e8-a20f-4c52-88a4-b2e689fa7d53 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:27.729142 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40552b3-4ada-4128-b314-2a22d6af89aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:27.730028 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40552b3-4ada-4128-b314-2a22d6af89aa None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:27.730295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40552b3-4ada-4128-b314-2a22d6af89aa None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:27.730587 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40552b3-4ada-4128-b314-2a22d6af89aa None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:27.731832 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40552b3-4ada-4128-b314-2a22d6af89aa None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:27.733313 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40552b3-4ada-4128-b314-2a22d6af89aa None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:27.735398 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 43/89] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:27 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:27.736220 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40552b3-4ada-4128-b314-2a22d6af89aa None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:27.810775 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2cba63-c771-4611-942d-0caba48be45b None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:27.855698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758ca5c-cbeb-46e7-ac89-d3a2ec9f70a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:27.856012 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40552b3-4ada-4128-b314-2a22d6af89aa None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:27.856384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758ca5c-cbeb-46e7-ac89-d3a2ec9f70a1 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:27.856675 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758ca5c-cbeb-46e7-ac89-d3a2ec9f70a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:27.856959 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758ca5c-cbeb-46e7-ac89-d3a2ec9f70a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:27.857996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758ca5c-cbeb-46e7-ac89-d3a2ec9f70a1 None admin] RBAC: Adding query filter params (role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2, scope.project.id=ba315bae61f943debd8460a5fa9dd151, user.id=d5972d97f8c144e99fb7fb22399f0d94) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:27.858347 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758ca5c-cbeb-46e7-ac89-d3a2ec9f70a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:27.860756 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758ca5c-cbeb-46e7-ac89-d3a2ec9f70a1 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:27.868994 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 47/90] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:27 2022] GET /identity/v3/role_assignments?user.id=d5972d97f8c144e99fb7fb22399f0d94&scope.project.id=ba315bae61f943debd8460a5fa9dd151&role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:27.869207 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758ca5c-cbeb-46e7-ac89-d3a2ec9f70a1 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:27.875155 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:27.908969 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:27.912174 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7d362-98ec-4824-9242-7f460ee11739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:27.914528 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7d362-98ec-4824-9242-7f460ee11739 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:27.914528 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7d362-98ec-4824-9242-7f460ee11739 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:27.914528 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7d362-98ec-4824-9242-7f460ee11739 None admin] PATH_INFO: `/v3/users/d5972d97f8c144e99fb7fb22399f0d94` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:27.916578 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7d362-98ec-4824-9242-7f460ee11739 None admin] RBAC: Authorizing `identity:get_user(user_id=d5972d97f8c144e99fb7fb22399f0d94)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:27.918757 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7d362-98ec-4824-9242-7f460ee11739 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:27.921818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 44/91] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:27 2022] GET /identity/v3/users/d5972d97f8c144e99fb7fb22399f0d94 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:27.922939 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7d362-98ec-4824-9242-7f460ee11739 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:28.014889 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758ca5c-cbeb-46e7-ac89-d3a2ec9f70a1 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:28.050148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7d362-98ec-4824-9242-7f460ee11739 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:28.052258 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffd17d-723a-447b-93ad-f32f9c5f7cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:28.053162 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffd17d-723a-447b-93ad-f32f9c5f7cbc None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:28.053450 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffd17d-723a-447b-93ad-f32f9c5f7cbc None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:28.053755 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffd17d-723a-447b-93ad-f32f9c5f7cbc None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:28.055111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdffd17d-723a-447b-93ad-f32f9c5f7cbc None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:28.056690 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdffd17d-723a-447b-93ad-f32f9c5f7cbc None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:28.059219 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 48/92] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:27 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:28.059847 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffd17d-723a-447b-93ad-f32f9c5f7cbc None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:28.091900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d716f-caec-4130-94e2-f85710532ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:28.093301 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d716f-caec-4130-94e2-f85710532ebf None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:28.093851 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d716f-caec-4130-94e2-f85710532ebf None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:28.094355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d716f-caec-4130-94e2-f85710532ebf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:28.095413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d716f-caec-4130-94e2-f85710532ebf None admin] RBAC: Adding query filter params (role.id=666cfd1181184612ba462b8eefcad6a7, scope.project.id=ba315bae61f943debd8460a5fa9dd151, user.id=d5972d97f8c144e99fb7fb22399f0d94) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:28.095931 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d716f-caec-4130-94e2-f85710532ebf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:28.097594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d716f-caec-4130-94e2-f85710532ebf None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:28.104909 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 45/93] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:27 2022] GET /identity/v3/role_assignments?user.id=d5972d97f8c144e99fb7fb22399f0d94&scope.project.id=ba315bae61f943debd8460a5fa9dd151&role.id=666cfd1181184612ba462b8eefcad6a7 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:28.106164 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d716f-caec-4130-94e2-f85710532ebf None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:28.226000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:28.242690 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:28.250105 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffd17d-723a-447b-93ad-f32f9c5f7cbc None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:28.309086 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802781c-0db5-4ad3-a352-5f21044b0914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:28.310217 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802781c-0db5-4ad3-a352-5f21044b0914 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:28.310797 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802781c-0db5-4ad3-a352-5f21044b0914 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:28.311313 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802781c-0db5-4ad3-a352-5f21044b0914 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:28.376626 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d716f-caec-4130-94e2-f85710532ebf None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:28.431796 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142dfb7-268f-42be-9c39-ca07f7070e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:28.433314 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142dfb7-268f-42be-9c39-ca07f7070e08 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:28.433567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142dfb7-268f-42be-9c39-ca07f7070e08 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:28.433852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142dfb7-268f-42be-9c39-ca07f7070e08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:28.434741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142dfb7-268f-42be-9c39-ca07f7070e08 None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=ba315bae61f943debd8460a5fa9dd151, user.id=470d21b48683452bbca4f78884ff1182) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:28.435068 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142dfb7-268f-42be-9c39-ca07f7070e08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:28.436556 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142dfb7-268f-42be-9c39-ca07f7070e08 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:28.444651 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 46/94] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:28 2022] GET /identity/v3/role_assignments?user.id=470d21b48683452bbca4f78884ff1182&scope.project.id=ba315bae61f943debd8460a5fa9dd151&role.id=b239f941cb724331bf28dc489b70ad1d => generated 259 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:28.496851 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802781c-0db5-4ad3-a352-5f21044b0914 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:28.498806 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802781c-0db5-4ad3-a352-5f21044b0914 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:28.635405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:28.653169 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 49/95] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:28.659265 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142dfb7-268f-42be-9c39-ca07f7070e08 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:28.782518 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142dfb7-268f-42be-9c39-ca07f7070e08 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:28.795281 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:28.831970 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b658ba0-213c-443e-8f25-8d63df1a7378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:28.832875 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b658ba0-213c-443e-8f25-8d63df1a7378 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:28.833163 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b658ba0-213c-443e-8f25-8d63df1a7378 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:28.833445 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b658ba0-213c-443e-8f25-8d63df1a7378 None admin] PATH_INFO: `/v3/projects/3ae9b8c5024e40f0b79dcf1d40c57424` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:28.834766 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b658ba0-213c-443e-8f25-8d63df1a7378 None admin] RBAC: Authorizing `identity:get_project(project_id=3ae9b8c5024e40f0b79dcf1d40c57424)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:28.836242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b658ba0-213c-443e-8f25-8d63df1a7378 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:28.838898 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 47/96] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:28 2022] GET /identity/v3/projects/3ae9b8c5024e40f0b79dcf1d40c57424 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:28.846302 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802781c-0db5-4ad3-a352-5f21044b0914 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:28.988968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802781c-0db5-4ad3-a352-5f21044b0914 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:29.029346 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ad996-1233-4344-84be-1b1ed9489332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:29.030482 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ad996-1233-4344-84be-1b1ed9489332 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:29.031003 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ad996-1233-4344-84be-1b1ed9489332 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:29.031696 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ad996-1233-4344-84be-1b1ed9489332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:29.032769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ad996-1233-4344-84be-1b1ed9489332 None admin] RBAC: Adding query filter params (role.id=666cfd1181184612ba462b8eefcad6a7, scope.project.id=3ae9b8c5024e40f0b79dcf1d40c57424, user.id=d5972d97f8c144e99fb7fb22399f0d94) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:29.033330 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ad996-1233-4344-84be-1b1ed9489332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:29.035153 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ad996-1233-4344-84be-1b1ed9489332 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:29.042959 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 50/97] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:28 2022] GET /identity/v3/role_assignments?user.id=d5972d97f8c144e99fb7fb22399f0d94&scope.project.id=3ae9b8c5024e40f0b79dcf1d40c57424&role.id=666cfd1181184612ba462b8eefcad6a7 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:30.892157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7e31d-c987-4ed2-b83c-7d9df00c054f None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:30.892737 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7e31d-c987-4ed2-b83c-7d9df00c054f None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:30.892737 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7e31d-c987-4ed2-b83c-7d9df00c054f None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:30.894184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 48/98] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:30.901440 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d5fd9-51b6-4987-a769-bb40ed42933b None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:30.902591 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d5fd9-51b6-4987-a769-bb40ed42933b None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:30.903091 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d5fd9-51b6-4987-a769-bb40ed42933b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:30.930712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-2e0d5fd9-51b6-4987-a769-bb40ed42933b None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:30.932400 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-2e0d5fd9-51b6-4987-a769-bb40ed42933b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:31.002630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:31.020948 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 51/99] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:31.063983 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7e31d-c987-4ed2-b83c-7d9df00c054f None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:31.112610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f17e9-08f3-4393-8f85-3206d718db69 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:31.113062 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f17e9-08f3-4393-8f85-3206d718db69 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:31.113346 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f17e9-08f3-4393-8f85-3206d718db69 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:31.115688 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 52/100] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:31.125679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc08685-af31-49ee-8e0c-c1918a4bc2c4 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:31.125995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc08685-af31-49ee-8e0c-c1918a4bc2c4 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:31.126249 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc08685-af31-49ee-8e0c-c1918a4bc2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:31.155738 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-5dc08685-af31-49ee-8e0c-c1918a4bc2c4 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:31.156853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-5dc08685-af31-49ee-8e0c-c1918a4bc2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:31.278073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7e31d-c987-4ed2-b83c-7d9df00c054f None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:31.281062 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 53/101] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:31.319394 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc08685-af31-49ee-8e0c-c1918a4bc2c4 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:31.338994 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47324c6-68b2-4808-a9fb-ec247363dc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:31.340196 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47324c6-68b2-4808-a9fb-ec247363dc07 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:31.340482 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47324c6-68b2-4808-a9fb-ec247363dc07 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:31.340759 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47324c6-68b2-4808-a9fb-ec247363dc07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:31.342036 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47324c6-68b2-4808-a9fb-ec247363dc07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:31.343533 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47324c6-68b2-4808-a9fb-ec247363dc07 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:31.345214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 49/102] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:31.351097 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47324c6-68b2-4808-a9fb-ec247363dc07 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:31.407427 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:31.481536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc08685-af31-49ee-8e0c-c1918a4bc2c4 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:31.507345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:31.509860 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47324c6-68b2-4808-a9fb-ec247363dc07 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:31.538173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271ce5c-6474-474d-8a2f-7dfb797e82f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:31.539308 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271ce5c-6474-474d-8a2f-7dfb797e82f4 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:31.543118 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271ce5c-6474-474d-8a2f-7dfb797e82f4 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:31.543592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271ce5c-6474-474d-8a2f-7dfb797e82f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:31.564490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271ce5c-6474-474d-8a2f-7dfb797e82f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-2271ce5c-6474-474d-8a2f-7dfb797e82f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:31.574250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:31.575583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:31.576773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:31.576773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:31.576773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:31.576773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:31.576773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:31.576773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:31.576773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:31.576773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:31.576773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:31.576773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:31.576773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:17:31.576773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:31.582038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74ebd8-dbb3-476f-9891-da714761ad36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:31.583005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74ebd8-dbb3-476f-9891-da714761ad36 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:31.583475 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74ebd8-dbb3-476f-9891-da714761ad36 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:31.583911 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74ebd8-dbb3-476f-9891-da714761ad36 None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:31.584828 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74ebd8-dbb3-476f-9891-da714761ad36 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:31.586541 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74ebd8-dbb3-476f-9891-da714761ad36 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:31.590076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 54/103] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:17:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:31.596896 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e75f3-d238-480b-8ddf-08299f5e319d None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:31.599652 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e75f3-d238-480b-8ddf-08299f5e319d None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:31.601127 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e75f3-d238-480b-8ddf-08299f5e319d None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:31.607586 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 55/104] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:31 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:31.608757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e75f3-d238-480b-8ddf-08299f5e319d None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:31.655318 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 50/105] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:17:31 2022] POST /identity/v3/projects => generated 302 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:17:31.656927 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d6ed0-d667-4fb3-8c01-279c7665d263 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:31.657387 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d6ed0-d667-4fb3-8c01-279c7665d263 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:31.659010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d6ed0-d667-4fb3-8c01-279c7665d263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:31.684662 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-032d6ed0-d667-4fb3-8c01-279c7665d263 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:31.686295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-032d6ed0-d667-4fb3-8c01-279c7665d263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:31.815503 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 51/106] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:31.843900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e024f57-7bb7-4a28-ab0b-2fe4c3731822 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:31.845169 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e024f57-7bb7-4a28-ab0b-2fe4c3731822 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:31.846003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e024f57-7bb7-4a28-ab0b-2fe4c3731822 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:31.848227 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 52/107] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:31.852526 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e024f57-7bb7-4a28-ab0b-2fe4c3731822 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:31.852903 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e75f3-d238-480b-8ddf-08299f5e319d None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:31.892457 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:31.952626 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90a283-ac42-4078-95da-592fea6afcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:31.953330 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90a283-ac42-4078-95da-592fea6afcfa None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:31.953566 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90a283-ac42-4078-95da-592fea6afcfa None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:31.953789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90a283-ac42-4078-95da-592fea6afcfa None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:31.954595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90a283-ac42-4078-95da-592fea6afcfa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:31.954911 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90a283-ac42-4078-95da-592fea6afcfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:31.956383 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90a283-ac42-4078-95da-592fea6afcfa None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:31.982179 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 56/108] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:17:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:31.983641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd05268-0735-4bc1-8090-92ee40ff6ad4 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:31.983940 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd05268-0735-4bc1-8090-92ee40ff6ad4 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:31.984226 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd05268-0735-4bc1-8090-92ee40ff6ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:32.015102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-5dd05268-0735-4bc1-8090-92ee40ff6ad4 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:32.016333 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-5dd05268-0735-4bc1-8090-92ee40ff6ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:32.025008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e024f57-7bb7-4a28-ab0b-2fe4c3731822 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:32.059041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:32.071098 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16debf1-45e5-411e-ae21-5631b3de3fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:32.072276 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16debf1-45e5-411e-ae21-5631b3de3fe0 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:32.072789 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16debf1-45e5-411e-ae21-5631b3de3fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:32.073502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16debf1-45e5-411e-ae21-5631b3de3fe0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:32.088610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16debf1-45e5-411e-ae21-5631b3de3fe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-c16debf1-45e5-411e-ae21-5631b3de3fe0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:32.097818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:32.099014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:32.099964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:32.099964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:32.099964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:32.099964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:32.099964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:32.099964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:32.099964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:32.099964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:32.099964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:32.099964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:32.099964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:17:32.099964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:32.099964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 53/109] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:17:32.100746 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16debf1-45e5-411e-ae21-5631b3de3fe0 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:32.125053 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 57/110] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:32.126609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbf9cc-8678-4c80-b2c6-aa9d7ae8e611 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:32.126905 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbf9cc-8678-4c80-b2c6-aa9d7ae8e611 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:32.127194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbf9cc-8678-4c80-b2c6-aa9d7ae8e611 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:32.129373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 58/111] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:32.130086 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbf9cc-8678-4c80-b2c6-aa9d7ae8e611 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:32.222881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:32.252906 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16debf1-45e5-411e-ae21-5631b3de3fe0 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:32.271720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbf9cc-8678-4c80-b2c6-aa9d7ae8e611 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:32.321885 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47771b65-84fa-4b4d-86c9-fe1e8684039e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:32.321885 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47771b65-84fa-4b4d-86c9-fe1e8684039e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:32.321885 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47771b65-84fa-4b4d-86c9-fe1e8684039e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:32.321885 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47771b65-84fa-4b4d-86c9-fe1e8684039e None admin] PATH_INFO: `/v3/users/d5972d97f8c144e99fb7fb22399f0d94` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:32.322433 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47771b65-84fa-4b4d-86c9-fe1e8684039e None admin] RBAC: Authorizing `identity:get_user(user_id=d5972d97f8c144e99fb7fb22399f0d94)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:32.322433 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47771b65-84fa-4b4d-86c9-fe1e8684039e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:32.325137 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 54/112] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:32 2022] GET /identity/v3/users/d5972d97f8c144e99fb7fb22399f0d94 => generated 275 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:32.330214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82966a-f08a-41db-8ddd-d0ea97526552 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:32.330214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82966a-f08a-41db-8ddd-d0ea97526552 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:32.330214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82966a-f08a-41db-8ddd-d0ea97526552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:32.332057 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cffb231-9cfd-48ca-a2fa-54a24671b5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:32.332745 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cffb231-9cfd-48ca-a2fa-54a24671b5f2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:32.333028 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cffb231-9cfd-48ca-a2fa-54a24671b5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:32.333303 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cffb231-9cfd-48ca-a2fa-54a24671b5f2 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:32.334069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cffb231-9cfd-48ca-a2fa-54a24671b5f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:32.334405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cffb231-9cfd-48ca-a2fa-54a24671b5f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:32.335830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cffb231-9cfd-48ca-a2fa-54a24671b5f2 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:32.346707 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 59/113] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:32.347530 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cffb231-9cfd-48ca-a2fa-54a24671b5f2 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:32.351935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-ac82966a-f08a-41db-8ddd-d0ea97526552 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:32.353410 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-ac82966a-f08a-41db-8ddd-d0ea97526552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:32.379788 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:32.452233 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 55/114] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:32.454074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef03cbf-d0b1-4627-88c7-07a5b395734f None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:32.454655 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef03cbf-d0b1-4627-88c7-07a5b395734f None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:32.455162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef03cbf-d0b1-4627-88c7-07a5b395734f None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:32.456926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 56/115] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:32.457832 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef03cbf-d0b1-4627-88c7-07a5b395734f None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:32.470449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cffb231-9cfd-48ca-a2fa-54a24671b5f2 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:32.514713 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49f365-c549-4eab-a082-946a0b05edc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:32.515508 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49f365-c549-4eab-a082-946a0b05edc9 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:32.515848 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49f365-c549-4eab-a082-946a0b05edc9 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:32.516130 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49f365-c549-4eab-a082-946a0b05edc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:32.528792 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49f365-c549-4eab-a082-946a0b05edc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-6f49f365-c549-4eab-a082-946a0b05edc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:32.536865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:32.538082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:32.539069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:32.539069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:32.539069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:32.539069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:32.539069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:32.539069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:32.539069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:32.539069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:32.539069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:32.539069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:32.539069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:32.539069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:32.539069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 60/116] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:17:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:32.539069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49f365-c549-4eab-a082-946a0b05edc9 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:32.564093 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:32.588445 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef03cbf-d0b1-4627-88c7-07a5b395734f None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:32.640129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e5824-1d2d-44c1-b064-455e97a3baab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:32.641057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e5824-1d2d-44c1-b064-455e97a3baab None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:32.641336 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e5824-1d2d-44c1-b064-455e97a3baab None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:32.641612 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e5824-1d2d-44c1-b064-455e97a3baab None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:32.643108 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e5824-1d2d-44c1-b064-455e97a3baab None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:32.644579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e5824-1d2d-44c1-b064-455e97a3baab None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:32.645568 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:32.646816 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 57/117] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:32 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:32.648470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9756d7-4e4f-4abd-941f-71a0818b0132 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:32.648756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9756d7-4e4f-4abd-941f-71a0818b0132 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:32.649034 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9756d7-4e4f-4abd-941f-71a0818b0132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:32.665500 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49f365-c549-4eab-a082-946a0b05edc9 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:32.683678 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-ef9756d7-4e4f-4abd-941f-71a0818b0132 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:32.685146 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-ef9756d7-4e4f-4abd-941f-71a0818b0132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:32.710692 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d54dacd-8b91-47b2-a973-a4b6c231a880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:32.711963 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54dacd-8b91-47b2-a973-a4b6c231a880 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:32.712249 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54dacd-8b91-47b2-a973-a4b6c231a880 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:32.712525 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54dacd-8b91-47b2-a973-a4b6c231a880 None admin] PATH_INFO: `/v3/users/f79add60659848fda751b9e18c150aed` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:32.727373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d54dacd-8b91-47b2-a973-a4b6c231a880 None admin] RBAC: Authorizing `identity:get_user(user_id=f79add60659848fda751b9e18c150aed)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:32.728937 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d54dacd-8b91-47b2-a973-a4b6c231a880 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:32.735443 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 61/118] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:32 2022] GET /identity/v3/users/f79add60659848fda751b9e18c150aed => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:32.735443 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d54dacd-8b91-47b2-a973-a4b6c231a880 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:32.772436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 58/119] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:32.779229 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9756d7-4e4f-4abd-941f-71a0818b0132 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:32.843962 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d54dacd-8b91-47b2-a973-a4b6c231a880 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:32.871948 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:32.882106 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801b1ab-70b9-42ed-9657-9d696dc05304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:32.883136 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801b1ab-70b9-42ed-9657-9d696dc05304 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:32.883429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801b1ab-70b9-42ed-9657-9d696dc05304 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:32.883721 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801b1ab-70b9-42ed-9657-9d696dc05304 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:32.893645 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9756d7-4e4f-4abd-941f-71a0818b0132 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:32.895099 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801b1ab-70b9-42ed-9657-9d696dc05304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-0801b1ab-70b9-42ed-9657-9d696dc05304 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:32.903273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:32.904478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:32.905509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:32.905509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:32.905509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:32.905509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:32.905509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:32.905509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:32.905509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:32.905509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:32.905509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:32.905509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:32.905509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:32.905509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:32.905509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 62/120] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:32.909998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801b1ab-70b9-42ed-9657-9d696dc05304 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:32.920300 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:32.932551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f781c-b926-4bcb-b6af-61dfb0d0cce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:32.933241 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f781c-b926-4bcb-b6af-61dfb0d0cce8 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:32.933532 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f781c-b926-4bcb-b6af-61dfb0d0cce8 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:32.933812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f781c-b926-4bcb-b6af-61dfb0d0cce8 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:32.934598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f781c-b926-4bcb-b6af-61dfb0d0cce8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:32.934911 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f781c-b926-4bcb-b6af-61dfb0d0cce8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:32.936326 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f781c-b926-4bcb-b6af-61dfb0d0cce8 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:32.944814 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 59/121] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:32.945429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f781c-b926-4bcb-b6af-61dfb0d0cce8 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:33.027108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801b1ab-70b9-42ed-9657-9d696dc05304 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:33.048690 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f781c-b926-4bcb-b6af-61dfb0d0cce8 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:33.063464 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c0253-ff66-4b50-ab42-217bdcc222d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:33.064397 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c0253-ff66-4b50-ab42-217bdcc222d7 None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:33.064683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c0253-ff66-4b50-ab42-217bdcc222d7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:33.064963 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c0253-ff66-4b50-ab42-217bdcc222d7 None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151/users/d5972d97f8c144e99fb7fb22399f0d94/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:33.067485 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c0253-ff66-4b50-ab42-217bdcc222d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba315bae61f943debd8460a5fa9dd151, user_id=d5972d97f8c144e99fb7fb22399f0d94, role_id=6d25238fbc0e4d11bac1bcdc7e50d2e2)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:33.069786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c0253-ff66-4b50-ab42-217bdcc222d7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:33.085933 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 63/122] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:17:32 2022] PUT /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151/users/d5972d97f8c144e99fb7fb22399f0d94/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:17:33.086558 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c0253-ff66-4b50-ab42-217bdcc222d7 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:33.100182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383af5d-040c-427e-8039-c1a53c59c67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:33.100876 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383af5d-040c-427e-8039-c1a53c59c67e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:33.101172 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383af5d-040c-427e-8039-c1a53c59c67e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:33.101554 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383af5d-040c-427e-8039-c1a53c59c67e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:33.129442 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:33.169096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:33.213116 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c0253-ff66-4b50-ab42-217bdcc222d7 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:33.215837 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383af5d-040c-427e-8039-c1a53c59c67e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:33.218531 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383af5d-040c-427e-8039-c1a53c59c67e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:33.252522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9631dd-51b6-437f-86dc-2584d49375c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:33.253581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9631dd-51b6-437f-86dc-2584d49375c0 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:33.253965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9631dd-51b6-437f-86dc-2584d49375c0 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:33.254372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9631dd-51b6-437f-86dc-2584d49375c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:33.266359 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9631dd-51b6-437f-86dc-2584d49375c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-fe9631dd-51b6-437f-86dc-2584d49375c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:33.275022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:33.276430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:33.277954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:33.277954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:33.277954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:33.277954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:33.277954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:33.277954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:33.277954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:33.277954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:33.277954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:33.277954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:33.277954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:33.277954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:33.279657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 64/123] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:33.285308 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9631dd-51b6-437f-86dc-2584d49375c0 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:33.364105 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 60/124] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:33.366666 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383af5d-040c-427e-8039-c1a53c59c67e None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:33.399238 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9631dd-51b6-437f-86dc-2584d49375c0 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:33.430874 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:33.431903 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:33.444041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0a484-eeaa-426b-8df7-87630089a991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:33.444809 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0a484-eeaa-426b-8df7-87630089a991 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:33.445388 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0a484-eeaa-426b-8df7-87630089a991 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:33.445740 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0a484-eeaa-426b-8df7-87630089a991 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:33.446997 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0a484-eeaa-426b-8df7-87630089a991 None admin] RBAC: Adding query filter params (name=member) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:33.447398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0a484-eeaa-426b-8df7-87630089a991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:33.449429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0a484-eeaa-426b-8df7-87630089a991 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:33.459718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 65/125] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:33.460894 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0a484-eeaa-426b-8df7-87630089a991 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:33.505937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383af5d-040c-427e-8039-c1a53c59c67e None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:33.555879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0b3f9-61b2-4bed-8b49-8d63a4907e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:33.556860 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0b3f9-61b2-4bed-8b49-8d63a4907e27 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:33.557309 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0b3f9-61b2-4bed-8b49-8d63a4907e27 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:33.557666 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0b3f9-61b2-4bed-8b49-8d63a4907e27 None admin] PATH_INFO: `/v3/users/470d21b48683452bbca4f78884ff1182` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:33.559533 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0b3f9-61b2-4bed-8b49-8d63a4907e27 None admin] RBAC: Authorizing `identity:get_user(user_id=470d21b48683452bbca4f78884ff1182)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:33.561346 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0b3f9-61b2-4bed-8b49-8d63a4907e27 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:33.568532 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 61/126] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:33 2022] GET /identity/v3/users/470d21b48683452bbca4f78884ff1182 => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:33.568532 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0b3f9-61b2-4bed-8b49-8d63a4907e27 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:33.576764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0a484-eeaa-426b-8df7-87630089a991 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:33.618980 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a01d8-7b18-47da-9bb9-4f991a80d7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:33.620319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a01d8-7b18-47da-9bb9-4f991a80d7ce None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:33.621151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a01d8-7b18-47da-9bb9-4f991a80d7ce None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:33.621650 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a01d8-7b18-47da-9bb9-4f991a80d7ce None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:33.622703 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a01d8-7b18-47da-9bb9-4f991a80d7ce None admin] RBAC: Adding query filter params (name=member) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:33.623256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a01d8-7b18-47da-9bb9-4f991a80d7ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:33.624903 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a01d8-7b18-47da-9bb9-4f991a80d7ce None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:33.634477 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 66/127] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:33.640556 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a01d8-7b18-47da-9bb9-4f991a80d7ce None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:33.696898 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0b3f9-61b2-4bed-8b49-8d63a4907e27 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:33.704433 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:33.713420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:33.740680 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90d3d2-7f95-4f38-b947-9200ba443db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:33.742009 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90d3d2-7f95-4f38-b947-9200ba443db3 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:33.742541 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90d3d2-7f95-4f38-b947-9200ba443db3 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:33.743033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90d3d2-7f95-4f38-b947-9200ba443db3 None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:33.744463 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90d3d2-7f95-4f38-b947-9200ba443db3 None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:33.746225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90d3d2-7f95-4f38-b947-9200ba443db3 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:33.748234 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 62/128] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:33 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:33.749238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90d3d2-7f95-4f38-b947-9200ba443db3 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:33.773549 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a01d8-7b18-47da-9bb9-4f991a80d7ce None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:33.818463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb87d6-6244-401d-893c-e732c122ec29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:33.819279 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb87d6-6244-401d-893c-e732c122ec29 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:33.819578 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb87d6-6244-401d-893c-e732c122ec29 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:33.819997 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb87d6-6244-401d-893c-e732c122ec29 None admin] PATH_INFO: `/v3/users/d5972d97f8c144e99fb7fb22399f0d94` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:33.821723 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb87d6-6244-401d-893c-e732c122ec29 None admin] RBAC: Authorizing `identity:get_user(user_id=d5972d97f8c144e99fb7fb22399f0d94)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:33.823644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb87d6-6244-401d-893c-e732c122ec29 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:33.826681 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 67/129] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:33 2022] GET /identity/v3/users/d5972d97f8c144e99fb7fb22399f0d94 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:33.827627 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb87d6-6244-401d-893c-e732c122ec29 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:33.858151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90d3d2-7f95-4f38-b947-9200ba443db3 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:33.894015 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab189064-bbf7-4d44-a72b-c60bfbfc26a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:33.895715 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab189064-bbf7-4d44-a72b-c60bfbfc26a8 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:33.896202 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab189064-bbf7-4d44-a72b-c60bfbfc26a8 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:33.896710 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab189064-bbf7-4d44-a72b-c60bfbfc26a8 None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:33.898200 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab189064-bbf7-4d44-a72b-c60bfbfc26a8 None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:33.899912 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab189064-bbf7-4d44-a72b-c60bfbfc26a8 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:33.907132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 63/130] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:33 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:33.908314 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab189064-bbf7-4d44-a72b-c60bfbfc26a8 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:33.959177 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb87d6-6244-401d-893c-e732c122ec29 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:33.966649 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:33.985275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:34.008717 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195173ae-f238-4b94-b4f6-e46cd749d0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:34.010667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195173ae-f238-4b94-b4f6-e46cd749d0d7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:34.011192 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195173ae-f238-4b94-b4f6-e46cd749d0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:34.012119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195173ae-f238-4b94-b4f6-e46cd749d0d7 None admin] PATH_INFO: `/v3/users/d5972d97f8c144e99fb7fb22399f0d94` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:34.014042 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195173ae-f238-4b94-b4f6-e46cd749d0d7 None admin] RBAC: Authorizing `identity:get_user(user_id=d5972d97f8c144e99fb7fb22399f0d94)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:34.016732 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195173ae-f238-4b94-b4f6-e46cd749d0d7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:34.019417 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 68/131] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:33 2022] GET /identity/v3/users/d5972d97f8c144e99fb7fb22399f0d94 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:34.020760 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195173ae-f238-4b94-b4f6-e46cd749d0d7 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:34.026767 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab189064-bbf7-4d44-a72b-c60bfbfc26a8 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:34.065328 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9217d8b-667c-4e0c-b3a5-076893feeb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:34.066108 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9217d8b-667c-4e0c-b3a5-076893feeb58 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:34.066521 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9217d8b-667c-4e0c-b3a5-076893feeb58 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:34.066862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9217d8b-667c-4e0c-b3a5-076893feeb58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:34.067806 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9217d8b-667c-4e0c-b3a5-076893feeb58 None admin] RBAC: Adding query filter params (role.id=086ec02a15114cc3b858b231d55e19aa, scope.project.id=ba315bae61f943debd8460a5fa9dd151, user.id=f79add60659848fda751b9e18c150aed) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:34.068185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9217d8b-667c-4e0c-b3a5-076893feeb58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:34.070327 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9217d8b-667c-4e0c-b3a5-076893feeb58 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:34.076132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 64/132] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:33 2022] GET /identity/v3/role_assignments?user.id=f79add60659848fda751b9e18c150aed&scope.project.id=ba315bae61f943debd8460a5fa9dd151&role.id=086ec02a15114cc3b858b231d55e19aa => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:34.076792 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9217d8b-667c-4e0c-b3a5-076893feeb58 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:34.159327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195173ae-f238-4b94-b4f6-e46cd749d0d7 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:34.191660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9217d8b-667c-4e0c-b3a5-076893feeb58 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:34.204942 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97895409-6927-4213-89d4-4c91fe05b1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:34.205965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97895409-6927-4213-89d4-4c91fe05b1aa None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:34.206346 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97895409-6927-4213-89d4-4c91fe05b1aa None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:34.206743 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97895409-6927-4213-89d4-4c91fe05b1aa None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:34.208148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97895409-6927-4213-89d4-4c91fe05b1aa None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:34.209673 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97895409-6927-4213-89d4-4c91fe05b1aa None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:34.211897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 69/133] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:34 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:34.212549 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97895409-6927-4213-89d4-4c91fe05b1aa None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:34.213490 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:34.240575 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab006052-c020-4489-8389-9f6a0a240481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:34.241786 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab006052-c020-4489-8389-9f6a0a240481 None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:34.242297 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab006052-c020-4489-8389-9f6a0a240481 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:34.242664 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab006052-c020-4489-8389-9f6a0a240481 None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151/users/470d21b48683452bbca4f78884ff1182/roles/b239f941cb724331bf28dc489b70ad1d` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:34.245011 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab006052-c020-4489-8389-9f6a0a240481 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba315bae61f943debd8460a5fa9dd151, user_id=470d21b48683452bbca4f78884ff1182, role_id=b239f941cb724331bf28dc489b70ad1d)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:34.246882 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab006052-c020-4489-8389-9f6a0a240481 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:34.267531 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 65/134] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:17:34 2022] PUT /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151/users/470d21b48683452bbca4f78884ff1182/roles/b239f941cb724331bf28dc489b70ad1d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:17:34.269219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb0a59-e75a-4fed-83ee-f4c4e4e9c2cb None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:34.269584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb0a59-e75a-4fed-83ee-f4c4e4e9c2cb None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:34.269937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb0a59-e75a-4fed-83ee-f4c4e4e9c2cb None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:34.272260 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 66/135] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:34.277157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb0a59-e75a-4fed-83ee-f4c4e4e9c2cb None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:34.304768 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:34.406760 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97895409-6927-4213-89d4-4c91fe05b1aa None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:34.469257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb0a59-e75a-4fed-83ee-f4c4e4e9c2cb None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:34.489079 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9088f2-295b-4747-8f0c-c4bf43d78e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:34.489986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9088f2-295b-4747-8f0c-c4bf43d78e8b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:34.490288 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9088f2-295b-4747-8f0c-c4bf43d78e8b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:34.490595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9088f2-295b-4747-8f0c-c4bf43d78e8b None admin] PATH_INFO: `/v3/projects/3ae9b8c5024e40f0b79dcf1d40c57424` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:34.492017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9088f2-295b-4747-8f0c-c4bf43d78e8b None admin] RBAC: Authorizing `identity:get_project(project_id=3ae9b8c5024e40f0b79dcf1d40c57424)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:34.493532 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9088f2-295b-4747-8f0c-c4bf43d78e8b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:34.495713 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 70/136] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:34 2022] GET /identity/v3/projects/3ae9b8c5024e40f0b79dcf1d40c57424 => generated 312 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:34.496984 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23d9b5-ac17-435f-925c-4acf5d05bf72 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:34.497278 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23d9b5-ac17-435f-925c-4acf5d05bf72 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:34.497561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23d9b5-ac17-435f-925c-4acf5d05bf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:34.521792 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081acc3-bd4d-4c1a-acda-c42f7afca49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:34.523230 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081acc3-bd4d-4c1a-acda-c42f7afca49f None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:34.523831 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081acc3-bd4d-4c1a-acda-c42f7afca49f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:34.524345 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081acc3-bd4d-4c1a-acda-c42f7afca49f None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151/users/d5972d97f8c144e99fb7fb22399f0d94/roles/666cfd1181184612ba462b8eefcad6a7` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:34.526794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081acc3-bd4d-4c1a-acda-c42f7afca49f None admin] RBAC: Authorizing `identity:create_grant(project_id=ba315bae61f943debd8460a5fa9dd151, user_id=d5972d97f8c144e99fb7fb22399f0d94, role_id=666cfd1181184612ba462b8eefcad6a7)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:34.527968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-df23d9b5-ac17-435f-925c-4acf5d05bf72 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:34.528924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081acc3-bd4d-4c1a-acda-c42f7afca49f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:34.529197 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-df23d9b5-ac17-435f-925c-4acf5d05bf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:34.552494 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 67/137] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:17:34 2022] PUT /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151/users/d5972d97f8c144e99fb7fb22399f0d94/roles/666cfd1181184612ba462b8eefcad6a7 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:17:34.554102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081acc3-bd4d-4c1a-acda-c42f7afca49f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:34.595889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:34.626335 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 71/138] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:34.655068 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23d9b5-ac17-435f-925c-4acf5d05bf72 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:34.681646 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081acc3-bd4d-4c1a-acda-c42f7afca49f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:34.718312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:34.729165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ee658-58b2-4f27-a17f-8fca20b97e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:34.731202 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ee658-58b2-4f27-a17f-8fca20b97e84 None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:34.731656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ee658-58b2-4f27-a17f-8fca20b97e84 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:34.732133 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ee658-58b2-4f27-a17f-8fca20b97e84 None admin] PATH_INFO: `/v3/projects/3ae9b8c5024e40f0b79dcf1d40c57424/users/d5972d97f8c144e99fb7fb22399f0d94/roles/666cfd1181184612ba462b8eefcad6a7` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:34.735031 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ee658-58b2-4f27-a17f-8fca20b97e84 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ae9b8c5024e40f0b79dcf1d40c57424, user_id=d5972d97f8c144e99fb7fb22399f0d94, role_id=666cfd1181184612ba462b8eefcad6a7)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:34.736832 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ee658-58b2-4f27-a17f-8fca20b97e84 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:34.770588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 68/139] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:17:34 2022] PUT /identity/v3/projects/3ae9b8c5024e40f0b79dcf1d40c57424/users/d5972d97f8c144e99fb7fb22399f0d94/roles/666cfd1181184612ba462b8eefcad6a7 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:17:34.813658 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23d9b5-ac17-435f-925c-4acf5d05bf72 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:34.865230 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79133b71-ad7d-4d4b-9e44-8cc6d04bd96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:34.866062 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79133b71-ad7d-4d4b-9e44-8cc6d04bd96f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:34.866390 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79133b71-ad7d-4d4b-9e44-8cc6d04bd96f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:34.866672 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79133b71-ad7d-4d4b-9e44-8cc6d04bd96f None admin] PATH_INFO: `/v3/domains/default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:34.868011 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79133b71-ad7d-4d4b-9e44-8cc6d04bd96f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:34.869767 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79133b71-ad7d-4d4b-9e44-8cc6d04bd96f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:34.872028 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 72/140] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:34.877849 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ee658-58b2-4f27-a17f-8fca20b97e84 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:35.010426 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:35.019611 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ee658-58b2-4f27-a17f-8fca20b97e84 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:35.083928 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1b9b3-bf96-4750-86a3-b7c2f938bf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:35.084631 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1b9b3-bf96-4750-86a3-b7c2f938bf3e None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:35.084919 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1b9b3-bf96-4750-86a3-b7c2f938bf3e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:35.085197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1b9b3-bf96-4750-86a3-b7c2f938bf3e None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:35.085937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1b9b3-bf96-4750-86a3-b7c2f938bf3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:35.087629 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1b9b3-bf96-4750-86a3-b7c2f938bf3e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:35.117375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 69/141] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:17:34 2022] POST /identity/v3/users => generated 283 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:17:35.932518 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02632fc3-9d62-4678-8b9c-de3053394527 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:35.933232 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02632fc3-9d62-4678-8b9c-de3053394527 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:35.933232 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02632fc3-9d62-4678-8b9c-de3053394527 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:35.934759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 73/142] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:35.940214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414796f-2309-49a6-ae66-03657e52dead None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:35.940511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414796f-2309-49a6-ae66-03657e52dead None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:35.940799 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414796f-2309-49a6-ae66-03657e52dead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:35.960780 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-b414796f-2309-49a6-ae66-03657e52dead None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:35.961987 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-b414796f-2309-49a6-ae66-03657e52dead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:36.045299 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 70/143] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:36.073404 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02632fc3-9d62-4678-8b9c-de3053394527 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:36.180839 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02632fc3-9d62-4678-8b9c-de3053394527 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:36.203737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:36.231360 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597cc74-ae69-4409-abcc-284dd0b50ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:36.232190 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597cc74-ae69-4409-abcc-284dd0b50ddf None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:36.232471 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597cc74-ae69-4409-abcc-284dd0b50ddf None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:36.232745 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597cc74-ae69-4409-abcc-284dd0b50ddf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:36.243624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597cc74-ae69-4409-abcc-284dd0b50ddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-c597cc74-ae69-4409-abcc-284dd0b50ddf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:36.258720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:36.259866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:36.261835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:36.261835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:36.261835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:36.261835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:36.261835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:36.261835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:36.261835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:36.261835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:36.261835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:36.261835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:36.261835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:17:36.261835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:36.261835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 74/144] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:17:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:36.266876 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414796f-2309-49a6-ae66-03657e52dead None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:36.370377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:36.402789 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414796f-2309-49a6-ae66-03657e52dead None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:36.453467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6f265-e119-4194-9f49-f4de997f96f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:36.454222 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6f265-e119-4194-9f49-f4de997f96f9 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:36.454478 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6f265-e119-4194-9f49-f4de997f96f9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:36.454763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6f265-e119-4194-9f49-f4de997f96f9 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:36.455529 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6f265-e119-4194-9f49-f4de997f96f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:36.455840 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6f265-e119-4194-9f49-f4de997f96f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:36.457312 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6f265-e119-4194-9f49-f4de997f96f9 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:36.466925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 71/145] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:17:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:36.475370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597cc74-ae69-4409-abcc-284dd0b50ddf None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:36.607498 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597cc74-ae69-4409-abcc-284dd0b50ddf None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:36.645779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2f709-d757-41dc-9a59-365c61c39aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:36.646686 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2f709-d757-41dc-9a59-365c61c39aab None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:36.646998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2f709-d757-41dc-9a59-365c61c39aab None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:36.647240 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2f709-d757-41dc-9a59-365c61c39aab None admin] PATH_INFO: `/v3/users/d5972d97f8c144e99fb7fb22399f0d94` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:36.648436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2f709-d757-41dc-9a59-365c61c39aab None admin] RBAC: Authorizing `identity:get_user(user_id=d5972d97f8c144e99fb7fb22399f0d94)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:36.649970 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2f709-d757-41dc-9a59-365c61c39aab None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:36.651657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 75/146] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:36 2022] GET /identity/v3/users/d5972d97f8c144e99fb7fb22399f0d94 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:36.657374 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6f265-e119-4194-9f49-f4de997f96f9 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:36.811299 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6f265-e119-4194-9f49-f4de997f96f9 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:36.848873 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3390160-4265-48f3-9aca-6412778e898d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:36.849538 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3390160-4265-48f3-9aca-6412778e898d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:36.849820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3390160-4265-48f3-9aca-6412778e898d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:36.850165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3390160-4265-48f3-9aca-6412778e898d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:36.864380 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:36.932060 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3390160-4265-48f3-9aca-6412778e898d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:36.933741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3390160-4265-48f3-9aca-6412778e898d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:37.059339 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 72/147] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:37.064844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2f709-d757-41dc-9a59-365c61c39aab None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:37.117496 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:37.176587 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2f709-d757-41dc-9a59-365c61c39aab None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:37.216601 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a6b11-bdaf-48a4-bb8d-25466c8cce6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:37.217476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a6b11-bdaf-48a4-bb8d-25466c8cce6e None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:37.217755 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a6b11-bdaf-48a4-bb8d-25466c8cce6e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:37.218028 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a6b11-bdaf-48a4-bb8d-25466c8cce6e None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:37.219286 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a6b11-bdaf-48a4-bb8d-25466c8cce6e None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:37.220781 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a6b11-bdaf-48a4-bb8d-25466c8cce6e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:37.222530 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 76/148] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:37 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:37.228445 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3390160-4265-48f3-9aca-6412778e898d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:37.284447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:37.359081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3390160-4265-48f3-9aca-6412778e898d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:37.412892 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a2b75-b2de-4f8f-9e35-065ecef9884a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:37.413982 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a2b75-b2de-4f8f-9e35-065ecef9884a None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:37.414504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a2b75-b2de-4f8f-9e35-065ecef9884a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:37.415031 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a2b75-b2de-4f8f-9e35-065ecef9884a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:37.421523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a2b75-b2de-4f8f-9e35-065ecef9884a None admin] RBAC: Adding query filter params (role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2, scope.project.id=ba315bae61f943debd8460a5fa9dd151, user.id=d5972d97f8c144e99fb7fb22399f0d94) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:37.422212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a2b75-b2de-4f8f-9e35-065ecef9884a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:37.423963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a2b75-b2de-4f8f-9e35-065ecef9884a None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:37.435693 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 73/149] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:37 2022] GET /identity/v3/role_assignments?user.id=d5972d97f8c144e99fb7fb22399f0d94&scope.project.id=ba315bae61f943debd8460a5fa9dd151&role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:37.872810 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e517d23-2b85-4e2d-bcff-8a228c72122c None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:37.873082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e517d23-2b85-4e2d-bcff-8a228c72122c None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:37.873354 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e517d23-2b85-4e2d-bcff-8a228c72122c None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:37.875031 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 77/150] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:37.888581 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dd610-ea32-4c87-92cb-1c376d3419ec None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:37.888891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dd610-ea32-4c87-92cb-1c376d3419ec None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:37.889140 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dd610-ea32-4c87-92cb-1c376d3419ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:37.910396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-715dd610-ea32-4c87-92cb-1c376d3419ec None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:37.911580 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-715dd610-ea32-4c87-92cb-1c376d3419ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:37.995081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 74/151] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:38.033825 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e517d23-2b85-4e2d-bcff-8a228c72122c None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:38.105592 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d37f9-45b2-41e3-8618-219ed24dc1d3 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:38.106628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d37f9-45b2-41e3-8618-219ed24dc1d3 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:38.107144 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d37f9-45b2-41e3-8618-219ed24dc1d3 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:38.109965 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 75/152] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:38.114418 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94cef0-955f-4cc1-9df1-bbfa2b0f695f None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:38.115065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94cef0-955f-4cc1-9df1-bbfa2b0f695f None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:38.115559 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94cef0-955f-4cc1-9df1-bbfa2b0f695f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:38.141217 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-fe94cef0-955f-4cc1-9df1-bbfa2b0f695f None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:38.146831 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-fe94cef0-955f-4cc1-9df1-bbfa2b0f695f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:38.187428 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e517d23-2b85-4e2d-bcff-8a228c72122c None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:38.200632 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:38.231077 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:38.255810 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6a7b8-66a3-41d8-bb03-c49b961d74fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:38.256629 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6a7b8-66a3-41d8-bb03-c49b961d74fe None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:38.256915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6a7b8-66a3-41d8-bb03-c49b961d74fe None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:38.257196 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6a7b8-66a3-41d8-bb03-c49b961d74fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:38.286738 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6a7b8-66a3-41d8-bb03-c49b961d74fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:38.292626 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 76/153] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-95b6a7b8-66a3-41d8-bb03-c49b961d74fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:38.300271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:38.301395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:38.302343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:38.302343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:38.302343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:38.302343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:38.302343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:38.302343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:38.302343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:38.302343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:38.302343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:38.302343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:38.302343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:17:38.302343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:38.302343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 78/154] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:38.310108 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94cef0-955f-4cc1-9df1-bbfa2b0f695f None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:38.332160 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6a7b8-66a3-41d8-bb03-c49b961d74fe None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:38.499518 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94cef0-955f-4cc1-9df1-bbfa2b0f695f None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:38.500081 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6a7b8-66a3-41d8-bb03-c49b961d74fe None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:38.539763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d75c87-531a-47be-993f-211e25720431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:38.540812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d75c87-531a-47be-993f-211e25720431 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:38.541396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d75c87-531a-47be-993f-211e25720431 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:38.542041 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d75c87-531a-47be-993f-211e25720431 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:38.543144 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d75c87-531a-47be-993f-211e25720431 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:38.543725 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d75c87-531a-47be-993f-211e25720431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:38.545650 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d75c87-531a-47be-993f-211e25720431 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:38.556304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 77/155] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:38.561272 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be80361-faaa-4cc5-b0a1-9ad5a59260c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:38.562208 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be80361-faaa-4cc5-b0a1-9ad5a59260c4 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:38.562541 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be80361-faaa-4cc5-b0a1-9ad5a59260c4 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:38.562821 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be80361-faaa-4cc5-b0a1-9ad5a59260c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:38.563824 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d75c87-531a-47be-993f-211e25720431 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:38.589797 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be80361-faaa-4cc5-b0a1-9ad5a59260c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:38.596255 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-7be80361-faaa-4cc5-b0a1-9ad5a59260c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:38.597345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:38.598429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:38.599356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:38.599356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:38.599356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:38.599356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:38.599356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:38.599356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:38.599356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:38.599356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:38.599356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:38.599356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:38.599356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:38.599356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:38.607157 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 79/156] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:17:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:38.612598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be80361-faaa-4cc5-b0a1-9ad5a59260c4 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:38.679287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:38.685852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d75c87-531a-47be-993f-211e25720431 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:38.722723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99583f47-a34d-4be6-918b-6ca43bd47f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:38.723981 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99583f47-a34d-4be6-918b-6ca43bd47f51 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:38.724471 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99583f47-a34d-4be6-918b-6ca43bd47f51 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:38.724961 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99583f47-a34d-4be6-918b-6ca43bd47f51 None admin] PATH_INFO: `/v3/users/f79add60659848fda751b9e18c150aed` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:38.726445 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99583f47-a34d-4be6-918b-6ca43bd47f51 None admin] RBAC: Authorizing `identity:get_user(user_id=f79add60659848fda751b9e18c150aed)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:38.728146 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99583f47-a34d-4be6-918b-6ca43bd47f51 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:38.730213 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 78/157] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:38 2022] GET /identity/v3/users/f79add60659848fda751b9e18c150aed => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:17:38.735943 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99583f47-a34d-4be6-918b-6ca43bd47f51 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:38.770876 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be80361-faaa-4cc5-b0a1-9ad5a59260c4 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:38.807884 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7696b-9dae-47ad-89cc-8edb417bac23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:38.808571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7696b-9dae-47ad-89cc-8edb417bac23 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:38.808857 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7696b-9dae-47ad-89cc-8edb417bac23 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:38.809139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7696b-9dae-47ad-89cc-8edb417bac23 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:38.809968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda7696b-9dae-47ad-89cc-8edb417bac23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:38.810301 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda7696b-9dae-47ad-89cc-8edb417bac23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:38.811964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda7696b-9dae-47ad-89cc-8edb417bac23 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:38.820788 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 80/158] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:38.827248 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7696b-9dae-47ad-89cc-8edb417bac23 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:38.896304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:38.912348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99583f47-a34d-4be6-918b-6ca43bd47f51 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:38.951765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7b200-d04d-4ac6-aa42-8e4c8d8f87b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:38.952565 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7b200-d04d-4ac6-aa42-8e4c8d8f87b9 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:38.952850 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7b200-d04d-4ac6-aa42-8e4c8d8f87b9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:38.953134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7b200-d04d-4ac6-aa42-8e4c8d8f87b9 None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:38.953380 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:38.954632 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7b200-d04d-4ac6-aa42-8e4c8d8f87b9 None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:38.955758 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7696b-9dae-47ad-89cc-8edb417bac23 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:38.955878 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7b200-d04d-4ac6-aa42-8e4c8d8f87b9 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:38.957863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 79/159] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:38 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:17:38.959104 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3bb58-557f-4e44-8fb4-f45cd964e67c None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:38.959393 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3bb58-557f-4e44-8fb4-f45cd964e67c None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:38.959670 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3bb58-557f-4e44-8fb4-f45cd964e67c None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:38.961131 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 80/160] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:38.962071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e118fb3-29d6-4dd3-892c-e33aecffc15a None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:38.962388 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e118fb3-29d6-4dd3-892c-e33aecffc15a None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:38.962670 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e118fb3-29d6-4dd3-892c-e33aecffc15a None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:38.964621 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 81/161] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:38.970411 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01124425-b327-454d-8361-90c7b2250c8f None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:38.970706 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01124425-b327-454d-8361-90c7b2250c8f None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:38.970990 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01124425-b327-454d-8361-90c7b2250c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:38.988026 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-01124425-b327-454d-8361-90c7b2250c8f None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:38.989452 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-01124425-b327-454d-8361-90c7b2250c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:38.992370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bb6c2-9430-4e2f-ae1b-b44941543687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:38.993266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bb6c2-9430-4e2f-ae1b-b44941543687 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:38.993551 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bb6c2-9430-4e2f-ae1b-b44941543687 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:38.993832 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bb6c2-9430-4e2f-ae1b-b44941543687 None admin] PATH_INFO: `/v3/users/470d21b48683452bbca4f78884ff1182` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:38.995800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591bb6c2-9430-4e2f-ae1b-b44941543687 None admin] RBAC: Authorizing `identity:get_user(user_id=470d21b48683452bbca4f78884ff1182)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:38.997276 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591bb6c2-9430-4e2f-ae1b-b44941543687 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:38.998979 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 81/162] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:38 2022] GET /identity/v3/users/470d21b48683452bbca4f78884ff1182 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:39.000168 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca534d6-68fd-4c3d-8739-c987bc6cb152 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:39.000454 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca534d6-68fd-4c3d-8739-c987bc6cb152 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:39.000736 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca534d6-68fd-4c3d-8739-c987bc6cb152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:39.032864 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-fca534d6-68fd-4c3d-8739-c987bc6cb152 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:39.036867 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-fca534d6-68fd-4c3d-8739-c987bc6cb152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:39.127214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 82/163] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:17:39.128098 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01124425-b327-454d-8361-90c7b2250c8f None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:39.183168 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 82/164] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:39.184334 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca534d6-68fd-4c3d-8739-c987bc6cb152 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:39.307042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:39.327275 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01124425-b327-454d-8361-90c7b2250c8f None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:39.365950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:39.375569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca534d6-68fd-4c3d-8739-c987bc6cb152 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:39.394935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e53f3-c4d6-4fa6-8b2c-df64d8283ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:39.395875 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e53f3-c4d6-4fa6-8b2c-df64d8283ae3 None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:39.396159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e53f3-c4d6-4fa6-8b2c-df64d8283ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:39.396438 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e53f3-c4d6-4fa6-8b2c-df64d8283ae3 None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151/users/f79add60659848fda751b9e18c150aed/roles/086ec02a15114cc3b858b231d55e19aa` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:39.403684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e53f3-c4d6-4fa6-8b2c-df64d8283ae3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba315bae61f943debd8460a5fa9dd151, user_id=f79add60659848fda751b9e18c150aed, role_id=086ec02a15114cc3b858b231d55e19aa)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:39.405177 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e53f3-c4d6-4fa6-8b2c-df64d8283ae3 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:39.424222 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 83/165] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:17:39 2022] PUT /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151/users/f79add60659848fda751b9e18c150aed/roles/086ec02a15114cc3b858b231d55e19aa => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:17:39.425286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e53f3-c4d6-4fa6-8b2c-df64d8283ae3 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:39.426383 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbce54-f1be-4cde-b01a-c05348abca31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:39.427102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbce54-f1be-4cde-b01a-c05348abca31 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:39.427391 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbce54-f1be-4cde-b01a-c05348abca31 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:39.427669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbce54-f1be-4cde-b01a-c05348abca31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:39.566153 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbce54-f1be-4cde-b01a-c05348abca31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:39.567952 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbce54-f1be-4cde-b01a-c05348abca31 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:39.658507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e53f3-c4d6-4fa6-8b2c-df64d8283ae3 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:39.747113 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa71a7-32fb-4448-9231-47abe131aaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:39.747980 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa71a7-32fb-4448-9231-47abe131aaf6 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:39.748266 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa71a7-32fb-4448-9231-47abe131aaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:39.748545 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa71a7-32fb-4448-9231-47abe131aaf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:39.779579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa71a7-32fb-4448-9231-47abe131aaf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:39.783583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-e5fa71a7-32fb-4448-9231-47abe131aaf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:39.787845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:39.788984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:39.789967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:39.789967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:39.789967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:39.789967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:39.789967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:39.789967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:39.789967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:39.789967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:39.789967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:39.789967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:39.789967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:39.789967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:39.789967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 84/166] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:39.792337 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa71a7-32fb-4448-9231-47abe131aaf6 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:39.802764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 83/167] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:39.804219 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbce54-f1be-4cde-b01a-c05348abca31 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:39.829474 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:39.929593 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa71a7-32fb-4448-9231-47abe131aaf6 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:39.970199 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60128cfd-56e2-46b5-9423-7ce303bfaba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:39.971042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60128cfd-56e2-46b5-9423-7ce303bfaba8 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:39.971324 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60128cfd-56e2-46b5-9423-7ce303bfaba8 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:39.971602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60128cfd-56e2-46b5-9423-7ce303bfaba8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:39.990042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60128cfd-56e2-46b5-9423-7ce303bfaba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:40.002100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbce54-f1be-4cde-b01a-c05348abca31 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-60128cfd-56e2-46b5-9423-7ce303bfaba8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:40.005065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:40.006175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:40.007151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:40.007151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:40.007151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:40.007151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:40.007151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:40.007151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:40.007151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:40.007151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:40.007151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:40.007151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:40.007151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:40.007151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:40.013214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 85/168] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:40.014099 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60128cfd-56e2-46b5-9423-7ce303bfaba8 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:40.064181 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7c5d7-e7a3-4143-adaa-16515904929a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:40.064869 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7c5d7-e7a3-4143-adaa-16515904929a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:40.065149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7c5d7-e7a3-4143-adaa-16515904929a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:40.065424 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7c5d7-e7a3-4143-adaa-16515904929a None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:40.066188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7c5d7-e7a3-4143-adaa-16515904929a None admin] RBAC: Adding query filter params (name=member) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:40.066525 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7c5d7-e7a3-4143-adaa-16515904929a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:40.067984 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7c5d7-e7a3-4143-adaa-16515904929a None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:40.092415 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 84/169] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:40.093307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7c5d7-e7a3-4143-adaa-16515904929a None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:40.146045 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:40.171860 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60128cfd-56e2-46b5-9423-7ce303bfaba8 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:40.206848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ce089-ddd3-4270-ac3f-36a5cc9de4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:40.207745 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ce089-ddd3-4270-ac3f-36a5cc9de4f5 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:40.208028 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ce089-ddd3-4270-ac3f-36a5cc9de4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:40.208307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ce089-ddd3-4270-ac3f-36a5cc9de4f5 None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:40.209555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ce089-ddd3-4270-ac3f-36a5cc9de4f5 None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:40.210096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:40.212197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ce089-ddd3-4270-ac3f-36a5cc9de4f5 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:40.212871 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 86/170] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:40 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:40.214545 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0de44-9a1b-4b82-bb56-3f13a77cb956 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:40.214851 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0de44-9a1b-4b82-bb56-3f13a77cb956 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:40.215133 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0de44-9a1b-4b82-bb56-3f13a77cb956 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:40.216655 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 87/171] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:40.217331 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0de44-9a1b-4b82-bb56-3f13a77cb956 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:40.283437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7c5d7-e7a3-4143-adaa-16515904929a None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:40.334960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20644a-1609-4845-b529-d62de50ade4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:40.335668 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20644a-1609-4845-b529-d62de50ade4a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:40.336128 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20644a-1609-4845-b529-d62de50ade4a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:40.336416 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20644a-1609-4845-b529-d62de50ade4a None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:40.337352 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20644a-1609-4845-b529-d62de50ade4a None admin] RBAC: Adding query filter params (name=member) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:40.337672 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20644a-1609-4845-b529-d62de50ade4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:40.339945 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20644a-1609-4845-b529-d62de50ade4a None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:40.344567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0de44-9a1b-4b82-bb56-3f13a77cb956 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:40.350406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 85/172] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:40.351640 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053ce0a-b402-4e8e-939d-cc75141b4b22 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:40.351939 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053ce0a-b402-4e8e-939d-cc75141b4b22 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:40.352288 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053ce0a-b402-4e8e-939d-cc75141b4b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:40.371452 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-3053ce0a-b402-4e8e-939d-cc75141b4b22 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:40.372844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-3053ce0a-b402-4e8e-939d-cc75141b4b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:40.394037 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b99b3-b644-4081-962a-455c32c9da80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:40.394967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b99b3-b644-4081-962a-455c32c9da80 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:40.395248 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b99b3-b644-4081-962a-455c32c9da80 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:40.395524 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b99b3-b644-4081-962a-455c32c9da80 None admin] PATH_INFO: `/v3/users/d5972d97f8c144e99fb7fb22399f0d94` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:40.398182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b99b3-b644-4081-962a-455c32c9da80 None admin] RBAC: Authorizing `identity:get_user(user_id=d5972d97f8c144e99fb7fb22399f0d94)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:40.399672 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b99b3-b644-4081-962a-455c32c9da80 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:40.402938 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 88/173] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:40 2022] GET /identity/v3/users/d5972d97f8c144e99fb7fb22399f0d94 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:40.403751 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b99b3-b644-4081-962a-455c32c9da80 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:40.425009 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:40.470919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 86/174] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:40.471865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053ce0a-b402-4e8e-939d-cc75141b4b22 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:40.525498 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:40.538805 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b99b3-b644-4081-962a-455c32c9da80 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:40.577534 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa61ea-bdb2-4c7b-95b0-b4d67fdd152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:40.577810 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053ce0a-b402-4e8e-939d-cc75141b4b22 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:40.578340 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa61ea-bdb2-4c7b-95b0-b4d67fdd152f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:40.578704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa61ea-bdb2-4c7b-95b0-b4d67fdd152f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:40.579061 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa61ea-bdb2-4c7b-95b0-b4d67fdd152f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:40.579921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa61ea-bdb2-4c7b-95b0-b4d67fdd152f None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=ba315bae61f943debd8460a5fa9dd151, user.id=470d21b48683452bbca4f78884ff1182) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:40.580828 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa61ea-bdb2-4c7b-95b0-b4d67fdd152f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:40.581739 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa61ea-bdb2-4c7b-95b0-b4d67fdd152f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:40.588071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 89/175] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:40 2022] GET /identity/v3/role_assignments?user.id=470d21b48683452bbca4f78884ff1182&scope.project.id=ba315bae61f943debd8460a5fa9dd151&role.id=b239f941cb724331bf28dc489b70ad1d => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:40.590052 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa61ea-bdb2-4c7b-95b0-b4d67fdd152f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:40.629623 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e3d69-efba-4bdb-bfa1-65b129e7d7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:40.631359 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e3d69-efba-4bdb-bfa1-65b129e7d7a3 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:40.632138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e3d69-efba-4bdb-bfa1-65b129e7d7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:40.632619 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e3d69-efba-4bdb-bfa1-65b129e7d7a3 None admin] PATH_INFO: `/v3/users/d5972d97f8c144e99fb7fb22399f0d94` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:40.634496 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e3d69-efba-4bdb-bfa1-65b129e7d7a3 None admin] RBAC: Authorizing `identity:get_user(user_id=d5972d97f8c144e99fb7fb22399f0d94)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:40.637342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e3d69-efba-4bdb-bfa1-65b129e7d7a3 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:40.639422 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 87/176] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:40 2022] GET /identity/v3/users/d5972d97f8c144e99fb7fb22399f0d94 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:40.641117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73baa0f-2723-4741-89e2-b267a2e52e54 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:40.641618 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73baa0f-2723-4741-89e2-b267a2e52e54 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:40.642117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73baa0f-2723-4741-89e2-b267a2e52e54 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:40.644419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 88/177] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:40.645974 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73baa0f-2723-4741-89e2-b267a2e52e54 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:40.682608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:40.723446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa61ea-bdb2-4c7b-95b0-b4d67fdd152f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:40.773771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62443c39-ab1d-4ff5-9164-4369a2edbc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:40.775018 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62443c39-ab1d-4ff5-9164-4369a2edbc90 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:40.775575 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62443c39-ab1d-4ff5-9164-4369a2edbc90 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:40.776012 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62443c39-ab1d-4ff5-9164-4369a2edbc90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:40.784278 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73baa0f-2723-4741-89e2-b267a2e52e54 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:40.808600 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:40.836010 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f68fd-5aa2-4ff6-9ee2-38c015a678b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:40.837171 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f68fd-5aa2-4ff6-9ee2-38c015a678b9 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:40.837171 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f68fd-5aa2-4ff6-9ee2-38c015a678b9 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:40.837359 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f68fd-5aa2-4ff6-9ee2-38c015a678b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:40.854436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f68fd-5aa2-4ff6-9ee2-38c015a678b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-f02f68fd-5aa2-4ff6-9ee2-38c015a678b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:40.862748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:40.863802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:40.864782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:40.864782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:40.864782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:40.864782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:40.864782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:40.864782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:40.864782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:40.864782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:40.864782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:40.864782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:40.864782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:40.864782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:40.869900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62443c39-ab1d-4ff5-9164-4369a2edbc90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:40.870083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 89/178] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:17:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:40.870179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62443c39-ab1d-4ff5-9164-4369a2edbc90 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:40.870288 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca64660-c7ca-4d72-bd32-1acbbd3f4ed7 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:40.870288 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca64660-c7ca-4d72-bd32-1acbbd3f4ed7 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:40.870288 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca64660-c7ca-4d72-bd32-1acbbd3f4ed7 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:40.871624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 90/179] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:40.873515 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca64660-c7ca-4d72-bd32-1acbbd3f4ed7 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:40.950591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:40.986778 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 90/180] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:40.992099 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca64660-c7ca-4d72-bd32-1acbbd3f4ed7 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:40.993099 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dd9f2-9ef0-476e-8144-6aab185c526a None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:40.993392 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dd9f2-9ef0-476e-8144-6aab185c526a None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:40.993680 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dd9f2-9ef0-476e-8144-6aab185c526a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:41.012744 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-2a2dd9f2-9ef0-476e-8144-6aab185c526a None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:41.013964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-2a2dd9f2-9ef0-476e-8144-6aab185c526a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:41.032402 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ea941-66ca-4bb6-8f33-04e43d0380b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:41.033079 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ea941-66ca-4bb6-8f33-04e43d0380b2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:41.033365 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ea941-66ca-4bb6-8f33-04e43d0380b2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:41.033645 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ea941-66ca-4bb6-8f33-04e43d0380b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:41.087210 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:41.096874 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 91/181] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:41.098012 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dd9f2-9ef0-476e-8144-6aab185c526a None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:41.112066 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ea941-66ca-4bb6-8f33-04e43d0380b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:41.113575 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ea941-66ca-4bb6-8f33-04e43d0380b2 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:41.223348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 91/182] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:41.224240 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dd9f2-9ef0-476e-8144-6aab185c526a None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:41.225431 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882d6ec-7fc4-48fb-b2e9-7b256baddf25 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:41.225875 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882d6ec-7fc4-48fb-b2e9-7b256baddf25 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:41.226333 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882d6ec-7fc4-48fb-b2e9-7b256baddf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:41.227288 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:41.246595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-7882d6ec-7fc4-48fb-b2e9-7b256baddf25 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:41.248059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-7882d6ec-7fc4-48fb-b2e9-7b256baddf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:41.264221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7bb07-8483-42eb-96c8-30e234e212da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:41.264907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7bb07-8483-42eb-96c8-30e234e212da None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:41.265193 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7bb07-8483-42eb-96c8-30e234e212da None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:41.265469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7bb07-8483-42eb-96c8-30e234e212da None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:41.266216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7bb07-8483-42eb-96c8-30e234e212da None admin] RBAC: Adding query filter params (name=admin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:41.266560 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7bb07-8483-42eb-96c8-30e234e212da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:41.267947 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7bb07-8483-42eb-96c8-30e234e212da None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:41.276432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 92/183] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:41.277223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7bb07-8483-42eb-96c8-30e234e212da None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:41.328634 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 92/184] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:41.329809 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882d6ec-7fc4-48fb-b2e9-7b256baddf25 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:41.348762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:41.405659 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7bb07-8483-42eb-96c8-30e234e212da None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:41.450418 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d44a27-d6c8-4b6b-a0ac-183f26c78867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:41.451342 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d44a27-d6c8-4b6b-a0ac-183f26c78867 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:41.451625 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d44a27-d6c8-4b6b-a0ac-183f26c78867 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:41.451898 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d44a27-d6c8-4b6b-a0ac-183f26c78867 None admin] PATH_INFO: `/v3/projects/3ae9b8c5024e40f0b79dcf1d40c57424` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:41.453025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882d6ec-7fc4-48fb-b2e9-7b256baddf25 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:41.453499 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d44a27-d6c8-4b6b-a0ac-183f26c78867 None admin] RBAC: Authorizing `identity:get_project(project_id=3ae9b8c5024e40f0b79dcf1d40c57424)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:41.454712 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d44a27-d6c8-4b6b-a0ac-183f26c78867 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:41.461553 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 93/185] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:41 2022] GET /identity/v3/projects/3ae9b8c5024e40f0b79dcf1d40c57424 => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:41.461553 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d44a27-d6c8-4b6b-a0ac-183f26c78867 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:41.491493 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac12189-6f4e-49cb-a49f-f1febc10d00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:41.491720 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:41.492221 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac12189-6f4e-49cb-a49f-f1febc10d00a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:41.492505 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac12189-6f4e-49cb-a49f-f1febc10d00a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:41.492809 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac12189-6f4e-49cb-a49f-f1febc10d00a None admin] PATH_INFO: `/v3/domains/default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:41.494157 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac12189-6f4e-49cb-a49f-f1febc10d00a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:41.495965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac12189-6f4e-49cb-a49f-f1febc10d00a None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:41.497970 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 93/186] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:41.498854 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac12189-6f4e-49cb-a49f-f1febc10d00a None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:41.577423 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d44a27-d6c8-4b6b-a0ac-183f26c78867 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:41.581004 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:41.606637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac12189-6f4e-49cb-a49f-f1febc10d00a None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:41.613048 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147d2a2-ad33-4d01-bc5f-8d9bdaf4a3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:41.613929 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147d2a2-ad33-4d01-bc5f-8d9bdaf4a3dd None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:41.614212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147d2a2-ad33-4d01-bc5f-8d9bdaf4a3dd None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:41.614534 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147d2a2-ad33-4d01-bc5f-8d9bdaf4a3dd None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:41.615764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147d2a2-ad33-4d01-bc5f-8d9bdaf4a3dd None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:41.617235 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147d2a2-ad33-4d01-bc5f-8d9bdaf4a3dd None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:41.619044 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 94/187] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:41 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:41.620525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147d2a2-ad33-4d01-bc5f-8d9bdaf4a3dd None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:41.648310 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f4eb1-f2a9-4122-b8e7-1a8bc4e7c8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:41.649299 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f4eb1-f2a9-4122-b8e7-1a8bc4e7c8eb None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:41.649579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f4eb1-f2a9-4122-b8e7-1a8bc4e7c8eb None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:41.649969 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f4eb1-f2a9-4122-b8e7-1a8bc4e7c8eb None admin] PATH_INFO: `/v3/users/a651e8aad31c4414842d66f96e07187d` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:41.664977 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f4eb1-f2a9-4122-b8e7-1a8bc4e7c8eb None admin] RBAC: Authorizing `identity:get_user(user_id=a651e8aad31c4414842d66f96e07187d)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:41.666560 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f4eb1-f2a9-4122-b8e7-1a8bc4e7c8eb None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:41.672935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 94/188] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:41 2022] GET /identity/v3/users/a651e8aad31c4414842d66f96e07187d => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:41.673720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f4eb1-f2a9-4122-b8e7-1a8bc4e7c8eb None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:41.732961 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:41.735999 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147d2a2-ad33-4d01-bc5f-8d9bdaf4a3dd None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:41.771943 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d972b-f0ef-44f5-ab7f-a06912297049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:41.772687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d972b-f0ef-44f5-ab7f-a06912297049 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:41.772967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d972b-f0ef-44f5-ab7f-a06912297049 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:41.773241 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d972b-f0ef-44f5-ab7f-a06912297049 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:41.784434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d972b-f0ef-44f5-ab7f-a06912297049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:41.784789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f4eb1-f2a9-4122-b8e7-1a8bc4e7c8eb None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-d22d972b-f0ef-44f5-ab7f-a06912297049 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:41.792998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:41.794183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:41.799295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:41.799295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:41.799295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:41.799295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:41.799295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:41.799295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:41.799295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:41.799295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:41.799295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:41.799295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:41.799295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:17:41.799295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:41.799295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 95/189] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:17:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:41.799295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d972b-f0ef-44f5-ab7f-a06912297049 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:41.829086 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eccd71-b4ec-4f61-bdb8-1d9a0a54c873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:41.829752 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eccd71-b4ec-4f61-bdb8-1d9a0a54c873 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:41.830035 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eccd71-b4ec-4f61-bdb8-1d9a0a54c873 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:41.830367 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eccd71-b4ec-4f61-bdb8-1d9a0a54c873 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:41.831186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eccd71-b4ec-4f61-bdb8-1d9a0a54c873 None admin] RBAC: Adding query filter params (role.id=666cfd1181184612ba462b8eefcad6a7, scope.project.id=3ae9b8c5024e40f0b79dcf1d40c57424, user.id=d5972d97f8c144e99fb7fb22399f0d94) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:41.831498 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eccd71-b4ec-4f61-bdb8-1d9a0a54c873 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:41.833172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eccd71-b4ec-4f61-bdb8-1d9a0a54c873 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:41.835053 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:41.854553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 95/190] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:41 2022] GET /identity/v3/role_assignments?user.id=d5972d97f8c144e99fb7fb22399f0d94&scope.project.id=3ae9b8c5024e40f0b79dcf1d40c57424&role.id=666cfd1181184612ba462b8eefcad6a7 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:41.855573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eccd71-b4ec-4f61-bdb8-1d9a0a54c873 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:41.911320 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d972b-f0ef-44f5-ab7f-a06912297049 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:41.966424 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6592b2d-c648-4890-b943-d228d32e073b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:41.968042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6592b2d-c648-4890-b943-d228d32e073b None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:41.968754 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6592b2d-c648-4890-b943-d228d32e073b None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:41.969190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6592b2d-c648-4890-b943-d228d32e073b None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:41.970371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6592b2d-c648-4890-b943-d228d32e073b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:41.972672 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6592b2d-c648-4890-b943-d228d32e073b None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:41.988963 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eccd71-b4ec-4f61-bdb8-1d9a0a54c873 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:42.001550 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 96/191] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:17:41 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:17:42.003278 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6592b2d-c648-4890-b943-d228d32e073b None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:42.041104 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:42.045918 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f669a-e078-4d30-ad6e-0a57a5bf9746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:42.046782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f669a-e078-4d30-ad6e-0a57a5bf9746 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:42.047108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f669a-e078-4d30-ad6e-0a57a5bf9746 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:42.047381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f669a-e078-4d30-ad6e-0a57a5bf9746 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:42.048320 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f669a-e078-4d30-ad6e-0a57a5bf9746 None admin] RBAC: Adding query filter params (role.id=666cfd1181184612ba462b8eefcad6a7, scope.project.id=ba315bae61f943debd8460a5fa9dd151, user.id=d5972d97f8c144e99fb7fb22399f0d94) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:42.050381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f669a-e078-4d30-ad6e-0a57a5bf9746 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:42.053540 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f669a-e078-4d30-ad6e-0a57a5bf9746 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:42.066934 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 96/192] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:41 2022] GET /identity/v3/role_assignments?user.id=d5972d97f8c144e99fb7fb22399f0d94&scope.project.id=ba315bae61f943debd8460a5fa9dd151&role.id=666cfd1181184612ba462b8eefcad6a7 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:42.067818 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f669a-e078-4d30-ad6e-0a57a5bf9746 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:42.170174 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:42.188141 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6592b2d-c648-4890-b943-d228d32e073b None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:42.234236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a762a-3849-4ff4-844b-19896456542e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:42.235805 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a762a-3849-4ff4-844b-19896456542e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:42.236298 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a762a-3849-4ff4-844b-19896456542e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:42.236775 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a762a-3849-4ff4-844b-19896456542e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:42.237136 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f669a-e078-4d30-ad6e-0a57a5bf9746 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:42.297033 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577430a-617d-4fd8-995f-4744ea625911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:42.297818 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577430a-617d-4fd8-995f-4744ea625911 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:42.298105 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577430a-617d-4fd8-995f-4744ea625911 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:42.298467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577430a-617d-4fd8-995f-4744ea625911 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:42.299256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c577430a-617d-4fd8-995f-4744ea625911 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:42.299571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c577430a-617d-4fd8-995f-4744ea625911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:42.301054 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c577430a-617d-4fd8-995f-4744ea625911 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:42.316053 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 97/193] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:17:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:42.317501 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac39b7-5002-4815-88a9-043657b393dd None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:42.317793 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac39b7-5002-4815-88a9-043657b393dd None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:42.318071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac39b7-5002-4815-88a9-043657b393dd None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:42.319410 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 98/194] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:42.324916 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac39b7-5002-4815-88a9-043657b393dd None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:42.341549 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a762a-3849-4ff4-844b-19896456542e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:42.343887 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a762a-3849-4ff4-844b-19896456542e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:42.366775 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:42.470863 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac39b7-5002-4815-88a9-043657b393dd None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:42.490425 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 97/195] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:42.490425 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224a3a9-1d9b-4bbb-b935-4b59bccf2b98 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:42.490425 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224a3a9-1d9b-4bbb-b935-4b59bccf2b98 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:42.490425 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224a3a9-1d9b-4bbb-b935-4b59bccf2b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:42.505127 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:42.508474 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-9224a3a9-1d9b-4bbb-b935-4b59bccf2b98 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:42.509860 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-9224a3a9-1d9b-4bbb-b935-4b59bccf2b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:42.524656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d67cac-6de0-4a18-a925-07dc902deb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:42.525613 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d67cac-6de0-4a18-a925-07dc902deb12 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:42.526078 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d67cac-6de0-4a18-a925-07dc902deb12 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:42.526605 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d67cac-6de0-4a18-a925-07dc902deb12 None admin] PATH_INFO: `/v3/users/a651e8aad31c4414842d66f96e07187d` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:42.527830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d67cac-6de0-4a18-a925-07dc902deb12 None admin] RBAC: Authorizing `identity:get_user(user_id=a651e8aad31c4414842d66f96e07187d)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:42.529325 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d67cac-6de0-4a18-a925-07dc902deb12 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:42.531040 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 99/196] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:42 2022] GET /identity/v3/users/a651e8aad31c4414842d66f96e07187d => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:42.531822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d67cac-6de0-4a18-a925-07dc902deb12 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:42.603773 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 98/197] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:42.604527 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224a3a9-1d9b-4bbb-b935-4b59bccf2b98 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:42.665468 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d67cac-6de0-4a18-a925-07dc902deb12 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:42.672028 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:42.704658 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65176194-fc8c-4d20-b11d-576f0952673e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:42.705538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65176194-fc8c-4d20-b11d-576f0952673e None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:42.705824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65176194-fc8c-4d20-b11d-576f0952673e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:42.706101 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65176194-fc8c-4d20-b11d-576f0952673e None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:42.707342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65176194-fc8c-4d20-b11d-576f0952673e None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:42.708875 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65176194-fc8c-4d20-b11d-576f0952673e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:42.710887 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 100/198] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:42 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:42.711489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65176194-fc8c-4d20-b11d-576f0952673e None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:42.743229 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224a3a9-1d9b-4bbb-b935-4b59bccf2b98 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:42.776258 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:42.781876 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b985928-1a41-466d-a0a2-5465bad1f8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:42.782577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b985928-1a41-466d-a0a2-5465bad1f8f5 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:42.782856 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b985928-1a41-466d-a0a2-5465bad1f8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:42.783129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b985928-1a41-466d-a0a2-5465bad1f8f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:42.827044 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65176194-fc8c-4d20-b11d-576f0952673e None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:42.856526 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b985928-1a41-466d-a0a2-5465bad1f8f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:42.857918 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b985928-1a41-466d-a0a2-5465bad1f8f5 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:42.865259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dbf5b-d35c-42d9-b67e-f81c18c152f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:42.866119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dbf5b-d35c-42d9-b67e-f81c18c152f4 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:42.866448 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dbf5b-d35c-42d9-b67e-f81c18c152f4 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:42.866825 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dbf5b-d35c-42d9-b67e-f81c18c152f4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:42.878799 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52dbf5b-d35c-42d9-b67e-f81c18c152f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-c52dbf5b-d35c-42d9-b67e-f81c18c152f4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:42.887399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:42.888457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:42.891032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:42.891032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:42.891032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:42.891032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:42.891032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:42.891032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:42.891032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:42.891032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:42.891032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:42.891032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:42.891032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:17:42.891032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:42.891032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 101/199] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:42.891032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dbf5b-d35c-42d9-b67e-f81c18c152f4 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:42.905883 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:42.966912 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 99/200] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:42.967690 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b985928-1a41-466d-a0a2-5465bad1f8f5 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:43.028606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dbf5b-d35c-42d9-b67e-f81c18c152f4 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:43.057483 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:43.071720 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b985928-1a41-466d-a0a2-5465bad1f8f5 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:43.077685 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86763dd-71c4-4d18-9ac6-eac7c9a64674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:43.078576 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86763dd-71c4-4d18-9ac6-eac7c9a64674 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:43.079051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86763dd-71c4-4d18-9ac6-eac7c9a64674 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:43.079495 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86763dd-71c4-4d18-9ac6-eac7c9a64674 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:43.080997 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86763dd-71c4-4d18-9ac6-eac7c9a64674 None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=3022d4bdd09249328ba6a2f60404faf0, user.id=a651e8aad31c4414842d66f96e07187d) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:43.081469 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86763dd-71c4-4d18-9ac6-eac7c9a64674 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:43.083587 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86763dd-71c4-4d18-9ac6-eac7c9a64674 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:43.092573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 102/201] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:42 2022] GET /identity/v3/role_assignments?user.id=a651e8aad31c4414842d66f96e07187d&scope.project.id=3022d4bdd09249328ba6a2f60404faf0&role.id=b239f941cb724331bf28dc489b70ad1d => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:43.093652 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86763dd-71c4-4d18-9ac6-eac7c9a64674 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:43.119013 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306319b-3807-434c-b2ff-3f4756dfbac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:43.119013 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306319b-3807-434c-b2ff-3f4756dfbac5 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:43.119013 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306319b-3807-434c-b2ff-3f4756dfbac5 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:43.120290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306319b-3807-434c-b2ff-3f4756dfbac5 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:43.122731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306319b-3807-434c-b2ff-3f4756dfbac5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:43.123040 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306319b-3807-434c-b2ff-3f4756dfbac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:43.124490 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306319b-3807-434c-b2ff-3f4756dfbac5 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:43.142135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 100/202] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:43.148375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306319b-3807-434c-b2ff-3f4756dfbac5 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:43.168596 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:43.302150 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86763dd-71c4-4d18-9ac6-eac7c9a64674 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:43.308459 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306319b-3807-434c-b2ff-3f4756dfbac5 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:43.357597 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeebad0-54e4-4a73-bab0-48963956739f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:43.358893 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeebad0-54e4-4a73-bab0-48963956739f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:43.359176 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeebad0-54e4-4a73-bab0-48963956739f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:43.359448 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeebad0-54e4-4a73-bab0-48963956739f None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:43.361652 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeebad0-54e4-4a73-bab0-48963956739f None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:43.363175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeebad0-54e4-4a73-bab0-48963956739f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:43.364909 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 103/203] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:43 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:43.371214 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeebad0-54e4-4a73-bab0-48963956739f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:43.377927 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba8501-12dd-4707-b47d-6c9a9981cf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:43.378973 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba8501-12dd-4707-b47d-6c9a9981cf1f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:43.379321 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba8501-12dd-4707-b47d-6c9a9981cf1f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:43.379658 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba8501-12dd-4707-b47d-6c9a9981cf1f None admin] PATH_INFO: `/v3/users/f79add60659848fda751b9e18c150aed` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:43.380937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba8501-12dd-4707-b47d-6c9a9981cf1f None admin] RBAC: Authorizing `identity:get_user(user_id=f79add60659848fda751b9e18c150aed)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:43.382966 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba8501-12dd-4707-b47d-6c9a9981cf1f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:43.385266 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 101/204] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:43 2022] GET /identity/v3/users/f79add60659848fda751b9e18c150aed => generated 289 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:43.392679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba8501-12dd-4707-b47d-6c9a9981cf1f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:43.438975 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:43.490677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeebad0-54e4-4a73-bab0-48963956739f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:43.558005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:43.560424 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e7023-a981-4752-8621-61614a74528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:43.561260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e7023-a981-4752-8621-61614a74528d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:43.561379 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e7023-a981-4752-8621-61614a74528d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:43.561640 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e7023-a981-4752-8621-61614a74528d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:43.562511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e7023-a981-4752-8621-61614a74528d None admin] RBAC: Adding query filter params (role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2, scope.project.id=3022d4bdd09249328ba6a2f60404faf0, user.id=a651e8aad31c4414842d66f96e07187d) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:43.562825 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e7023-a981-4752-8621-61614a74528d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:43.564293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e7023-a981-4752-8621-61614a74528d None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:43.578438 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 104/205] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:43 2022] GET /identity/v3/role_assignments?user.id=a651e8aad31c4414842d66f96e07187d&scope.project.id=3022d4bdd09249328ba6a2f60404faf0&role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:43.613518 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba8501-12dd-4707-b47d-6c9a9981cf1f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:43.672148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e7644-e459-47ef-a0fa-d82e0a3c582c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:43.672750 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e7644-e459-47ef-a0fa-d82e0a3c582c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:43.673030 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e7644-e459-47ef-a0fa-d82e0a3c582c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:43.673305 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e7644-e459-47ef-a0fa-d82e0a3c582c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:43.758520 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e7644-e459-47ef-a0fa-d82e0a3c582c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:43.760078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e7644-e459-47ef-a0fa-d82e0a3c582c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:43.841163 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:43.865406 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 102/206] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:43.871302 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e7023-a981-4752-8621-61614a74528d None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:43.988311 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e7023-a981-4752-8621-61614a74528d None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:44.013853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:44.035175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b81f9-d0ff-42f8-86c7-502b2df7d3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:44.036117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b81f9-d0ff-42f8-86c7-502b2df7d3c8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:44.036404 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b81f9-d0ff-42f8-86c7-502b2df7d3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:44.036723 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b81f9-d0ff-42f8-86c7-502b2df7d3c8 None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:44.042984 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b81f9-d0ff-42f8-86c7-502b2df7d3c8 None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:44.044484 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b81f9-d0ff-42f8-86c7-502b2df7d3c8 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:44.046226 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 105/207] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:43 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:17:44.052241 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e7644-e459-47ef-a0fa-d82e0a3c582c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:44.209983 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e7644-e459-47ef-a0fa-d82e0a3c582c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:44.260308 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc453b-5dfb-4363-b4ca-38a50e27d818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:44.260990 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc453b-5dfb-4363-b4ca-38a50e27d818 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:44.261273 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc453b-5dfb-4363-b4ca-38a50e27d818 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:44.261549 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc453b-5dfb-4363-b4ca-38a50e27d818 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:44.262428 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc453b-5dfb-4363-b4ca-38a50e27d818 None admin] RBAC: Adding query filter params (role.id=086ec02a15114cc3b858b231d55e19aa, scope.project.id=ba315bae61f943debd8460a5fa9dd151, user.id=f79add60659848fda751b9e18c150aed) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:44.262742 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc453b-5dfb-4363-b4ca-38a50e27d818 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:44.264208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc453b-5dfb-4363-b4ca-38a50e27d818 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:44.275228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 103/208] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:44 2022] GET /identity/v3/role_assignments?user.id=f79add60659848fda751b9e18c150aed&scope.project.id=ba315bae61f943debd8460a5fa9dd151&role.id=086ec02a15114cc3b858b231d55e19aa => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:44.752285 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eab774-0512-4f79-95cc-dbd8f75e334c None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:44.753360 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eab774-0512-4f79-95cc-dbd8f75e334c None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:44.753875 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eab774-0512-4f79-95cc-dbd8f75e334c None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:44.755837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 106/209] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:44.761183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1d975-4ccf-4991-9504-82b7a7575cd7 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:44.762050 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1d975-4ccf-4991-9504-82b7a7575cd7 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:44.762586 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1d975-4ccf-4991-9504-82b7a7575cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:44.780660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-e0f1d975-4ccf-4991-9504-82b7a7575cd7 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:44.782213 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-e0f1d975-4ccf-4991-9504-82b7a7575cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:44.819454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:44.842836 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d3193-f19a-469d-aae6-66677ca29028 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:44.843687 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d3193-f19a-469d-aae6-66677ca29028 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:44.844271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d3193-f19a-469d-aae6-66677ca29028 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:44.847045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 107/210] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:17:44.852612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651e2d2-631b-4a96-a030-4c8b8f54e3a8 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:44.853158 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651e2d2-631b-4a96-a030-4c8b8f54e3a8 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:44.853763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651e2d2-631b-4a96-a030-4c8b8f54e3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:44.870723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 104/211] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:44.877680 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-a651e2d2-631b-4a96-a030-4c8b8f54e3a8 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:44.878964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-a651e2d2-631b-4a96-a030-4c8b8f54e3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:44.894107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1d975-4ccf-4991-9504-82b7a7575cd7 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:44.985885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 108/212] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:17:45.013383 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651e2d2-631b-4a96-a030-4c8b8f54e3a8 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:45.025927 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1d975-4ccf-4991-9504-82b7a7575cd7 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:45.067116 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b679cd-dc16-4919-986f-7ebb1eed920d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:45.068095 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b679cd-dc16-4919-986f-7ebb1eed920d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:45.068448 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b679cd-dc16-4919-986f-7ebb1eed920d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:45.068799 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b679cd-dc16-4919-986f-7ebb1eed920d None admin] PATH_INFO: `/v3/domains/default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:45.070155 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b679cd-dc16-4919-986f-7ebb1eed920d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:45.071947 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b679cd-dc16-4919-986f-7ebb1eed920d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:45.079125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 105/213] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:17:45.080090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b679cd-dc16-4919-986f-7ebb1eed920d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:45.124593 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:45.138255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651e2d2-631b-4a96-a030-4c8b8f54e3a8 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:45.154694 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:45.177346 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff376b7c-0ee6-42e4-8da1-26c188c73c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:45.178362 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff376b7c-0ee6-42e4-8da1-26c188c73c9b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:45.178751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff376b7c-0ee6-42e4-8da1-26c188c73c9b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:45.179151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff376b7c-0ee6-42e4-8da1-26c188c73c9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:45.191168 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff376b7c-0ee6-42e4-8da1-26c188c73c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-ff376b7c-0ee6-42e4-8da1-26c188c73c9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:45.199598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:45.200853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:45.201826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:45.201826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:45.201826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:45.201826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:45.201826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:45.201826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:45.201826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:45.201826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:45.201826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:45.201826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:45.201826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:45.201826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:45.202836 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 109/214] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:17:45.203370 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b679cd-dc16-4919-986f-7ebb1eed920d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:45.207809 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff376b7c-0ee6-42e4-8da1-26c188c73c9b None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:45.240474 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8191ce-f5cb-47c7-b734-1014b5ac5904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:45.241520 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8191ce-f5cb-47c7-b734-1014b5ac5904 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:45.242007 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8191ce-f5cb-47c7-b734-1014b5ac5904 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:45.242510 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8191ce-f5cb-47c7-b734-1014b5ac5904 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:45.243432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8191ce-f5cb-47c7-b734-1014b5ac5904 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:45.245192 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8191ce-f5cb-47c7-b734-1014b5ac5904 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:45.267880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 106/215] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:17:45 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:17:45.329316 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff376b7c-0ee6-42e4-8da1-26c188c73c9b None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:45.367523 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ceba14-d52a-4cd3-8576-c604b627fdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:45.368569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ceba14-d52a-4cd3-8576-c604b627fdcd None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:45.369219 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ceba14-d52a-4cd3-8576-c604b627fdcd None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:45.369676 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ceba14-d52a-4cd3-8576-c604b627fdcd None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:45.370934 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ceba14-d52a-4cd3-8576-c604b627fdcd None admin] RBAC: Adding query filter params (name=member) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:45.371428 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ceba14-d52a-4cd3-8576-c604b627fdcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:45.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ceba14-d52a-4cd3-8576-c604b627fdcd None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:45.383625 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 110/216] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:45.391468 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8191ce-f5cb-47c7-b734-1014b5ac5904 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:45.503882 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:45.516286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8191ce-f5cb-47c7-b734-1014b5ac5904 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:45.552628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e763571-4824-4f90-b37b-82f57bccc102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:45.553529 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e763571-4824-4f90-b37b-82f57bccc102 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:45.553815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e763571-4824-4f90-b37b-82f57bccc102 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:45.554097 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e763571-4824-4f90-b37b-82f57bccc102 None admin] PATH_INFO: `/v3/users/fe3cbf808fa14fcebbf1cff8d2b2cf29` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:45.569535 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e763571-4824-4f90-b37b-82f57bccc102 None admin] RBAC: Authorizing `identity:get_user(user_id=fe3cbf808fa14fcebbf1cff8d2b2cf29)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:45.571061 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e763571-4824-4f90-b37b-82f57bccc102 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:45.573553 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 107/217] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:45 2022] GET /identity/v3/users/fe3cbf808fa14fcebbf1cff8d2b2cf29 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:45.579284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ceba14-d52a-4cd3-8576-c604b627fdcd None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:45.622767 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:45.695423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ceba14-d52a-4cd3-8576-c604b627fdcd None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:45.732796 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffafaf6-5ddb-4bde-a831-503d2236993a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:45.733900 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffafaf6-5ddb-4bde-a831-503d2236993a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:45.734522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffafaf6-5ddb-4bde-a831-503d2236993a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:45.735021 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffafaf6-5ddb-4bde-a831-503d2236993a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:45.803598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8195d9-7734-442f-be27-4af850b19684 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:45.804588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8195d9-7734-442f-be27-4af850b19684 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:45.804868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8195d9-7734-442f-be27-4af850b19684 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:45.806560 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 108/218] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:45.811589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffafaf6-5ddb-4bde-a831-503d2236993a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:45.813102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffafaf6-5ddb-4bde-a831-503d2236993a None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:45.813990 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356665d9-a05c-4144-9772-36f7ac4a5bb0 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:45.814297 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356665d9-a05c-4144-9772-36f7ac4a5bb0 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:45.814687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356665d9-a05c-4144-9772-36f7ac4a5bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:45.835484 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-356665d9-a05c-4144-9772-36f7ac4a5bb0 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:45.836680 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-356665d9-a05c-4144-9772-36f7ac4a5bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:45.883395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:45.916171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 109/219] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:45.956098 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356665d9-a05c-4144-9772-36f7ac4a5bb0 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:46.049242 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 111/220] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:46.054594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffafaf6-5ddb-4bde-a831-503d2236993a None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:46.056716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:46.080362 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356665d9-a05c-4144-9772-36f7ac4a5bb0 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:46.119918 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d877cf-2319-4f64-8505-c9c5431d94e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:46.121166 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d877cf-2319-4f64-8505-c9c5431d94e7 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:46.121678 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d877cf-2319-4f64-8505-c9c5431d94e7 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:46.122165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d877cf-2319-4f64-8505-c9c5431d94e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:46.134211 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d877cf-2319-4f64-8505-c9c5431d94e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-e3d877cf-2319-4f64-8505-c9c5431d94e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:46.142880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:46.143934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:46.144848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:46.144848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:46.144848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:46.144848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:46.144848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:46.144848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:46.144848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:46.144848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:46.144848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:46.144848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:46.144848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:46.144848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:46.146125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 110/221] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:17:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:17:46.151763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d877cf-2319-4f64-8505-c9c5431d94e7 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:46.269462 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffafaf6-5ddb-4bde-a831-503d2236993a None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:46.280261 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d877cf-2319-4f64-8505-c9c5431d94e7 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:46.305906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18826bc4-b58f-4790-a2b0-79012e753fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:46.306866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18826bc4-b58f-4790-a2b0-79012e753fe7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:46.307151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18826bc4-b58f-4790-a2b0-79012e753fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:46.307431 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18826bc4-b58f-4790-a2b0-79012e753fe7 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:46.308626 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18826bc4-b58f-4790-a2b0-79012e753fe7 None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:46.310098 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18826bc4-b58f-4790-a2b0-79012e753fe7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:46.312166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 112/222] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:46 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:17:46.313487 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5340ad-0687-4bea-87a7-b6e4d49def75 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:46.313772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5340ad-0687-4bea-87a7-b6e4d49def75 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:46.314047 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5340ad-0687-4bea-87a7-b6e4d49def75 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:46.315503 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 113/223] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:46.320335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f53047-37fa-48a0-b73c-951fba24da5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:46.320523 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5340ad-0687-4bea-87a7-b6e4d49def75 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:46.321025 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f53047-37fa-48a0-b73c-951fba24da5c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:46.321310 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f53047-37fa-48a0-b73c-951fba24da5c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:46.321591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f53047-37fa-48a0-b73c-951fba24da5c None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:46.322499 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f53047-37fa-48a0-b73c-951fba24da5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:46.322774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f53047-37fa-48a0-b73c-951fba24da5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:46.324177 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f53047-37fa-48a0-b73c-951fba24da5c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:46.324420 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:46.338804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 111/224] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:46.338804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86cd12-d6e2-4281-b10c-c29aa1bd0276 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:46.338804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86cd12-d6e2-4281-b10c-c29aa1bd0276 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:46.338804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86cd12-d6e2-4281-b10c-c29aa1bd0276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:46.346685 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:46.358623 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-1f86cd12-d6e2-4281-b10c-c29aa1bd0276 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:46.359917 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-1f86cd12-d6e2-4281-b10c-c29aa1bd0276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:46.441794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 112/225] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:46.442759 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86cd12-d6e2-4281-b10c-c29aa1bd0276 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:46.480418 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5340ad-0687-4bea-87a7-b6e4d49def75 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:46.517388 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185f869-9687-4046-89ce-095de3f060b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:46.518052 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185f869-9687-4046-89ce-095de3f060b2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:46.518360 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185f869-9687-4046-89ce-095de3f060b2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:46.518639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185f869-9687-4046-89ce-095de3f060b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:46.519482 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185f869-9687-4046-89ce-095de3f060b2 None admin] RBAC: Adding query filter params (role.id=666cfd1181184612ba462b8eefcad6a7, scope.project.id=3022d4bdd09249328ba6a2f60404faf0, user.id=fe3cbf808fa14fcebbf1cff8d2b2cf29) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:46.519792 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185f869-9687-4046-89ce-095de3f060b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:46.521265 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185f869-9687-4046-89ce-095de3f060b2 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:46.527783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 114/226] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:46 2022] GET /identity/v3/role_assignments?user.id=fe3cbf808fa14fcebbf1cff8d2b2cf29&scope.project.id=3022d4bdd09249328ba6a2f60404faf0&role.id=666cfd1181184612ba462b8eefcad6a7 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:17:46.528477 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185f869-9687-4046-89ce-095de3f060b2 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:46.565635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86cd12-d6e2-4281-b10c-c29aa1bd0276 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:46.637032 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4fac9-6fee-4121-96df-93fe72d3f248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:46.637956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4fac9-6fee-4121-96df-93fe72d3f248 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:46.638238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4fac9-6fee-4121-96df-93fe72d3f248 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:46.638549 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4fac9-6fee-4121-96df-93fe72d3f248 None admin] PATH_INFO: `/v3/users/a651e8aad31c4414842d66f96e07187d` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:46.640673 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca4fac9-6fee-4121-96df-93fe72d3f248 None admin] RBAC: Authorizing `identity:get_user(user_id=a651e8aad31c4414842d66f96e07187d)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:46.643934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca4fac9-6fee-4121-96df-93fe72d3f248 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:46.645988 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 113/227] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:46 2022] GET /identity/v3/users/a651e8aad31c4414842d66f96e07187d => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:46.652212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4fac9-6fee-4121-96df-93fe72d3f248 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:46.655392 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:46.659868 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:46.741136 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185f869-9687-4046-89ce-095de3f060b2 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:46.778638 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4fac9-6fee-4121-96df-93fe72d3f248 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:46.786987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba5869-a677-4da6-bf13-8bb3ad4785f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:46.787826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba5869-a677-4da6-bf13-8bb3ad4785f8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:46.788108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba5869-a677-4da6-bf13-8bb3ad4785f8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:46.788845 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba5869-a677-4da6-bf13-8bb3ad4785f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:46.807827 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ba5869-a677-4da6-bf13-8bb3ad4785f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-24ba5869-a677-4da6-bf13-8bb3ad4785f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:46.823071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:46.826372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:46.828507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:46.828507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:46.828507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:46.828507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:46.828507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:46.828507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:46.828507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:46.828507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:46.828507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:46.828507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:46.828507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:17:46.828507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:46.828507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 115/228] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:17:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:46.833792 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba5869-a677-4da6-bf13-8bb3ad4785f8 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:46.845542 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23776bab-9852-429c-81a6-82cfcc35f929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:46.846500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23776bab-9852-429c-81a6-82cfcc35f929 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:46.846785 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23776bab-9852-429c-81a6-82cfcc35f929 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:46.847062 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23776bab-9852-429c-81a6-82cfcc35f929 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:46.851891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23776bab-9852-429c-81a6-82cfcc35f929 None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:46.853402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23776bab-9852-429c-81a6-82cfcc35f929 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:46.855524 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 114/229] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:46 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:46.861137 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23776bab-9852-429c-81a6-82cfcc35f929 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:46.996063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba5869-a677-4da6-bf13-8bb3ad4785f8 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:47.026955 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:47.044454 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8987ce-06b2-4c4c-ae9a-44524ae8dd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:47.045156 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8987ce-06b2-4c4c-ae9a-44524ae8dd5a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:47.045351 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:47.045515 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8987ce-06b2-4c4c-ae9a-44524ae8dd5a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:47.045698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8987ce-06b2-4c4c-ae9a-44524ae8dd5a None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:47.046520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8987ce-06b2-4c4c-ae9a-44524ae8dd5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:47.046802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8987ce-06b2-4c4c-ae9a-44524ae8dd5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:47.048298 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8987ce-06b2-4c4c-ae9a-44524ae8dd5a None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:47.068045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 116/230] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:17:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:47.074128 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8987ce-06b2-4c4c-ae9a-44524ae8dd5a None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:47.108691 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23776bab-9852-429c-81a6-82cfcc35f929 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:47.175849 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879c05a-dbea-49b2-95bf-1eb9e87e5a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:47.177263 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879c05a-dbea-49b2-95bf-1eb9e87e5a08 None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:47.177758 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879c05a-dbea-49b2-95bf-1eb9e87e5a08 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:47.178289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879c05a-dbea-49b2-95bf-1eb9e87e5a08 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0/users/a651e8aad31c4414842d66f96e07187d/roles/b239f941cb724331bf28dc489b70ad1d` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:47.183694 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879c05a-dbea-49b2-95bf-1eb9e87e5a08 None admin] RBAC: Authorizing `identity:create_grant(project_id=3022d4bdd09249328ba6a2f60404faf0, user_id=a651e8aad31c4414842d66f96e07187d, role_id=b239f941cb724331bf28dc489b70ad1d)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:47.185664 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879c05a-dbea-49b2-95bf-1eb9e87e5a08 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:47.204137 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8987ce-06b2-4c4c-ae9a-44524ae8dd5a None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:47.210853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 115/231] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:17:46 2022] PUT /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0/users/a651e8aad31c4414842d66f96e07187d/roles/b239f941cb724331bf28dc489b70ad1d => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:17:47.242719 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315db6bb-8e80-494d-8eac-b14e0f1d3620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:47.243634 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315db6bb-8e80-494d-8eac-b14e0f1d3620 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:47.245059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315db6bb-8e80-494d-8eac-b14e0f1d3620 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:47.245368 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315db6bb-8e80-494d-8eac-b14e0f1d3620 None admin] PATH_INFO: `/v3/users/a651e8aad31c4414842d66f96e07187d` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:47.246833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315db6bb-8e80-494d-8eac-b14e0f1d3620 None admin] RBAC: Authorizing `identity:get_user(user_id=a651e8aad31c4414842d66f96e07187d)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:47.248328 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315db6bb-8e80-494d-8eac-b14e0f1d3620 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:47.250025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 117/232] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:47 2022] GET /identity/v3/users/a651e8aad31c4414842d66f96e07187d => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:47.255887 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879c05a-dbea-49b2-95bf-1eb9e87e5a08 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:47.396482 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879c05a-dbea-49b2-95bf-1eb9e87e5a08 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:47.445974 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:47.455790 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e30050-8ce5-48d4-8558-a2df742f20e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:47.457094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e30050-8ce5-48d4-8558-a2df742f20e0 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:47.457612 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e30050-8ce5-48d4-8558-a2df742f20e0 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:47.458093 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e30050-8ce5-48d4-8558-a2df742f20e0 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:47.459798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e30050-8ce5-48d4-8558-a2df742f20e0 None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:47.461522 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e30050-8ce5-48d4-8558-a2df742f20e0 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:47.464003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 116/233] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:47 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:47.469849 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315db6bb-8e80-494d-8eac-b14e0f1d3620 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:47.550362 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:47.598062 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315db6bb-8e80-494d-8eac-b14e0f1d3620 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:47.633592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367429dd-18f8-4f4c-9129-c502e8003bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:47.634595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367429dd-18f8-4f4c-9129-c502e8003bd4 None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:47.634881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367429dd-18f8-4f4c-9129-c502e8003bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:47.635138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367429dd-18f8-4f4c-9129-c502e8003bd4 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0/users/a651e8aad31c4414842d66f96e07187d/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:47.637390 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367429dd-18f8-4f4c-9129-c502e8003bd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=3022d4bdd09249328ba6a2f60404faf0, user_id=a651e8aad31c4414842d66f96e07187d, role_id=6d25238fbc0e4d11bac1bcdc7e50d2e2)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:47.638894 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367429dd-18f8-4f4c-9129-c502e8003bd4 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:47.656419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 118/234] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:17:47 2022] PUT /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0/users/a651e8aad31c4414842d66f96e07187d/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:17:48.230849 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb30026-5bef-4be1-a4d0-b8e3111b6f75 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:48.231393 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb30026-5bef-4be1-a4d0-b8e3111b6f75 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:48.231393 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb30026-5bef-4be1-a4d0-b8e3111b6f75 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:48.233113 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 117/235] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:48.238305 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4c4e7-2489-4b0a-ad43-8c26a5c332d2 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:48.238775 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4c4e7-2489-4b0a-ad43-8c26a5c332d2 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:48.238858 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4c4e7-2489-4b0a-ad43-8c26a5c332d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:48.267301 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-c2d4c4e7-2489-4b0a-ad43-8c26a5c332d2 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:48.268529 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-c2d4c4e7-2489-4b0a-ad43-8c26a5c332d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:48.340175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 119/236] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:48.374693 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501c969-6bd3-47c9-a9db-af727016f8b2 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:48.375045 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501c969-6bd3-47c9-a9db-af727016f8b2 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:48.375262 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501c969-6bd3-47c9-a9db-af727016f8b2 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:48.376856 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 118/237] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:48.378973 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4c4e7-2489-4b0a-ad43-8c26a5c332d2 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:48.383122 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eedad-344a-47ff-b393-f2e37f37aa15 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:48.383409 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eedad-344a-47ff-b393-f2e37f37aa15 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:48.383688 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eedad-344a-47ff-b393-f2e37f37aa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:48.415162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-db3eedad-344a-47ff-b393-f2e37f37aa15 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:48.416835 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-db3eedad-344a-47ff-b393-f2e37f37aa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:48.448661 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:48.511946 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4c4e7-2489-4b0a-ad43-8c26a5c332d2 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:48.556506 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93e5fc-720c-4399-9168-e69e02ad4a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:48.558391 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93e5fc-720c-4399-9168-e69e02ad4a4a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:48.558951 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93e5fc-720c-4399-9168-e69e02ad4a4a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:48.559408 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93e5fc-720c-4399-9168-e69e02ad4a4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:48.560807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93e5fc-720c-4399-9168-e69e02ad4a4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:48.562470 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93e5fc-720c-4399-9168-e69e02ad4a4a None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:48.564583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 120/238] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:48.570663 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93e5fc-720c-4399-9168-e69e02ad4a4a None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:48.607648 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 119/239] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:17:48.643374 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eedad-344a-47ff-b393-f2e37f37aa15 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:48.681107 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93e5fc-720c-4399-9168-e69e02ad4a4a None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:48.715109 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:48.726933 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138c319-e7dc-478a-9c53-e734bfdd30e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:48.728108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138c319-e7dc-478a-9c53-e734bfdd30e1 None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:48.728893 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138c319-e7dc-478a-9c53-e734bfdd30e1 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:48.729562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138c319-e7dc-478a-9c53-e734bfdd30e1 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:48.730706 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138c319-e7dc-478a-9c53-e734bfdd30e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:48.732586 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138c319-e7dc-478a-9c53-e734bfdd30e1 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:48.765020 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 121/240] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:17:48 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:17:48.909480 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eedad-344a-47ff-b393-f2e37f37aa15 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:48.952004 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:48.962825 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e187ed-044c-4bed-af9f-6fbe89035965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:48.963854 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e187ed-044c-4bed-af9f-6fbe89035965 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:48.964435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e187ed-044c-4bed-af9f-6fbe89035965 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:48.964720 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e187ed-044c-4bed-af9f-6fbe89035965 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:48.985749 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e187ed-044c-4bed-af9f-6fbe89035965 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-d8e187ed-044c-4bed-af9f-6fbe89035965 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:48.993610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:48.995219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:48.996143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:48.996143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:48.996143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:48.996143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:48.996143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:48.996143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:48.996143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:48.996143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:48.996143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:48.996143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:48.996143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:17:48.996143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:48.996143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 120/241] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:49.001406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138c319-e7dc-478a-9c53-e734bfdd30e1 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:49.138649 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138c319-e7dc-478a-9c53-e734bfdd30e1 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:49.157129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f553a2f-a563-4eee-88b0-209ad757be25 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:49.157493 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f553a2f-a563-4eee-88b0-209ad757be25 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:49.157686 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f553a2f-a563-4eee-88b0-209ad757be25 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:49.159482 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 121/242] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:17:49.166824 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d64314-70e2-4fb3-a736-1f0a257a1c09 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:49.167499 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d64314-70e2-4fb3-a736-1f0a257a1c09 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:49.167991 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d64314-70e2-4fb3-a736-1f0a257a1c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:49.183979 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c024a-62e6-4748-a781-c4dd45a9a18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:49.185179 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c024a-62e6-4748-a781-c4dd45a9a18d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:49.185691 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c024a-62e6-4748-a781-c4dd45a9a18d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:49.186184 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c024a-62e6-4748-a781-c4dd45a9a18d None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:49.187971 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c024a-62e6-4748-a781-c4dd45a9a18d None admin] RBAC: Adding query filter params (name=reader) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:49.187971 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c024a-62e6-4748-a781-c4dd45a9a18d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:49.189355 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c024a-62e6-4748-a781-c4dd45a9a18d None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:49.190648 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-c5d64314-70e2-4fb3-a736-1f0a257a1c09 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:49.192645 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-c5d64314-70e2-4fb3-a736-1f0a257a1c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:49.204942 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 122/243] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:49.211198 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c024a-62e6-4748-a781-c4dd45a9a18d None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:49.258321 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:49.285970 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 122/244] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:17:49.309464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d64314-70e2-4fb3-a736-1f0a257a1c09 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:49.335953 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c024a-62e6-4748-a781-c4dd45a9a18d None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:49.376900 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d5467-cfd6-4ec4-a204-df921bd1484c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:49.378962 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d5467-cfd6-4ec4-a204-df921bd1484c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:49.378962 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d5467-cfd6-4ec4-a204-df921bd1484c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:49.379364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d5467-cfd6-4ec4-a204-df921bd1484c None admin] PATH_INFO: `/v3/users/0bb16cb2fac14bba8cfd5a2b4905e381` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:49.396555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d5467-cfd6-4ec4-a204-df921bd1484c None admin] RBAC: Authorizing `identity:get_user(user_id=0bb16cb2fac14bba8cfd5a2b4905e381)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:49.398429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d5467-cfd6-4ec4-a204-df921bd1484c None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:49.401855 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 123/245] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:49 2022] GET /identity/v3/users/0bb16cb2fac14bba8cfd5a2b4905e381 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:49.408192 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d5467-cfd6-4ec4-a204-df921bd1484c None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:49.450495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d64314-70e2-4fb3-a736-1f0a257a1c09 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:49.477043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:49.511752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5868e3a-4e98-4f6a-aa0e-1d6829c0f7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:49.513633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5868e3a-4e98-4f6a-aa0e-1d6829c0f7d1 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:49.514409 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5868e3a-4e98-4f6a-aa0e-1d6829c0f7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:49.514891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5868e3a-4e98-4f6a-aa0e-1d6829c0f7d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:49.532010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5868e3a-4e98-4f6a-aa0e-1d6829c0f7d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-e5868e3a-4e98-4f6a-aa0e-1d6829c0f7d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:49.542863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:49.543916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:49.544865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:49.544865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:49.544865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:49.544865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:49.544865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:49.544865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:49.544865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:49.544865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:49.544865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:49.544865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:49.544865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:49.544865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:49.547957 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 123/246] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:17:49.553388 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5868e3a-4e98-4f6a-aa0e-1d6829c0f7d1 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:49.556023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:49.572908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d5467-cfd6-4ec4-a204-df921bd1484c None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:49.612856 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fc72f-fa93-4b46-9d2a-fc1a674ca007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:49.613600 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fc72f-fa93-4b46-9d2a-fc1a674ca007 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:49.613976 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fc72f-fa93-4b46-9d2a-fc1a674ca007 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:49.614399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fc72f-fa93-4b46-9d2a-fc1a674ca007 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:49.667894 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5868e3a-4e98-4f6a-aa0e-1d6829c0f7d1 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:49.690321 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fc72f-fa93-4b46-9d2a-fc1a674ca007 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:49.692099 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fc72f-fa93-4b46-9d2a-fc1a674ca007 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:49.704953 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5a098-5d77-4b67-9069-2e96b655cfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:49.705918 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5a098-5d77-4b67-9069-2e96b655cfea None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:49.706421 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5a098-5d77-4b67-9069-2e96b655cfea None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:49.706909 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5a098-5d77-4b67-9069-2e96b655cfea None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:49.707908 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5a098-5d77-4b67-9069-2e96b655cfea None admin] RBAC: Adding query filter params (name=member) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:49.708444 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5a098-5d77-4b67-9069-2e96b655cfea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:49.710062 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5a098-5d77-4b67-9069-2e96b655cfea None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:49.719959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 124/247] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:49.728771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5a098-5d77-4b67-9069-2e96b655cfea None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:49.812555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:49.815897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:49.832869 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 124/248] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:49.839788 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fc72f-fa93-4b46-9d2a-fc1a674ca007 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:49.933505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5a098-5d77-4b67-9069-2e96b655cfea None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:49.998615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fc72f-fa93-4b46-9d2a-fc1a674ca007 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:50.007713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18959af-95af-4b3e-9e17-d6319cd454fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:50.008635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18959af-95af-4b3e-9e17-d6319cd454fb None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:50.008922 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18959af-95af-4b3e-9e17-d6319cd454fb None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:50.009224 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18959af-95af-4b3e-9e17-d6319cd454fb None admin] PATH_INFO: `/v3/users/fe3cbf808fa14fcebbf1cff8d2b2cf29` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:50.015020 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18959af-95af-4b3e-9e17-d6319cd454fb None admin] RBAC: Authorizing `identity:get_user(user_id=fe3cbf808fa14fcebbf1cff8d2b2cf29)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:50.016631 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18959af-95af-4b3e-9e17-d6319cd454fb None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:50.018367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 125/249] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:49 2022] GET /identity/v3/users/fe3cbf808fa14fcebbf1cff8d2b2cf29 => generated 297 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:17:50.027758 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18959af-95af-4b3e-9e17-d6319cd454fb None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:50.060554 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea992d-25c3-449d-8b70-1a26285fa347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:50.061450 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea992d-25c3-449d-8b70-1a26285fa347 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:50.061731 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea992d-25c3-449d-8b70-1a26285fa347 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:50.062003 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea992d-25c3-449d-8b70-1a26285fa347 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:50.067019 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea992d-25c3-449d-8b70-1a26285fa347 None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:50.068654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea992d-25c3-449d-8b70-1a26285fa347 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:50.070788 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 125/250] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:49 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:50.076693 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea992d-25c3-449d-8b70-1a26285fa347 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:50.170667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:50.238146 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18959af-95af-4b3e-9e17-d6319cd454fb None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:50.253205 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea992d-25c3-449d-8b70-1a26285fa347 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:50.254656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:50.290691 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1c3f5-6bca-4111-8b12-47a140b72f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:50.291412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1c3f5-6bca-4111-8b12-47a140b72f31 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:50.291644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1c3f5-6bca-4111-8b12-47a140b72f31 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:50.291918 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1c3f5-6bca-4111-8b12-47a140b72f31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:50.292757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1c3f5-6bca-4111-8b12-47a140b72f31 None admin] RBAC: Adding query filter params (role.id=086ec02a15114cc3b858b231d55e19aa, scope.project.id=3022d4bdd09249328ba6a2f60404faf0, user.id=0bb16cb2fac14bba8cfd5a2b4905e381) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:50.293067 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1c3f5-6bca-4111-8b12-47a140b72f31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:50.293193 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636ad0b-59c9-4f2a-ba88-54509b3c2c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:50.294039 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636ad0b-59c9-4f2a-ba88-54509b3c2c6c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:50.294352 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636ad0b-59c9-4f2a-ba88-54509b3c2c6c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:50.294570 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1c3f5-6bca-4111-8b12-47a140b72f31 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:50.294691 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636ad0b-59c9-4f2a-ba88-54509b3c2c6c None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:50.295827 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636ad0b-59c9-4f2a-ba88-54509b3c2c6c None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:50.297280 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636ad0b-59c9-4f2a-ba88-54509b3c2c6c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:50.300692 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 126/251] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:50 2022] GET /identity/v3/role_assignments?user.id=0bb16cb2fac14bba8cfd5a2b4905e381&scope.project.id=3022d4bdd09249328ba6a2f60404faf0&role.id=086ec02a15114cc3b858b231d55e19aa => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:17:50.307286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 126/252] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:50 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:17:50.313609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1c3f5-6bca-4111-8b12-47a140b72f31 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:50.469235 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1c3f5-6bca-4111-8b12-47a140b72f31 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:50.503541 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:50.519062 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb624fa9-4af8-4423-9883-0f733a115d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:50.520026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb624fa9-4af8-4423-9883-0f733a115d33 None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:50.520301 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb624fa9-4af8-4423-9883-0f733a115d33 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:50.520572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb624fa9-4af8-4423-9883-0f733a115d33 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0/users/fe3cbf808fa14fcebbf1cff8d2b2cf29/roles/666cfd1181184612ba462b8eefcad6a7` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:50.522745 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb624fa9-4af8-4423-9883-0f733a115d33 None admin] RBAC: Authorizing `identity:create_grant(project_id=3022d4bdd09249328ba6a2f60404faf0, user_id=fe3cbf808fa14fcebbf1cff8d2b2cf29, role_id=666cfd1181184612ba462b8eefcad6a7)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:50.524285 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb624fa9-4af8-4423-9883-0f733a115d33 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:50.550081 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 127/253] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:17:50 2022] PUT /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0/users/fe3cbf808fa14fcebbf1cff8d2b2cf29/roles/666cfd1181184612ba462b8eefcad6a7 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:17:50.631936 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1e22d-8dfd-4c19-99fb-5bd1b0b7c4d2 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:50.632287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1e22d-8dfd-4c19-99fb-5bd1b0b7c4d2 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:50.632488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1e22d-8dfd-4c19-99fb-5bd1b0b7c4d2 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:50.634425 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 127/254] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:50.640548 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3cbb9-c770-4e17-a9fb-7d525269f79f None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:50.640809 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3cbb9-c770-4e17-a9fb-7d525269f79f None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:50.641098 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3cbb9-c770-4e17-a9fb-7d525269f79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:50.645767 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9c7c1-85d0-449b-9e84-d7bc8ba7fe53 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:50.646390 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9c7c1-85d0-449b-9e84-d7bc8ba7fe53 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:50.646921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9c7c1-85d0-449b-9e84-d7bc8ba7fe53 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:50.648559 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 128/255] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:17:50.660309 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a512b-b658-435f-8e8e-e925f89e72da None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:50.661182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a512b-b658-435f-8e8e-e925f89e72da None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:50.661809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a512b-b658-435f-8e8e-e925f89e72da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:50.665620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-b5d3cbb9-c770-4e17-a9fb-7d525269f79f None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:50.666924 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-b5d3cbb9-c770-4e17-a9fb-7d525269f79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:50.696741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-551a512b-b658-435f-8e8e-e925f89e72da None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:50.700037 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-551a512b-b658-435f-8e8e-e925f89e72da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:50.805986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 128/256] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:50.827665 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 129/257] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:50.854951 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3cbb9-c770-4e17-a9fb-7d525269f79f None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:50.858085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a512b-b658-435f-8e8e-e925f89e72da None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:51.069211 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a512b-b658-435f-8e8e-e925f89e72da None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:51.078747 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:51.101851 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3cbb9-c770-4e17-a9fb-7d525269f79f None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:51.123944 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:51.136376 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67e7f0-b616-4b7e-9429-bda62f3f2749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:51.137264 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67e7f0-b616-4b7e-9429-bda62f3f2749 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:51.137548 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67e7f0-b616-4b7e-9429-bda62f3f2749 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:51.137832 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67e7f0-b616-4b7e-9429-bda62f3f2749 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:51.160035 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67e7f0-b616-4b7e-9429-bda62f3f2749 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-bb67e7f0-b616-4b7e-9429-bda62f3f2749 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:51.169185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:51.170397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:51.171355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:51.171355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:51.171355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:51.171355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:51.171355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:51.171355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:51.171355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:51.171355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:51.171355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:51.171355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:51.171355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:17:51.171355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:51.178903 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 130/258] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:17:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:17:51.179822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1a972-b393-4986-8676-6b23c43fd1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:51.180886 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1a972-b393-4986-8676-6b23c43fd1fd None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:51.181616 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1a972-b393-4986-8676-6b23c43fd1fd None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:51.182207 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1a972-b393-4986-8676-6b23c43fd1fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:51.184969 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67e7f0-b616-4b7e-9429-bda62f3f2749 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:51.203371 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1a972-b393-4986-8676-6b23c43fd1fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-b3f1a972-b393-4986-8676-6b23c43fd1fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:51.211926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:51.213082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:51.214027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:51.214027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:51.214027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:51.214027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:51.214027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:51.214027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:51.214027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:51.214027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:51.214027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:51.214027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:51.214027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:17:51.214027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:51.223879 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 129/259] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:17:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:51.229760 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1a972-b393-4986-8676-6b23c43fd1fd None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:51.439963 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1a972-b393-4986-8676-6b23c43fd1fd None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:51.484418 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67e7f0-b616-4b7e-9429-bda62f3f2749 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:51.505565 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb64ba-f163-499c-8496-bf7ecf0aa424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:51.506248 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb64ba-f163-499c-8496-bf7ecf0aa424 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:51.506563 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb64ba-f163-499c-8496-bf7ecf0aa424 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:51.506839 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb64ba-f163-499c-8496-bf7ecf0aa424 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:51.507605 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb64ba-f163-499c-8496-bf7ecf0aa424 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:51.507920 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb64ba-f163-499c-8496-bf7ecf0aa424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:51.509420 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb64ba-f163-499c-8496-bf7ecf0aa424 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:51.519182 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 130/260] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:17:51.526567 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb64ba-f163-499c-8496-bf7ecf0aa424 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:51.561084 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22f734-bdb8-413c-a3a0-926bd89ffdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:51.561791 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22f734-bdb8-413c-a3a0-926bd89ffdf5 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:51.562078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22f734-bdb8-413c-a3a0-926bd89ffdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:51.562383 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22f734-bdb8-413c-a3a0-926bd89ffdf5 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:51.563156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22f734-bdb8-413c-a3a0-926bd89ffdf5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:51.563466 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22f734-bdb8-413c-a3a0-926bd89ffdf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:51.564880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22f734-bdb8-413c-a3a0-926bd89ffdf5 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:51.586757 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 131/261] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:17:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:51.590744 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22f734-bdb8-413c-a3a0-926bd89ffdf5 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:51.596196 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:51.639616 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:51.732542 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb64ba-f163-499c-8496-bf7ecf0aa424 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:51.765092 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22f734-bdb8-413c-a3a0-926bd89ffdf5 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:51.829626 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0493f8-9cb5-470d-b2e7-63212dc7ddbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:51.830555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0493f8-9cb5-470d-b2e7-63212dc7ddbb None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:51.830834 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0493f8-9cb5-470d-b2e7-63212dc7ddbb None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:51.831104 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0493f8-9cb5-470d-b2e7-63212dc7ddbb None admin] PATH_INFO: `/v3/users/a651e8aad31c4414842d66f96e07187d` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:51.831377 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627b88e-dd66-4aff-8aa9-beae63c5ef48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:51.832510 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627b88e-dd66-4aff-8aa9-beae63c5ef48 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:51.833001 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627b88e-dd66-4aff-8aa9-beae63c5ef48 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:51.833480 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627b88e-dd66-4aff-8aa9-beae63c5ef48 None admin] PATH_INFO: `/v3/users/a651e8aad31c4414842d66f96e07187d` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:51.834995 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0493f8-9cb5-470d-b2e7-63212dc7ddbb None admin] RBAC: Authorizing `identity:get_user(user_id=a651e8aad31c4414842d66f96e07187d)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:51.835853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a627b88e-dd66-4aff-8aa9-beae63c5ef48 None admin] RBAC: Authorizing `identity:get_user(user_id=a651e8aad31c4414842d66f96e07187d)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:51.836537 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0493f8-9cb5-470d-b2e7-63212dc7ddbb None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:51.837843 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a627b88e-dd66-4aff-8aa9-beae63c5ef48 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:51.838626 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 132/262] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:51 2022] GET /identity/v3/users/a651e8aad31c4414842d66f96e07187d => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:51.840530 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 131/263] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:51 2022] GET /identity/v3/users/a651e8aad31c4414842d66f96e07187d => generated 283 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:51.844729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0493f8-9cb5-470d-b2e7-63212dc7ddbb None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:51.845813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627b88e-dd66-4aff-8aa9-beae63c5ef48 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:52.043432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:52.048037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627b88e-dd66-4aff-8aa9-beae63c5ef48 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:52.053379 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0493f8-9cb5-470d-b2e7-63212dc7ddbb None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:52.055446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:52.127691 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9f490-1f76-40e2-8005-5897773089ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:52.128731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9f490-1f76-40e2-8005-5897773089ab None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:52.129190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9f490-1f76-40e2-8005-5897773089ab None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:52.129628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9f490-1f76-40e2-8005-5897773089ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:52.133218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0acba1-0d9b-43f3-89f1-4b2c465fcbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:52.133997 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0acba1-0d9b-43f3-89f1-4b2c465fcbf5 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:52.134550 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0acba1-0d9b-43f3-89f1-4b2c465fcbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:52.134927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0acba1-0d9b-43f3-89f1-4b2c465fcbf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:52.250835 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9f490-1f76-40e2-8005-5897773089ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:52.253286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9f490-1f76-40e2-8005-5897773089ab None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:52.303619 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0acba1-0d9b-43f3-89f1-4b2c465fcbf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:52.303619 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0acba1-0d9b-43f3-89f1-4b2c465fcbf5 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:52.434642 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:52.449234 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 133/264] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:52.450712 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3673b-bd0b-48c5-bef4-cbeb1bea2d34 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:52.451075 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3673b-bd0b-48c5-bef4-cbeb1bea2d34 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:52.452017 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3673b-bd0b-48c5-bef4-cbeb1bea2d34 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:52.453838 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 134/265] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:52.459118 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3673b-bd0b-48c5-bef4-cbeb1bea2d34 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:52.496459 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 132/266] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:52.497878 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad4288-8f49-4cf1-ac94-04067dfac39b None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:52.498166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad4288-8f49-4cf1-ac94-04067dfac39b None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:52.498471 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad4288-8f49-4cf1-ac94-04067dfac39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:52.511269 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:52.519239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-85ad4288-8f49-4cf1-ac94-04067dfac39b None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:52.520449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-85ad4288-8f49-4cf1-ac94-04067dfac39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:52.637079 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 133/267] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:52.637861 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad4288-8f49-4cf1-ac94-04067dfac39b None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:52.658938 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3673b-bd0b-48c5-bef4-cbeb1bea2d34 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:52.731017 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5ca8d-803f-4715-9af6-5b8e64d43c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:52.731909 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5ca8d-803f-4715-9af6-5b8e64d43c47 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:52.732192 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5ca8d-803f-4715-9af6-5b8e64d43c47 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:52.732470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5ca8d-803f-4715-9af6-5b8e64d43c47 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:52.733724 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5ca8d-803f-4715-9af6-5b8e64d43c47 None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:52.735212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5ca8d-803f-4715-9af6-5b8e64d43c47 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:52.737244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 135/268] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:52 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:52.738024 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5ca8d-803f-4715-9af6-5b8e64d43c47 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:52.743170 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad4288-8f49-4cf1-ac94-04067dfac39b None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:52.779642 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2998a9-c24b-4e23-9c8b-decf92b18f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:52.780499 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2998a9-c24b-4e23-9c8b-decf92b18f8a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:52.780782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2998a9-c24b-4e23-9c8b-decf92b18f8a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:52.781063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2998a9-c24b-4e23-9c8b-decf92b18f8a None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:52.782331 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2998a9-c24b-4e23-9c8b-decf92b18f8a None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:52.783891 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2998a9-c24b-4e23-9c8b-decf92b18f8a None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:52.785926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 134/269] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:52 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:52.787150 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c2259-327e-4766-b5b9-643688d01e19 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:52.787438 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c2259-327e-4766-b5b9-643688d01e19 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:52.787719 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c2259-327e-4766-b5b9-643688d01e19 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:52.794203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 135/270] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:52.798569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c2259-327e-4766-b5b9-643688d01e19 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:52.800949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:52.814677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:52.850606 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5ca8d-803f-4715-9af6-5b8e64d43c47 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:52.888988 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dff6ed-e560-4e43-b763-c9591791f11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:52.890474 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dff6ed-e560-4e43-b763-c9591791f11f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:52.890981 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dff6ed-e560-4e43-b763-c9591791f11f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:52.891463 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dff6ed-e560-4e43-b763-c9591791f11f None admin] PATH_INFO: `/v3/roles/member` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:52.903366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dff6ed-e560-4e43-b763-c9591791f11f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:52.910170 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c2259-327e-4766-b5b9-643688d01e19 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-03dff6ed-e560-4e43-b763-c9591791f11f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:52.912546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:52.913551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:52.914635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:52.914635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:52.914635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:52.914635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:52.914635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:52.914635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:52.914635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:52.914635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:52.914635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:52.914635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:52.914635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:52.914635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:52.917652 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 136/271] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:52.919029 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dff6ed-e560-4e43-b763-c9591791f11f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:52.949269 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752e0f1-2506-46f7-806d-2140ef504321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:52.949949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752e0f1-2506-46f7-806d-2140ef504321 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:52.950237 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752e0f1-2506-46f7-806d-2140ef504321 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:52.950548 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752e0f1-2506-46f7-806d-2140ef504321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:52.951411 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752e0f1-2506-46f7-806d-2140ef504321 None admin] RBAC: Adding query filter params (role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2, scope.project.id=3022d4bdd09249328ba6a2f60404faf0, user.id=a651e8aad31c4414842d66f96e07187d) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:52.951913 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752e0f1-2506-46f7-806d-2140ef504321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:52.953266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752e0f1-2506-46f7-806d-2140ef504321 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:52.960213 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 136/272] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:52 2022] GET /identity/v3/role_assignments?user.id=a651e8aad31c4414842d66f96e07187d&scope.project.id=3022d4bdd09249328ba6a2f60404faf0&role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:52.960867 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7cddd-6588-43ba-ab78-a1979a457247 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:52.961174 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7cddd-6588-43ba-ab78-a1979a457247 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:52.961429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7cddd-6588-43ba-ab78-a1979a457247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:52.980534 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-dfc7cddd-6588-43ba-ab78-a1979a457247 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:52.981737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-dfc7cddd-6588-43ba-ab78-a1979a457247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:53.034984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dff6ed-e560-4e43-b763-c9591791f11f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:53.049444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:53.059208 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 137/273] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:53.060108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7cddd-6588-43ba-ab78-a1979a457247 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:53.087878 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:53.100525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb154d-47bd-4964-bee5-54b7badb3b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:53.101611 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb154d-47bd-4964-bee5-54b7badb3b48 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:53.102106 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb154d-47bd-4964-bee5-54b7badb3b48 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:53.105329 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb154d-47bd-4964-bee5-54b7badb3b48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:53.106658 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb154d-47bd-4964-bee5-54b7badb3b48 None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=3022d4bdd09249328ba6a2f60404faf0, user.id=a651e8aad31c4414842d66f96e07187d) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:53.107212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb154d-47bd-4964-bee5-54b7badb3b48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:53.109041 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb154d-47bd-4964-bee5-54b7badb3b48 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:53.119771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 137/274] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:52 2022] GET /identity/v3/role_assignments?user.id=a651e8aad31c4414842d66f96e07187d&scope.project.id=3022d4bdd09249328ba6a2f60404faf0&role.id=b239f941cb724331bf28dc489b70ad1d => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:53.121066 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb154d-47bd-4964-bee5-54b7badb3b48 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:53.202798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7cddd-6588-43ba-ab78-a1979a457247 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:53.238038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb154d-47bd-4964-bee5-54b7badb3b48 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:53.282307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926877a-b44e-4971-ae0d-338b3eefea2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:53.282307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926877a-b44e-4971-ae0d-338b3eefea2b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:53.282307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926877a-b44e-4971-ae0d-338b3eefea2b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:53.282307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926877a-b44e-4971-ae0d-338b3eefea2b None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:53.282809 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926877a-b44e-4971-ae0d-338b3eefea2b None admin] RBAC: Adding query filter params (name=member) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:53.282809 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926877a-b44e-4971-ae0d-338b3eefea2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:53.282809 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926877a-b44e-4971-ae0d-338b3eefea2b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:53.285056 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770954e-e3d5-4188-a885-32a806f48a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:53.285924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770954e-e3d5-4188-a885-32a806f48a5e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:53.286209 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770954e-e3d5-4188-a885-32a806f48a5e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:53.286515 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770954e-e3d5-4188-a885-32a806f48a5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:53.287930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770954e-e3d5-4188-a885-32a806f48a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:53.289557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770954e-e3d5-4188-a885-32a806f48a5e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:53.294070 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 138/275] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:53.299498 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926877a-b44e-4971-ae0d-338b3eefea2b None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:53.299796 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 138/276] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:53.305412 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770954e-e3d5-4188-a885-32a806f48a5e None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:53.371520 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:53.424089 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:53.438109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770954e-e3d5-4188-a885-32a806f48a5e None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:53.476796 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d013bfb-9139-4159-9c51-45e5e2e40ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:53.477413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d013bfb-9139-4159-9c51-45e5e2e40ceb None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:53.477702 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d013bfb-9139-4159-9c51-45e5e2e40ceb None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:53.478006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d013bfb-9139-4159-9c51-45e5e2e40ceb None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:53.478770 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d013bfb-9139-4159-9c51-45e5e2e40ceb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:53.480477 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d013bfb-9139-4159-9c51-45e5e2e40ceb None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:53.505143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 139/277] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:17:53 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:17:53.523957 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926877a-b44e-4971-ae0d-338b3eefea2b None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:53.570221 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2b1a8-8f16-4f36-bda8-b016448d9a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:53.571576 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2b1a8-8f16-4f36-bda8-b016448d9a9d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:53.572114 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2b1a8-8f16-4f36-bda8-b016448d9a9d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:53.572627 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2b1a8-8f16-4f36-bda8-b016448d9a9d None admin] PATH_INFO: `/v3/users/2536e98f4c39445c9902c6895d193ac4` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:53.599688 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2b1a8-8f16-4f36-bda8-b016448d9a9d None admin] RBAC: Authorizing `identity:get_user(user_id=2536e98f4c39445c9902c6895d193ac4)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:53.601844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2b1a8-8f16-4f36-bda8-b016448d9a9d None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:53.605055 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 139/278] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:53 2022] GET /identity/v3/users/2536e98f4c39445c9902c6895d193ac4 => generated 293 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:53.611720 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d013bfb-9139-4159-9c51-45e5e2e40ceb None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:53.749505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d013bfb-9139-4159-9c51-45e5e2e40ceb None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:53.766798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:53.795376 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6dac7-4552-45e3-8b71-0e4bfe1bffc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:53.796064 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6dac7-4552-45e3-8b71-0e4bfe1bffc6 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:53.796348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6dac7-4552-45e3-8b71-0e4bfe1bffc6 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:53.796626 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6dac7-4552-45e3-8b71-0e4bfe1bffc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:53.797469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6dac7-4552-45e3-8b71-0e4bfe1bffc6 None admin] RBAC: Adding query filter params (role.id=666cfd1181184612ba462b8eefcad6a7, user.id=2536e98f4c39445c9902c6895d193ac4, scope.system=all) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:53.797786 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6dac7-4552-45e3-8b71-0e4bfe1bffc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:53.799269 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6dac7-4552-45e3-8b71-0e4bfe1bffc6 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:53.806285 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 140/279] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 20:17:53 2022] GET /identity/v3/role_assignments?user.id=2536e98f4c39445c9902c6895d193ac4&scope.system=all&role.id=666cfd1181184612ba462b8eefcad6a7 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:53.821280 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a1d82-eb0c-440b-aa8e-c14bf1fad27a None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:53.821649 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a1d82-eb0c-440b-aa8e-c14bf1fad27a None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:53.821924 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a1d82-eb0c-440b-aa8e-c14bf1fad27a None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:53.822661 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e825f0-5d22-4381-9a68-68bbef17eeac None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:53.822971 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e825f0-5d22-4381-9a68-68bbef17eeac None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:53.823242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e825f0-5d22-4381-9a68-68bbef17eeac None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:53.824099 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 140/280] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:53.824938 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 141/281] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:17:53.830920 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765e833-ed09-4763-8eb7-e3d86e9225af None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:53.831127 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b4943-121e-4b87-b5b7-266b7eb9cce2 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:53.831216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765e833-ed09-4763-8eb7-e3d86e9225af None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:53.831348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b4943-121e-4b87-b5b7-266b7eb9cce2 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:53.831600 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765e833-ed09-4763-8eb7-e3d86e9225af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:53.831702 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b4943-121e-4b87-b5b7-266b7eb9cce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:53.850682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-762b4943-121e-4b87-b5b7-266b7eb9cce2 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:53.852287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-762b4943-121e-4b87-b5b7-266b7eb9cce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:53.854772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-8765e833-ed09-4763-8eb7-e3d86e9225af None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:53.857443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-8765e833-ed09-4763-8eb7-e3d86e9225af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:53.964871 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 141/282] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:53.968756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 142/283] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:17:53.990165 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b4943-121e-4b87-b5b7-266b7eb9cce2 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:53.991064 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765e833-ed09-4763-8eb7-e3d86e9225af None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:54.066649 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:54.113765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765e833-ed09-4763-8eb7-e3d86e9225af None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:54.122718 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:54.124960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b4943-121e-4b87-b5b7-266b7eb9cce2 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:54.155326 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c637ff7-a28a-4cc0-acf4-7e2f44e150e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:54.156520 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c637ff7-a28a-4cc0-acf4-7e2f44e150e9 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:54.157002 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c637ff7-a28a-4cc0-acf4-7e2f44e150e9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:54.157517 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c637ff7-a28a-4cc0-acf4-7e2f44e150e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:54.168757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136acde-242d-4d66-b233-fca4e09f4c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:54.169709 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136acde-242d-4d66-b233-fca4e09f4c7d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:54.170089 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136acde-242d-4d66-b233-fca4e09f4c7d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:54.170481 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136acde-242d-4d66-b233-fca4e09f4c7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:54.172090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c637ff7-a28a-4cc0-acf4-7e2f44e150e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-6c637ff7-a28a-4cc0-acf4-7e2f44e150e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:54.180677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:54.182162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:54.183136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:54.183136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:54.183136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:54.183136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:54.183136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:54.183136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:54.183136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:54.183136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:54.183136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:54.183136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:54.183136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:17:54.183136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:54.183863 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136acde-242d-4d66-b233-fca4e09f4c7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:54.186052 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 143/284] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-4136acde-242d-4d66-b233-fca4e09f4c7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:54.194323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:54.195398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:54.196432 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:54.196432 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:54.196432 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:54.196432 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:54.196432 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:54.196432 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:54.196432 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:54.196432 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:54.196432 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:54.196432 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:54.196432 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:54.196432 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:54.196432 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 142/285] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:54.199028 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136acde-242d-4d66-b233-fca4e09f4c7d None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:54.199275 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c637ff7-a28a-4cc0-acf4-7e2f44e150e9 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:54.326707 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136acde-242d-4d66-b233-fca4e09f4c7d None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:54.340341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:54.389890 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4fd7b-1c2c-4f31-8b5b-cd70c1a8b826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:54.390595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4fd7b-1c2c-4f31-8b5b-cd70c1a8b826 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:54.390876 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4fd7b-1c2c-4f31-8b5b-cd70c1a8b826 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:54.391305 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4fd7b-1c2c-4f31-8b5b-cd70c1a8b826 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:54.392221 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4fd7b-1c2c-4f31-8b5b-cd70c1a8b826 None admin] RBAC: Adding query filter params (name=member) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:54.392533 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4fd7b-1c2c-4f31-8b5b-cd70c1a8b826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:54.394104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4fd7b-1c2c-4f31-8b5b-cd70c1a8b826 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:54.403765 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 143/286] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:54.412173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4fd7b-1c2c-4f31-8b5b-cd70c1a8b826 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:54.429732 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c637ff7-a28a-4cc0-acf4-7e2f44e150e9 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:54.491092 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6273ccb-0f58-4b2b-a57b-bbc101514ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:54.492286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6273ccb-0f58-4b2b-a57b-bbc101514ae9 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:54.492778 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6273ccb-0f58-4b2b-a57b-bbc101514ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:54.493280 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6273ccb-0f58-4b2b-a57b-bbc101514ae9 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:54.494859 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6273ccb-0f58-4b2b-a57b-bbc101514ae9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:54.495424 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6273ccb-0f58-4b2b-a57b-bbc101514ae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:54.497066 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6273ccb-0f58-4b2b-a57b-bbc101514ae9 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:54.513648 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 144/287] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:54.520070 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6273ccb-0f58-4b2b-a57b-bbc101514ae9 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:54.593941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4fd7b-1c2c-4f31-8b5b-cd70c1a8b826 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:54.604036 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:54.641607 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a64fd5-5217-46aa-8193-330912252ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:54.642963 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a64fd5-5217-46aa-8193-330912252ddf None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:54.644495 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a64fd5-5217-46aa-8193-330912252ddf None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:54.645003 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a64fd5-5217-46aa-8193-330912252ddf None admin] PATH_INFO: `/v3/users/fe3cbf808fa14fcebbf1cff8d2b2cf29` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:54.647978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a64fd5-5217-46aa-8193-330912252ddf None admin] RBAC: Authorizing `identity:get_user(user_id=fe3cbf808fa14fcebbf1cff8d2b2cf29)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:54.651744 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a64fd5-5217-46aa-8193-330912252ddf None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:54.657768 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 144/288] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:54 2022] GET /identity/v3/users/fe3cbf808fa14fcebbf1cff8d2b2cf29 => generated 297 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:17:54.662936 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a64fd5-5217-46aa-8193-330912252ddf None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:54.741246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:54.773126 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6273ccb-0f58-4b2b-a57b-bbc101514ae9 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:54.824383 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a64fd5-5217-46aa-8193-330912252ddf None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:54.864879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e72c2-1148-40f9-9d80-6a70db7811b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:54.866580 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e72c2-1148-40f9-9d80-6a70db7811b1 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:54.867383 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e72c2-1148-40f9-9d80-6a70db7811b1 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:54.868778 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7c663-f5d5-44dd-8649-0cae2d73d3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:54.869578 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e72c2-1148-40f9-9d80-6a70db7811b1 None admin] PATH_INFO: `/v3/users/0bb16cb2fac14bba8cfd5a2b4905e381` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:54.871090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7c663-f5d5-44dd-8649-0cae2d73d3ac None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:54.871602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7c663-f5d5-44dd-8649-0cae2d73d3ac None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:54.872454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e72c2-1148-40f9-9d80-6a70db7811b1 None admin] RBAC: Authorizing `identity:get_user(user_id=0bb16cb2fac14bba8cfd5a2b4905e381)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:54.872960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7c663-f5d5-44dd-8649-0cae2d73d3ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:54.875261 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e72c2-1148-40f9-9d80-6a70db7811b1 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:54.877493 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 145/289] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:54 2022] GET /identity/v3/users/0bb16cb2fac14bba8cfd5a2b4905e381 => generated 297 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:54.883225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e72c2-1148-40f9-9d80-6a70db7811b1 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:54.967351 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7c663-f5d5-44dd-8649-0cae2d73d3ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:54.968864 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7c663-f5d5-44dd-8649-0cae2d73d3ac None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:55.013078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:55.021673 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e72c2-1148-40f9-9d80-6a70db7811b1 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:55.065444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:55.067455 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11383ee9-7de7-4be2-81a4-090e61571940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:55.070814 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11383ee9-7de7-4be2-81a4-090e61571940 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:55.070814 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11383ee9-7de7-4be2-81a4-090e61571940 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:55.070814 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11383ee9-7de7-4be2-81a4-090e61571940 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:55.075341 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11383ee9-7de7-4be2-81a4-090e61571940 None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:55.076854 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11383ee9-7de7-4be2-81a4-090e61571940 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:55.081156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 146/290] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:54 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:55.085847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11383ee9-7de7-4be2-81a4-090e61571940 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:55.195473 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 145/291] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:17:55.200732 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7c663-f5d5-44dd-8649-0cae2d73d3ac None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:55.278178 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11383ee9-7de7-4be2-81a4-090e61571940 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:55.347573 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a97148-9979-4fe5-b9ab-6fd0abe49e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:55.349628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a97148-9979-4fe5-b9ab-6fd0abe49e50 None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:55.349924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a97148-9979-4fe5-b9ab-6fd0abe49e50 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:55.350209 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a97148-9979-4fe5-b9ab-6fd0abe49e50 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0/users/0bb16cb2fac14bba8cfd5a2b4905e381/roles/086ec02a15114cc3b858b231d55e19aa` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:55.355231 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a97148-9979-4fe5-b9ab-6fd0abe49e50 None admin] RBAC: Authorizing `identity:create_grant(project_id=3022d4bdd09249328ba6a2f60404faf0, user_id=0bb16cb2fac14bba8cfd5a2b4905e381, role_id=086ec02a15114cc3b858b231d55e19aa)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:55.356703 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a97148-9979-4fe5-b9ab-6fd0abe49e50 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:55.383361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 147/292] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:17:55 2022] PUT /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0/users/0bb16cb2fac14bba8cfd5a2b4905e381/roles/086ec02a15114cc3b858b231d55e19aa => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:17:55.386590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7c663-f5d5-44dd-8649-0cae2d73d3ac None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:55.437249 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6193f1-4c9a-44e1-b7ef-be26751aec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:55.438191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6193f1-4c9a-44e1-b7ef-be26751aec93 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:55.438505 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6193f1-4c9a-44e1-b7ef-be26751aec93 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:55.438784 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6193f1-4c9a-44e1-b7ef-be26751aec93 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:55.440010 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6193f1-4c9a-44e1-b7ef-be26751aec93 None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:55.441532 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6193f1-4c9a-44e1-b7ef-be26751aec93 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:55.443709 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 146/293] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:55 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:55.449919 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a97148-9979-4fe5-b9ab-6fd0abe49e50 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:55.484460 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:55.587804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a97148-9979-4fe5-b9ab-6fd0abe49e50 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:55.639295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace8e19-def4-4cce-89a5-e50e9923c7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:55.639975 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace8e19-def4-4cce-89a5-e50e9923c7aa None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:55.640260 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace8e19-def4-4cce-89a5-e50e9923c7aa None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:55.640538 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace8e19-def4-4cce-89a5-e50e9923c7aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:55.641379 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace8e19-def4-4cce-89a5-e50e9923c7aa None admin] RBAC: Adding query filter params (role.id=666cfd1181184612ba462b8eefcad6a7, scope.project.id=3022d4bdd09249328ba6a2f60404faf0, user.id=fe3cbf808fa14fcebbf1cff8d2b2cf29) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:55.641693 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace8e19-def4-4cce-89a5-e50e9923c7aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:55.643761 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace8e19-def4-4cce-89a5-e50e9923c7aa None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:55.651125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 148/294] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:55 2022] GET /identity/v3/role_assignments?user.id=fe3cbf808fa14fcebbf1cff8d2b2cf29&scope.project.id=3022d4bdd09249328ba6a2f60404faf0&role.id=666cfd1181184612ba462b8eefcad6a7 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:56.300926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739396f-a781-4f4a-85bb-96e133afd863 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:56.301455 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739396f-a781-4f4a-85bb-96e133afd863 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:56.301455 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739396f-a781-4f4a-85bb-96e133afd863 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:56.303097 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 147/295] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:56.308638 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb972cd-3b0c-43f8-b9ac-67e06fdbc78e None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:56.309112 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb972cd-3b0c-43f8-b9ac-67e06fdbc78e None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:56.309198 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb972cd-3b0c-43f8-b9ac-67e06fdbc78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:56.328565 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-2cb972cd-3b0c-43f8-b9ac-67e06fdbc78e None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:56.329748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-2cb972cd-3b0c-43f8-b9ac-67e06fdbc78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:56.366456 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a7d10-e886-440e-9b24-51e6f4ad85ab None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:56.366818 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a7d10-e886-440e-9b24-51e6f4ad85ab None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:56.367016 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a7d10-e886-440e-9b24-51e6f4ad85ab None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:56.368417 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 148/296] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:17:56.374539 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baefcd-2823-4134-b78b-fe06cc86c2cc None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:56.374848 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baefcd-2823-4134-b78b-fe06cc86c2cc None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:56.375107 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baefcd-2823-4134-b78b-fe06cc86c2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:56.381397 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:56.395691 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-e0baefcd-2823-4134-b78b-fe06cc86c2cc None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:56.397184 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-e0baefcd-2823-4134-b78b-fe06cc86c2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:56.413102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 149/297] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:56.435339 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb972cd-3b0c-43f8-b9ac-67e06fdbc78e None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:56.455128 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:56.480109 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 149/298] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:17:56.505029 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0baefcd-2823-4134-b78b-fe06cc86c2cc None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:56.542574 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb972cd-3b0c-43f8-b9ac-67e06fdbc78e None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:56.577998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e16b72-53ca-49c2-a69a-ce9529a81173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:56.578753 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e16b72-53ca-49c2-a69a-ce9529a81173 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:56.579033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e16b72-53ca-49c2-a69a-ce9529a81173 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:56.579308 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e16b72-53ca-49c2-a69a-ce9529a81173 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:56.590785 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e16b72-53ca-49c2-a69a-ce9529a81173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-49e16b72-53ca-49c2-a69a-ce9529a81173 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:56.599073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:56.600094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:56.601047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:56.601047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:56.601047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:56.601047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:56.601047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:56.601047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:56.601047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:56.601047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:56.601047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:56.601047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:56.601047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:56.601047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:56.601047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 150/299] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:17:56.601835 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8aab1-108c-420d-9ece-f9769dd51322 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:56.601907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8aab1-108c-420d-9ece-f9769dd51322 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:56.602127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8aab1-108c-420d-9ece-f9769dd51322 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:56.604110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 151/300] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:56.608016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8aab1-108c-420d-9ece-f9769dd51322 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:56.625756 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0baefcd-2823-4134-b78b-fe06cc86c2cc None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:56.662041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931274ce-8b43-42e4-ad29-9286cd15a123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:56.662855 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931274ce-8b43-42e4-ad29-9286cd15a123 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:56.663138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931274ce-8b43-42e4-ad29-9286cd15a123 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:56.663412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931274ce-8b43-42e4-ad29-9286cd15a123 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:56.665940 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:56.675747 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931274ce-8b43-42e4-ad29-9286cd15a123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-931274ce-8b43-42e4-ad29-9286cd15a123 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:56.683807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:56.684835 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:56.685801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:56.685801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:56.685801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:56.685801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:56.685801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:56.685801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:56.685801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:56.685801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:56.685801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:56.685801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:56.685801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:17:56.685801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:56.685801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 150/301] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:17:56.687282 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2d899-57d0-4e73-9f5c-bcbdb6cae764 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:56.687282 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2d899-57d0-4e73-9f5c-bcbdb6cae764 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:56.687282 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2d899-57d0-4e73-9f5c-bcbdb6cae764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:56.703118 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:56.703978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-40b2d899-57d0-4e73-9f5c-bcbdb6cae764 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:56.705350 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-40b2d899-57d0-4e73-9f5c-bcbdb6cae764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:56.732154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8aab1-108c-420d-9ece-f9769dd51322 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:56.770040 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6eedf7-8532-437c-9fd9-4637fc6bc8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:56.770857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eedf7-8532-437c-9fd9-4637fc6bc8f3 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:56.771145 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eedf7-8532-437c-9fd9-4637fc6bc8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:56.771420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eedf7-8532-437c-9fd9-4637fc6bc8f3 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:56.772389 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6eedf7-8532-437c-9fd9-4637fc6bc8f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:56.772704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6eedf7-8532-437c-9fd9-4637fc6bc8f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:56.774141 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6eedf7-8532-437c-9fd9-4637fc6bc8f3 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:56.776395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 151/302] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:56.777297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2d899-57d0-4e73-9f5c-bcbdb6cae764 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:56.792748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 152/303] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:17:56.794248 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6eedf7-8532-437c-9fd9-4637fc6bc8f3 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:56.891225 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2d899-57d0-4e73-9f5c-bcbdb6cae764 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:56.909892 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6eedf7-8532-437c-9fd9-4637fc6bc8f3 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:56.919482 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:56.932863 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac024868-2cb4-43ff-8711-9e1c95fedb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:56.933995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac024868-2cb4-43ff-8711-9e1c95fedb67 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:56.934305 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac024868-2cb4-43ff-8711-9e1c95fedb67 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:56.934587 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac024868-2cb4-43ff-8711-9e1c95fedb67 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:56.935401 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac024868-2cb4-43ff-8711-9e1c95fedb67 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:56.935715 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac024868-2cb4-43ff-8711-9e1c95fedb67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:56.937133 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac024868-2cb4-43ff-8711-9e1c95fedb67 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:56.945843 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 152/304] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:56.946724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac024868-2cb4-43ff-8711-9e1c95fedb67 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:56.952192 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdde0f-9986-4606-9ecf-8647f5587d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:56.953166 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdde0f-9986-4606-9ecf-8647f5587d86 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:56.953453 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdde0f-9986-4606-9ecf-8647f5587d86 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:56.953730 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdde0f-9986-4606-9ecf-8647f5587d86 None admin] PATH_INFO: `/v3/users/2536e98f4c39445c9902c6895d193ac4` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:56.954991 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdde0f-9986-4606-9ecf-8647f5587d86 None admin] RBAC: Authorizing `identity:get_user(user_id=2536e98f4c39445c9902c6895d193ac4)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:56.956444 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdde0f-9986-4606-9ecf-8647f5587d86 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:56.958227 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 153/305] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:56 2022] GET /identity/v3/users/2536e98f4c39445c9902c6895d193ac4 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:56.962947 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdde0f-9986-4606-9ecf-8647f5587d86 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:56.975878 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:57.065417 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac024868-2cb4-43ff-8711-9e1c95fedb67 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:57.079166 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdde0f-9986-4606-9ecf-8647f5587d86 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:57.108185 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91009496-521c-4d05-bce0-13afc941e640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:57.108661 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91009496-521c-4d05-bce0-13afc941e640 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:57.108944 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91009496-521c-4d05-bce0-13afc941e640 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:57.109220 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91009496-521c-4d05-bce0-13afc941e640 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:57.110403 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91009496-521c-4d05-bce0-13afc941e640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:57.111875 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91009496-521c-4d05-bce0-13afc941e640 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:57.113501 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 153/306] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:57.114234 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91009496-521c-4d05-bce0-13afc941e640 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:57.128709 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e8095-9ca1-40b0-a23c-626a8c98ab28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:57.130277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e8095-9ca1-40b0-a23c-626a8c98ab28 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:57.131076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e8095-9ca1-40b0-a23c-626a8c98ab28 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:57.131557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e8095-9ca1-40b0-a23c-626a8c98ab28 None admin] PATH_INFO: `/v3/users/f8570fdd261d477eaeeccb94ae4c9192` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:57.150184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e8095-9ca1-40b0-a23c-626a8c98ab28 None admin] RBAC: Authorizing `identity:get_user(user_id=f8570fdd261d477eaeeccb94ae4c9192)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:57.152213 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e8095-9ca1-40b0-a23c-626a8c98ab28 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:57.155336 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 154/307] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:56 2022] GET /identity/v3/users/f8570fdd261d477eaeeccb94ae4c9192 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:57.156398 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e8095-9ca1-40b0-a23c-626a8c98ab28 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:57.174580 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:57.222991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91009496-521c-4d05-bce0-13afc941e640 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:57.233057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:57.262240 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54741e4-0372-47ed-aff8-9429a10f9061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:57.263153 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54741e4-0372-47ed-aff8-9429a10f9061 None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:57.263455 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54741e4-0372-47ed-aff8-9429a10f9061 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:57.263713 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54741e4-0372-47ed-aff8-9429a10f9061 None admin] PATH_INFO: `/v3/system/users/2536e98f4c39445c9902c6895d193ac4/roles/666cfd1181184612ba462b8eefcad6a7` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:57.265258 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54741e4-0372-47ed-aff8-9429a10f9061 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2536e98f4c39445c9902c6895d193ac4, role_id=666cfd1181184612ba462b8eefcad6a7)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:57.266716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54741e4-0372-47ed-aff8-9429a10f9061 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:57.277113 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e8095-9ca1-40b0-a23c-626a8c98ab28 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:57.282584 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 154/308] 10.222.0.84 () {58 vars in 1334 bytes} [Thu May 19 20:17:57 2022] PUT /identity/v3/system/users/2536e98f4c39445c9902c6895d193ac4/roles/666cfd1181184612ba462b8eefcad6a7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:17:57.282584 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54741e4-0372-47ed-aff8-9429a10f9061 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:57.339855 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797caf2-857d-4b3b-b58b-c62745269b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:57.340778 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797caf2-857d-4b3b-b58b-c62745269b1f None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:57.341159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797caf2-857d-4b3b-b58b-c62745269b1f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:57.341536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797caf2-857d-4b3b-b58b-c62745269b1f None admin] PATH_INFO: `/v3/groups` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:57.342383 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797caf2-857d-4b3b-b58b-c62745269b1f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:57.344636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797caf2-857d-4b3b-b58b-c62745269b1f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:57.366699 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 155/309] 10.222.0.84 () {60 vars in 1134 bytes} [Thu May 19 20:17:57 2022] POST /identity/v3/groups => generated 229 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:17:57.436970 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54741e4-0372-47ed-aff8-9429a10f9061 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:57.460133 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:57.483673 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47127659-86b7-41a5-be1b-c36f8cefa0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:57.484544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47127659-86b7-41a5-be1b-c36f8cefa0b9 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:57.484968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47127659-86b7-41a5-be1b-c36f8cefa0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:57.485256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47127659-86b7-41a5-be1b-c36f8cefa0b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:57.486134 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47127659-86b7-41a5-be1b-c36f8cefa0b9 None admin] RBAC: Adding query filter params (role.id=086ec02a15114cc3b858b231d55e19aa, user.id=f8570fdd261d477eaeeccb94ae4c9192, scope.system=all) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:57.486489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47127659-86b7-41a5-be1b-c36f8cefa0b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:57.488123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47127659-86b7-41a5-be1b-c36f8cefa0b9 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:57.495652 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 155/310] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 20:17:57 2022] GET /identity/v3/role_assignments?user.id=f8570fdd261d477eaeeccb94ae4c9192&scope.system=all&role.id=086ec02a15114cc3b858b231d55e19aa => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:57.957678 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e85c9-3fee-49ff-8ec7-6de02f9936c3 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:57.958115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e85c9-3fee-49ff-8ec7-6de02f9936c3 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:57.958323 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e85c9-3fee-49ff-8ec7-6de02f9936c3 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:57.959927 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 156/311] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:57.965850 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5d3bf-f513-4e1b-9b6e-c18d59bd4b7b None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:57.966138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5d3bf-f513-4e1b-9b6e-c18d59bd4b7b None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:57.966463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5d3bf-f513-4e1b-9b6e-c18d59bd4b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:57.984483 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-29b5d3bf-f513-4e1b-9b6e-c18d59bd4b7b None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:57.985677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-29b5d3bf-f513-4e1b-9b6e-c18d59bd4b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:58.057345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 156/312] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:58.080503 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e85c9-3fee-49ff-8ec7-6de02f9936c3 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:58.190782 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e85c9-3fee-49ff-8ec7-6de02f9936c3 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:58.218552 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:58.229148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bf4e4-fd9d-4e9d-a896-e2340235e047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:58.230171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bf4e4-fd9d-4e9d-a896-e2340235e047 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:58.230682 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bf4e4-fd9d-4e9d-a896-e2340235e047 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:58.230962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bf4e4-fd9d-4e9d-a896-e2340235e047 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:58.242044 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8bf4e4-fd9d-4e9d-a896-e2340235e047 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-ad8bf4e4-fd9d-4e9d-a896-e2340235e047 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:58.249956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:58.252287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:58.254483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:58.254483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:58.254483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:58.254483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:58.254483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:58.254483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:58.254483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:58.254483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:58.254483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:58.254483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:58.254483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:17:58.254483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:17:58.256107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 157/313] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:17:58.258636 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5d3bf-f513-4e1b-9b6e-c18d59bd4b7b None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:58.364771 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5d3bf-f513-4e1b-9b6e-c18d59bd4b7b None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:58.378461 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:58.400084 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9100a13-40dc-46b7-8393-6b623188c40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:58.400765 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9100a13-40dc-46b7-8393-6b623188c40d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:58.401047 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9100a13-40dc-46b7-8393-6b623188c40d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:58.401324 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9100a13-40dc-46b7-8393-6b623188c40d None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:58.402082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9100a13-40dc-46b7-8393-6b623188c40d None admin] RBAC: Adding query filter params (name=reader) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:58.402421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9100a13-40dc-46b7-8393-6b623188c40d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:58.403856 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9100a13-40dc-46b7-8393-6b623188c40d None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:58.412864 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 157/314] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:58.423447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bf4e4-fd9d-4e9d-a896-e2340235e047 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:58.541366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bf4e4-fd9d-4e9d-a896-e2340235e047 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:58.588981 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec521c-8cbd-4d49-918d-40c676a90260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:58.590186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec521c-8cbd-4d49-918d-40c676a90260 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:58.590506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec521c-8cbd-4d49-918d-40c676a90260 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:58.590796 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec521c-8cbd-4d49-918d-40c676a90260 None admin] PATH_INFO: `/v3/users/0bb16cb2fac14bba8cfd5a2b4905e381` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:58.592116 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec521c-8cbd-4d49-918d-40c676a90260 None admin] RBAC: Authorizing `identity:get_user(user_id=0bb16cb2fac14bba8cfd5a2b4905e381)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:58.593625 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec521c-8cbd-4d49-918d-40c676a90260 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:58.595746 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 158/315] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:17:58 2022] GET /identity/v3/users/0bb16cb2fac14bba8cfd5a2b4905e381 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:58.601639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9100a13-40dc-46b7-8393-6b623188c40d None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:58.715114 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9100a13-40dc-46b7-8393-6b623188c40d None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:58.749907 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930ee06-8bbf-423a-a2fa-29c20a33d32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:58.750596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930ee06-8bbf-423a-a2fa-29c20a33d32c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:58.750881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930ee06-8bbf-423a-a2fa-29c20a33d32c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:58.751160 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930ee06-8bbf-423a-a2fa-29c20a33d32c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:58.802947 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:58.822586 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930ee06-8bbf-423a-a2fa-29c20a33d32c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:58.823863 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930ee06-8bbf-423a-a2fa-29c20a33d32c None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:58.926073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 158/316] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:17:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:17:58.932890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec521c-8cbd-4d49-918d-40c676a90260 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:58.992456 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:59.039576 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec521c-8cbd-4d49-918d-40c676a90260 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:59.074990 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e847d3-12e4-41d6-a298-ef1df1389ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:59.075785 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e847d3-12e4-41d6-a298-ef1df1389ea7 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:59.076066 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e847d3-12e4-41d6-a298-ef1df1389ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:59.076342 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e847d3-12e4-41d6-a298-ef1df1389ea7 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:59.077551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e847d3-12e4-41d6-a298-ef1df1389ea7 None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:59.079062 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e847d3-12e4-41d6-a298-ef1df1389ea7 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:59.081116 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 159/317] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:17:58 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:59.087369 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930ee06-8bbf-423a-a2fa-29c20a33d32c None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:59.170396 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:59.203723 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930ee06-8bbf-423a-a2fa-29c20a33d32c None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:59.239900 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb09d5-7b21-43a8-95e7-f4aeab9cd9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:59.240569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb09d5-7b21-43a8-95e7-f4aeab9cd9de None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:59.240851 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb09d5-7b21-43a8-95e7-f4aeab9cd9de None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:59.241132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb09d5-7b21-43a8-95e7-f4aeab9cd9de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:59.241970 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb09d5-7b21-43a8-95e7-f4aeab9cd9de None admin] RBAC: Adding query filter params (role.id=086ec02a15114cc3b858b231d55e19aa, scope.project.id=3022d4bdd09249328ba6a2f60404faf0, user.id=0bb16cb2fac14bba8cfd5a2b4905e381) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:17:59.242299 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb09d5-7b21-43a8-95e7-f4aeab9cd9de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:59.243939 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb09d5-7b21-43a8-95e7-f4aeab9cd9de None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:59.250722 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 159/318] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:17:59 2022] GET /identity/v3/role_assignments?user.id=0bb16cb2fac14bba8cfd5a2b4905e381&scope.project.id=3022d4bdd09249328ba6a2f60404faf0&role.id=086ec02a15114cc3b858b231d55e19aa => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:17:59.642062 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb2d9d-5b2b-4e0c-b0e8-a33c7bd1df3b None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:59.642689 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb2d9d-5b2b-4e0c-b0e8-a33c7bd1df3b None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:59.642689 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb2d9d-5b2b-4e0c-b0e8-a33c7bd1df3b None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:59.644235 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 160/319] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:59.649534 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d693fc0-3e7b-4fb3-9b4b-9134cf461f02 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:59.650028 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d693fc0-3e7b-4fb3-9b4b-9134cf461f02 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:59.650119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d693fc0-3e7b-4fb3-9b4b-9134cf461f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:59.667640 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-4d693fc0-3e7b-4fb3-9b4b-9134cf461f02 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:59.668821 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-4d693fc0-3e7b-4fb3-9b4b-9134cf461f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:59.703002 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142ff05-8c58-45f9-8408-4eedf0b728c4 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:59.703241 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142ff05-8c58-45f9-8408-4eedf0b728c4 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:59.703430 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142ff05-8c58-45f9-8408-4eedf0b728c4 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:59.704948 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 161/320] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:17:59.710768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6616d7-dea5-411d-92ae-c2a822c6d00f None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:59.711093 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6616d7-dea5-411d-92ae-c2a822c6d00f None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:59.711326 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6616d7-dea5-411d-92ae-c2a822c6d00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:59.732464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-2c6616d7-dea5-411d-92ae-c2a822c6d00f None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:59.733718 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-2c6616d7-dea5-411d-92ae-c2a822c6d00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:59.749213 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 160/321] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:59.772044 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d693fc0-3e7b-4fb3-9b4b-9134cf461f02 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:59.817809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 162/322] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:17:59.842777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6616d7-dea5-411d-92ae-c2a822c6d00f None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:59.844241 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:59.885574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d693fc0-3e7b-4fb3-9b4b-9134cf461f02 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:59.908715 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:59.926820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da337e51-6463-4513-845b-6fd42b4fae0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:59.927715 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da337e51-6463-4513-845b-6fd42b4fae0c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:59.927971 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da337e51-6463-4513-845b-6fd42b4fae0c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:59.928243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da337e51-6463-4513-845b-6fd42b4fae0c None admin] PATH_INFO: `/v3/roles/member` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:59.939227 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da337e51-6463-4513-845b-6fd42b4fae0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-da337e51-6463-4513-845b-6fd42b4fae0c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:17:59.946813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:17:59.948148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:17:59.954504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:17:59.954504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:17:59.954504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:17:59.954504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:17:59.954504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:17:59.954504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:17:59.954504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:17:59.954504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:17:59.954504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:17:59.954504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:17:59.954504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:17:59.954504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:17:59.954504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 161/323] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:17:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:17:59.954504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3507c19-3a6f-436c-992f-e1521bbb48dd None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:59.956121 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3507c19-3a6f-436c-992f-e1521bbb48dd None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:59.956121 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3507c19-3a6f-436c-992f-e1521bbb48dd None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:59.956121 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 162/324] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:59.956121 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3507c19-3a6f-436c-992f-e1521bbb48dd None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:59.981148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6616d7-dea5-411d-92ae-c2a822c6d00f None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:00.018508 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076ffca-eded-4a21-b06d-db791c1f0372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:00.019314 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076ffca-eded-4a21-b06d-db791c1f0372 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:00.019596 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076ffca-eded-4a21-b06d-db791c1f0372 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:00.019873 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076ffca-eded-4a21-b06d-db791c1f0372 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:00.021052 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076ffca-eded-4a21-b06d-db791c1f0372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:00.022563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076ffca-eded-4a21-b06d-db791c1f0372 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:00.024364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 163/325] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:17:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:00.025666 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bea752-1963-42f3-b89a-9d29adc65da6 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:00.025957 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bea752-1963-42f3-b89a-9d29adc65da6 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:00.026226 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bea752-1963-42f3-b89a-9d29adc65da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:00.042518 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-50bea752-1963-42f3-b89a-9d29adc65da6 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:00.043748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-50bea752-1963-42f3-b89a-9d29adc65da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:00.068968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3507c19-3a6f-436c-992f-e1521bbb48dd None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:00.096614 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:00.105896 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28596a75-8117-45ac-ba23-1a7b82b8d5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:00.106916 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28596a75-8117-45ac-ba23-1a7b82b8d5fa None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:00.107330 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28596a75-8117-45ac-ba23-1a7b82b8d5fa None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:00.107608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28596a75-8117-45ac-ba23-1a7b82b8d5fa None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:00.108388 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28596a75-8117-45ac-ba23-1a7b82b8d5fa None admin] RBAC: Adding query filter params (name=member) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:00.108763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28596a75-8117-45ac-ba23-1a7b82b8d5fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:00.110581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28596a75-8117-45ac-ba23-1a7b82b8d5fa None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:00.113296 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 164/326] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:00.114127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bea752-1963-42f3-b89a-9d29adc65da6 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:00.120262 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 163/327] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:17:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:00.128099 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28596a75-8117-45ac-ba23-1a7b82b8d5fa None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:00.170926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:00.231129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bea752-1963-42f3-b89a-9d29adc65da6 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:00.257302 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28596a75-8117-45ac-ba23-1a7b82b8d5fa None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:00.266583 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9237d2-563b-459f-b925-793f38f356a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:00.267289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9237d2-563b-459f-b925-793f38f356a1 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:00.267537 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9237d2-563b-459f-b925-793f38f356a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:00.267804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9237d2-563b-459f-b925-793f38f356a1 None admin] PATH_INFO: `/v3/groups` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:00.268510 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9237d2-563b-459f-b925-793f38f356a1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:00.269960 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9237d2-563b-459f-b925-793f38f356a1 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:00.283828 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 165/328] 10.222.0.84 () {60 vars in 1134 bytes} [Thu May 19 20:18:00 2022] POST /identity/v3/groups => generated 226 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:18:00.284639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9237d2-563b-459f-b925-793f38f356a1 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:00.291682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7dd25-d9d2-4c84-9900-a63a211fc57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:00.292587 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7dd25-d9d2-4c84-9900-a63a211fc57d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:00.292865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7dd25-d9d2-4c84-9900-a63a211fc57d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:00.293145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7dd25-d9d2-4c84-9900-a63a211fc57d None admin] PATH_INFO: `/v3/users/2536e98f4c39445c9902c6895d193ac4` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:00.295015 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7dd25-d9d2-4c84-9900-a63a211fc57d None admin] RBAC: Authorizing `identity:get_user(user_id=2536e98f4c39445c9902c6895d193ac4)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:00.296573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7dd25-d9d2-4c84-9900-a63a211fc57d None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:00.303798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 164/329] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:18:00 2022] GET /identity/v3/users/2536e98f4c39445c9902c6895d193ac4 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:00.304767 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7dd25-d9d2-4c84-9900-a63a211fc57d None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:00.372998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:00.400320 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9237d2-563b-459f-b925-793f38f356a1 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:00.412717 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:00.426914 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7dd25-d9d2-4c84-9900-a63a211fc57d None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:00.446411 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20615c7-1d6a-40e5-a1b5-d0d00afbcea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:00.447392 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20615c7-1d6a-40e5-a1b5-d0d00afbcea2 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:00.447639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20615c7-1d6a-40e5-a1b5-d0d00afbcea2 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:00.447881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20615c7-1d6a-40e5-a1b5-d0d00afbcea2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:00.459716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20615c7-1d6a-40e5-a1b5-d0d00afbcea2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:00.466939 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9499b3-1fdf-40f2-860f-c9f4268d8fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:00.467609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9499b3-1fdf-40f2-860f-c9f4268d8fc2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:00.467887 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9499b3-1fdf-40f2-860f-c9f4268d8fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:00.468159 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9499b3-1fdf-40f2-860f-c9f4268d8fc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-d20615c7-1d6a-40e5-a1b5-d0d00afbcea2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:00.468361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:00.469432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:00.471598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:00.471598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:00.471598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:00.471598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:00.471598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:00.471598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:00.471598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:00.471598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:00.471598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:00.471598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:00.471598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:18:00.471598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:00.472313 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9499b3-1fdf-40f2-860f-c9f4268d8fc2 None admin] RBAC: Adding query filter params (role.id=666cfd1181184612ba462b8eefcad6a7, user.id=2536e98f4c39445c9902c6895d193ac4, scope.system=all) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:00.472313 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9499b3-1fdf-40f2-860f-c9f4268d8fc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:00.472473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 166/330] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:00.472555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9499b3-1fdf-40f2-860f-c9f4268d8fc2 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:00.475562 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20615c7-1d6a-40e5-a1b5-d0d00afbcea2 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:00.484791 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 165/331] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 20:18:00 2022] GET /identity/v3/role_assignments?user.id=2536e98f4c39445c9902c6895d193ac4&scope.system=all&role.id=666cfd1181184612ba462b8eefcad6a7 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:00.615279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20615c7-1d6a-40e5-a1b5-d0d00afbcea2 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:00.657022 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9c27b-886d-4e73-a220-8c2f0380c6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:00.658549 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9c27b-886d-4e73-a220-8c2f0380c6a4 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:00.658844 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9c27b-886d-4e73-a220-8c2f0380c6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:00.659125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9c27b-886d-4e73-a220-8c2f0380c6a4 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:00.659903 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9c27b-886d-4e73-a220-8c2f0380c6a4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:00.660208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9c27b-886d-4e73-a220-8c2f0380c6a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:00.661643 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9c27b-886d-4e73-a220-8c2f0380c6a4 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:00.671778 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 167/332] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:18:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:00.677811 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9499b3-1fdf-40f2-860f-c9f4268d8fc2 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:00.796299 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9499b3-1fdf-40f2-860f-c9f4268d8fc2 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:00.815807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:00.835856 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247743e-55d4-493e-b8f2-2037b98d89f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:00.836820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247743e-55d4-493e-b8f2-2037b98d89f3 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:00.837040 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247743e-55d4-493e-b8f2-2037b98d89f3 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:00.837440 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247743e-55d4-493e-b8f2-2037b98d89f3 None admin] PATH_INFO: `/v3/users/f8570fdd261d477eaeeccb94ae4c9192` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:00.839882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7247743e-55d4-493e-b8f2-2037b98d89f3 None admin] RBAC: Authorizing `identity:get_user(user_id=f8570fdd261d477eaeeccb94ae4c9192)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:00.841365 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7247743e-55d4-493e-b8f2-2037b98d89f3 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:00.843449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 166/333] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:18:00 2022] GET /identity/v3/users/f8570fdd261d477eaeeccb94ae4c9192 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:00.849236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9c27b-886d-4e73-a220-8c2f0380c6a4 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:00.889576 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:00.963074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9c27b-886d-4e73-a220-8c2f0380c6a4 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:01.001370 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818e708-22ee-4cdd-aead-f86ecef0840d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:01.002656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818e708-22ee-4cdd-aead-f86ecef0840d None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:01.003164 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818e708-22ee-4cdd-aead-f86ecef0840d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:01.003668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818e708-22ee-4cdd-aead-f86ecef0840d None admin] PATH_INFO: `/v3/system/users/f8570fdd261d477eaeeccb94ae4c9192/roles/086ec02a15114cc3b858b231d55e19aa` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:01.005480 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818e708-22ee-4cdd-aead-f86ecef0840d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f8570fdd261d477eaeeccb94ae4c9192, role_id=086ec02a15114cc3b858b231d55e19aa)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:01.007185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818e708-22ee-4cdd-aead-f86ecef0840d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:01.019630 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 168/334] 10.222.0.84 () {58 vars in 1334 bytes} [Thu May 19 20:18:00 2022] PUT /identity/v3/system/users/f8570fdd261d477eaeeccb94ae4c9192/roles/086ec02a15114cc3b858b231d55e19aa => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:18:03.132497 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459c0fa-7f27-456f-bae6-4c66141973ed None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:03.133059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459c0fa-7f27-456f-bae6-4c66141973ed None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:03.133059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459c0fa-7f27-456f-bae6-4c66141973ed None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:03.134860 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 167/335] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:03.139929 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bfcd4-7df5-4807-8094-6ad4131243a4 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:03.140398 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bfcd4-7df5-4807-8094-6ad4131243a4 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:03.140491 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bfcd4-7df5-4807-8094-6ad4131243a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:03.167349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-d26bfcd4-7df5-4807-8094-6ad4131243a4 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:03.168555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-d26bfcd4-7df5-4807-8094-6ad4131243a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:03.260175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 169/336] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:03.311385 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459c0fa-7f27-456f-bae6-4c66141973ed None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:03.455134 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459c0fa-7f27-456f-bae6-4c66141973ed None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:03.501306 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038069e3-3b44-4363-b221-8aa6cbccc5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:03.502113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038069e3-3b44-4363-b221-8aa6cbccc5e7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:03.502422 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038069e3-3b44-4363-b221-8aa6cbccc5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:03.502700 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038069e3-3b44-4363-b221-8aa6cbccc5e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:03.519395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038069e3-3b44-4363-b221-8aa6cbccc5e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-038069e3-3b44-4363-b221-8aa6cbccc5e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:03.527737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:03.528769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:03.530370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:03.530370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:03.530370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:03.530370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:03.530370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:03.530370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:03.530370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:03.530370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:03.530370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:03.530370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:03.530370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:18:03.530370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:03.530370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 168/337] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:03.535366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bfcd4-7df5-4807-8094-6ad4131243a4 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:03.560346 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fc66a-b21f-4840-b0e9-5f68cdb92077 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:03.560683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fc66a-b21f-4840-b0e9-5f68cdb92077 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:03.560898 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fc66a-b21f-4840-b0e9-5f68cdb92077 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:03.562297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 169/338] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:18:03.567612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2656f0-1b3f-4b77-94ed-289a7baaffa8 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:03.567900 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2656f0-1b3f-4b77-94ed-289a7baaffa8 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:03.568176 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2656f0-1b3f-4b77-94ed-289a7baaffa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:03.584602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:03.607401 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-af2656f0-1b3f-4b77-94ed-289a7baaffa8 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:03.608629 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-af2656f0-1b3f-4b77-94ed-289a7baaffa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:03.663543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:03.686833 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bfcd4-7df5-4807-8094-6ad4131243a4 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:03.712587 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 170/339] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 20:18:03.714313 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed32be-f81c-4f11-a16e-386f60bc34a4 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:03.714595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed32be-f81c-4f11-a16e-386f60bc34a4 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:03.714865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed32be-f81c-4f11-a16e-386f60bc34a4 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:03.720140 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 171/340] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:03.721455 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf3c1-c82e-4cb6-a8b9-b8b97893c72a None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:03.721805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf3c1-c82e-4cb6-a8b9-b8b97893c72a None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:03.722071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf3c1-c82e-4cb6-a8b9-b8b97893c72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:03.731796 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203cfad5-de9c-4dac-8660-d859a0b8ad7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:03.733013 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cfad5-de9c-4dac-8660-d859a0b8ad7d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:03.734462 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cfad5-de9c-4dac-8660-d859a0b8ad7d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:03.734911 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cfad5-de9c-4dac-8660-d859a0b8ad7d None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:03.735836 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203cfad5-de9c-4dac-8660-d859a0b8ad7d None admin] RBAC: Adding query filter params (name=member) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:03.736303 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203cfad5-de9c-4dac-8660-d859a0b8ad7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:03.737836 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203cfad5-de9c-4dac-8660-d859a0b8ad7d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:03.749814 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 170/341] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:18:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:03.751280 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-f03bf3c1-c82e-4cb6-a8b9-b8b97893c72a None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:03.752672 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-f03bf3c1-c82e-4cb6-a8b9-b8b97893c72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:03.754505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203cfad5-de9c-4dac-8660-d859a0b8ad7d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:03.861529 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:03.879015 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203cfad5-de9c-4dac-8660-d859a0b8ad7d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:03.919055 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c01353-d23f-4b13-b4e2-b8081f6fc2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:03.920172 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c01353-d23f-4b13-b4e2-b8081f6fc2df None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:03.920729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c01353-d23f-4b13-b4e2-b8081f6fc2df None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:03.921237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c01353-d23f-4b13-b4e2-b8081f6fc2df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:03.923342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 172/342] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:03.924400 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bf3c1-c82e-4cb6-a8b9-b8b97893c72a None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:03.945155 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c01353-d23f-4b13-b4e2-b8081f6fc2df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-05c01353-d23f-4b13-b4e2-b8081f6fc2df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:03.957738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:03.959148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:03.960307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:03.960307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:03.960307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:03.960307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:03.960307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:03.960307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:03.960307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:03.960307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:03.960307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:03.960307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:03.960307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:18:03.960307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:03.963580 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 171/343] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:18:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:03.969343 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c01353-d23f-4b13-b4e2-b8081f6fc2df None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:04.078211 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bf3c1-c82e-4cb6-a8b9-b8b97893c72a None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:04.080717 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:04.098970 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c01353-d23f-4b13-b4e2-b8081f6fc2df None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:04.127910 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a309e9e-762b-4094-bae7-db701e6f61fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:04.128590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a309e9e-762b-4094-bae7-db701e6f61fa None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:04.128871 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a309e9e-762b-4094-bae7-db701e6f61fa None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:04.129142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a309e9e-762b-4094-bae7-db701e6f61fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:04.138826 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa74c9-1561-4432-bb55-6238c5735b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:04.139757 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa74c9-1561-4432-bb55-6238c5735b41 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:04.140206 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa74c9-1561-4432-bb55-6238c5735b41 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:04.141074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa74c9-1561-4432-bb55-6238c5735b41 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:04.142221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa74c9-1561-4432-bb55-6238c5735b41 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:04.142690 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa74c9-1561-4432-bb55-6238c5735b41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:04.144215 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa74c9-1561-4432-bb55-6238c5735b41 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:04.154232 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 172/344] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:04.155440 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa74c9-1561-4432-bb55-6238c5735b41 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:04.169073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:04.230743 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a309e9e-762b-4094-bae7-db701e6f61fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:04.232146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a309e9e-762b-4094-bae7-db701e6f61fa None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:04.272116 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa74c9-1561-4432-bb55-6238c5735b41 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:04.309671 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d6400-2653-4541-9340-21cafc9c6e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:04.310789 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d6400-2653-4541-9340-21cafc9c6e1d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:04.311263 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d6400-2653-4541-9340-21cafc9c6e1d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:04.311895 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d6400-2653-4541-9340-21cafc9c6e1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:04.323695 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d6400-2653-4541-9340-21cafc9c6e1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-005d6400-2653-4541-9340-21cafc9c6e1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:04.332208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:04.333469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:04.334561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:04.334561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:04.334561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:04.334561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:04.334561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:04.334561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:04.334561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:04.334561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:04.334561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:04.334561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:04.334561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:18:04.334561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:04.336725 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 173/345] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:04.337805 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d6400-2653-4541-9340-21cafc9c6e1d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:04.384994 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 173/346] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:04.385889 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a309e9e-762b-4094-bae7-db701e6f61fa None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:04.392923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:04.420926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:04.461237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d6400-2653-4541-9340-21cafc9c6e1d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:04.501892 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05529b5-653b-4855-8382-63c9e8838a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:04.502834 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05529b5-653b-4855-8382-63c9e8838a54 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:04.503279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05529b5-653b-4855-8382-63c9e8838a54 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:04.503724 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05529b5-653b-4855-8382-63c9e8838a54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:04.532669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a309e9e-762b-4094-bae7-db701e6f61fa None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:04.569690 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64ba82-c624-4fc0-97b4-2eb6a77db878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:04.570678 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64ba82-c624-4fc0-97b4-2eb6a77db878 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:04.571108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64ba82-c624-4fc0-97b4-2eb6a77db878 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:04.571686 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64ba82-c624-4fc0-97b4-2eb6a77db878 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:04.572761 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64ba82-c624-4fc0-97b4-2eb6a77db878 None admin] RBAC: Adding query filter params (name=member) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:04.573250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64ba82-c624-4fc0-97b4-2eb6a77db878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:04.574837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64ba82-c624-4fc0-97b4-2eb6a77db878 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:04.583745 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 174/347] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:18:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:04.584751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64ba82-c624-4fc0-97b4-2eb6a77db878 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:04.586471 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05529b5-653b-4855-8382-63c9e8838a54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:04.590151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05529b5-653b-4855-8382-63c9e8838a54 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:04.655592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:04.670804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:04.693689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64ba82-c624-4fc0-97b4-2eb6a77db878 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:04.709453 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 174/348] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:04.711334 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec300d-f41f-4527-a03e-66bc16611d6b None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:04.711782 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec300d-f41f-4527-a03e-66bc16611d6b None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:04.712298 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec300d-f41f-4527-a03e-66bc16611d6b None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:04.715027 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 175/349] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:04.715974 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec300d-f41f-4527-a03e-66bc16611d6b None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:04.730366 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e46833-78f4-427b-a399-0852286edef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:04.731310 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e46833-78f4-427b-a399-0852286edef7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:04.731609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e46833-78f4-427b-a399-0852286edef7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:04.731918 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e46833-78f4-427b-a399-0852286edef7 None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:04.733142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e46833-78f4-427b-a399-0852286edef7 None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:04.734953 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e46833-78f4-427b-a399-0852286edef7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:04.736522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 175/350] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:18:04 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:04.741868 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561857f-0aba-4968-989f-b4d305e0c59d None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:04.741868 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561857f-0aba-4968-989f-b4d305e0c59d None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:04.741868 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561857f-0aba-4968-989f-b4d305e0c59d None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:04.741868 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 176/351] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:04.741868 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561857f-0aba-4968-989f-b4d305e0c59d None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:04.864355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec300d-f41f-4527-a03e-66bc16611d6b None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:04.869458 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561857f-0aba-4968-989f-b4d305e0c59d None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:04.919835 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcce4a3-5bf0-4a67-b93a-fe338d8469ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:04.920822 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcce4a3-5bf0-4a67-b93a-fe338d8469ae None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:04.922130 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcce4a3-5bf0-4a67-b93a-fe338d8469ae None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:04.923281 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcce4a3-5bf0-4a67-b93a-fe338d8469ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:04.929203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4378be8-3d0a-4970-9a09-1a5c31d0aa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:04.930090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4378be8-3d0a-4970-9a09-1a5c31d0aa1d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:04.930430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4378be8-3d0a-4970-9a09-1a5c31d0aa1d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:04.930707 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4378be8-3d0a-4970-9a09-1a5c31d0aa1d None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:04.932165 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4378be8-3d0a-4970-9a09-1a5c31d0aa1d None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:04.934593 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4378be8-3d0a-4970-9a09-1a5c31d0aa1d None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:04.936511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 177/352] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:18:04 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:04.937820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348069c4-6e24-4a38-874e-c90826bdb39a None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:04.938607 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348069c4-6e24-4a38-874e-c90826bdb39a None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:04.938894 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348069c4-6e24-4a38-874e-c90826bdb39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:04.944971 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:04.966861 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-348069c4-6e24-4a38-874e-c90826bdb39a None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:04.968377 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-348069c4-6e24-4a38-874e-c90826bdb39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:04.976302 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:05.015445 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcce4a3-5bf0-4a67-b93a-fe338d8469ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:05.016949 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcce4a3-5bf0-4a67-b93a-fe338d8469ae None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:05.049666 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 178/353] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:05.050595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348069c4-6e24-4a38-874e-c90826bdb39a None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:05.154948 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 176/354] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:05.156417 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d349e4-2cfc-4fa2-80c6-62e032701036 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:05.156804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d349e4-2cfc-4fa2-80c6-62e032701036 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:05.157178 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d349e4-2cfc-4fa2-80c6-62e032701036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:05.167220 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348069c4-6e24-4a38-874e-c90826bdb39a None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:05.179321 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-64d349e4-2cfc-4fa2-80c6-62e032701036 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:05.180533 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-64d349e4-2cfc-4fa2-80c6-62e032701036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:05.205982 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:05.215796 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184edf2-bb7f-473d-a9fc-3b71bdf36ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:05.217099 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184edf2-bb7f-473d-a9fc-3b71bdf36ad2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:05.217593 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184edf2-bb7f-473d-a9fc-3b71bdf36ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:05.218078 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184edf2-bb7f-473d-a9fc-3b71bdf36ad2 None admin] PATH_INFO: `/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:05.224939 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184edf2-bb7f-473d-a9fc-3b71bdf36ad2 None admin] RBAC: Authorizing `identity:get_group(group_id=5a9c1d404b51489eb1e36f1cc3a722de)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:05.227247 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184edf2-bb7f-473d-a9fc-3b71bdf36ad2 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:05.228668 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 179/355] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:18:05 2022] GET /identity/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:05.230126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184edf2-bb7f-473d-a9fc-3b71bdf36ad2 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:05.266862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:05.274802 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 177/356] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:05.276144 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d349e4-2cfc-4fa2-80c6-62e032701036 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:05.368234 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184edf2-bb7f-473d-a9fc-3b71bdf36ad2 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:05.413419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1163d9d-2ac8-41c4-9207-238b4300be86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:05.414211 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1163d9d-2ac8-41c4-9207-238b4300be86 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:05.414521 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1163d9d-2ac8-41c4-9207-238b4300be86 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:05.414796 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1163d9d-2ac8-41c4-9207-238b4300be86 None admin] PATH_INFO: `/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:05.416367 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1163d9d-2ac8-41c4-9207-238b4300be86 None admin] RBAC: Authorizing `identity:get_group(group_id=5a9c1d404b51489eb1e36f1cc3a722de)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:05.419755 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1163d9d-2ac8-41c4-9207-238b4300be86 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:05.419998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 180/357] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:18:05 2022] GET /identity/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:05.423133 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1163d9d-2ac8-41c4-9207-238b4300be86 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:05.442982 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d349e4-2cfc-4fa2-80c6-62e032701036 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:05.486793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071d70d-05fd-474d-b414-0af739f0b22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:05.487806 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071d70d-05fd-474d-b414-0af739f0b22d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:05.488088 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071d70d-05fd-474d-b414-0af739f0b22d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:05.488363 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071d70d-05fd-474d-b414-0af739f0b22d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:05.500739 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071d70d-05fd-474d-b414-0af739f0b22d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-6071d70d-05fd-474d-b414-0af739f0b22d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:05.509184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:05.510366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:05.511364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:05.511364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:05.511364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:05.511364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:05.511364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:05.511364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:05.511364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:05.511364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:05.511364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:05.511364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:05.511364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:18:05.511364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:05.514800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 178/358] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:18:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:05.516089 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071d70d-05fd-474d-b414-0af739f0b22d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:05.518006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:05.574826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1163d9d-2ac8-41c4-9207-238b4300be86 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:05.587862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:05.614253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a807f26-95f6-4f1f-994b-9baaeb673c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:05.615166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a807f26-95f6-4f1f-994b-9baaeb673c5b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:05.615445 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a807f26-95f6-4f1f-994b-9baaeb673c5b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:05.615718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a807f26-95f6-4f1f-994b-9baaeb673c5b None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:05.616917 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a807f26-95f6-4f1f-994b-9baaeb673c5b None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:05.618511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a807f26-95f6-4f1f-994b-9baaeb673c5b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:05.620238 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 181/359] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:18:05 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:05.621014 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a807f26-95f6-4f1f-994b-9baaeb673c5b None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:05.642357 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071d70d-05fd-474d-b414-0af739f0b22d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:05.688684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9d604-b0f1-4f4d-b21f-a69c040a3655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:05.689288 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9d604-b0f1-4f4d-b21f-a69c040a3655 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:05.689689 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9d604-b0f1-4f4d-b21f-a69c040a3655 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:05.689970 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9d604-b0f1-4f4d-b21f-a69c040a3655 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:05.690951 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9d604-b0f1-4f4d-b21f-a69c040a3655 None admin] RBAC: Adding query filter params (group.id=5a9c1d404b51489eb1e36f1cc3a722de, role.id=666cfd1181184612ba462b8eefcad6a7, scope.project.id=ba315bae61f943debd8460a5fa9dd151) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:05.691429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9d604-b0f1-4f4d-b21f-a69c040a3655 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:05.692881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9d604-b0f1-4f4d-b21f-a69c040a3655 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:05.701025 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 179/360] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:18:05 2022] GET /identity/v3/role_assignments?group.id=5a9c1d404b51489eb1e36f1cc3a722de&scope.project.id=ba315bae61f943debd8460a5fa9dd151&role.id=666cfd1181184612ba462b8eefcad6a7 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:05.702547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9d604-b0f1-4f4d-b21f-a69c040a3655 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:05.753961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a807f26-95f6-4f1f-994b-9baaeb673c5b None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:05.793816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:05.807564 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca4bf5-3a8e-4833-9f63-b64cd0b43b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:05.809350 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca4bf5-3a8e-4833-9f63-b64cd0b43b4e None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:05.809636 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca4bf5-3a8e-4833-9f63-b64cd0b43b4e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:05.811027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca4bf5-3a8e-4833-9f63-b64cd0b43b4e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:05.827398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca4bf5-3a8e-4833-9f63-b64cd0b43b4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-90ca4bf5-3a8e-4833-9f63-b64cd0b43b4e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:05.835427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:05.836822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:05.838104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:05.838104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:05.838104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:05.838104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:05.838104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:05.838104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:05.838104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:05.838104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:05.838104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:05.838104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:05.838104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:18:05.838104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:05.840245 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 182/361] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:05.845995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca4bf5-3a8e-4833-9f63-b64cd0b43b4e None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:05.852527 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9d604-b0f1-4f4d-b21f-a69c040a3655 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:05.897587 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:05.912576 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75bf76-491f-447b-8bd3-afd8ec94d0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:05.913255 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75bf76-491f-447b-8bd3-afd8ec94d0ec None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:05.913544 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75bf76-491f-447b-8bd3-afd8ec94d0ec None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:05.913820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75bf76-491f-447b-8bd3-afd8ec94d0ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:05.916624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75bf76-491f-447b-8bd3-afd8ec94d0ec None admin] RBAC: Adding query filter params (group.id=5a9c1d404b51489eb1e36f1cc3a722de, role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2, scope.project.id=ba315bae61f943debd8460a5fa9dd151) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:05.916624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75bf76-491f-447b-8bd3-afd8ec94d0ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:05.917922 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75bf76-491f-447b-8bd3-afd8ec94d0ec None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:05.924427 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 180/362] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:18:05 2022] GET /identity/v3/role_assignments?group.id=5a9c1d404b51489eb1e36f1cc3a722de&scope.project.id=ba315bae61f943debd8460a5fa9dd151&role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:05.925875 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0c662-8328-49a2-bb3f-348671467fe2 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:05.926166 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0c662-8328-49a2-bb3f-348671467fe2 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:05.930198 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0c662-8328-49a2-bb3f-348671467fe2 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:05.934762 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 181/363] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:05.935685 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0c662-8328-49a2-bb3f-348671467fe2 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:05.991392 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca4bf5-3a8e-4833-9f63-b64cd0b43b4e None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:06.041441 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08351eff-33b9-4bdf-8bda-5eb99ec1d548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:06.042603 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08351eff-33b9-4bdf-8bda-5eb99ec1d548 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:06.043124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08351eff-33b9-4bdf-8bda-5eb99ec1d548 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:06.043631 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08351eff-33b9-4bdf-8bda-5eb99ec1d548 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:06.044656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08351eff-33b9-4bdf-8bda-5eb99ec1d548 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:06.045195 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08351eff-33b9-4bdf-8bda-5eb99ec1d548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:06.047660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08351eff-33b9-4bdf-8bda-5eb99ec1d548 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:06.060401 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 183/364] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:06.061815 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08351eff-33b9-4bdf-8bda-5eb99ec1d548 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:06.084887 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0c662-8328-49a2-bb3f-348671467fe2 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:06.137433 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:06.137650 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f55c44-3331-432d-8c70-50e883087313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:06.139021 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f55c44-3331-432d-8c70-50e883087313 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:06.139371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f55c44-3331-432d-8c70-50e883087313 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:06.139716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f55c44-3331-432d-8c70-50e883087313 None admin] PATH_INFO: `/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:06.141257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f55c44-3331-432d-8c70-50e883087313 None admin] RBAC: Authorizing `identity:get_group(group_id=5a9c1d404b51489eb1e36f1cc3a722de)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:06.143509 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f55c44-3331-432d-8c70-50e883087313 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:06.145882 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 182/365] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:18:05 2022] GET /identity/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:06.149179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae25eb0-6239-43b2-946d-d2b2d95d75cf None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:06.149387 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae25eb0-6239-43b2-946d-d2b2d95d75cf None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:06.150019 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae25eb0-6239-43b2-946d-d2b2d95d75cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:06.170927 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-9ae25eb0-6239-43b2-946d-d2b2d95d75cf None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:06.172322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-9ae25eb0-6239-43b2-946d-d2b2d95d75cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:06.202322 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08351eff-33b9-4bdf-8bda-5eb99ec1d548 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:06.238112 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:06.245711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd5986-03f4-4c06-856a-f6cf421f0bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:06.246369 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd5986-03f4-4c06-856a-f6cf421f0bd3 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:06.246656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd5986-03f4-4c06-856a-f6cf421f0bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:06.246931 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd5986-03f4-4c06-856a-f6cf421f0bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:06.247873 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd5986-03f4-4c06-856a-f6cf421f0bd3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:06.248184 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd5986-03f4-4c06-856a-f6cf421f0bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:06.249614 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd5986-03f4-4c06-856a-f6cf421f0bd3 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:06.259348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 184/366] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:18:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:06.260246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd5986-03f4-4c06-856a-f6cf421f0bd3 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:06.269063 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 183/367] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:06.270400 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae25eb0-6239-43b2-946d-d2b2d95d75cf None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:06.373020 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd5986-03f4-4c06-856a-f6cf421f0bd3 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:06.391623 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:06.394015 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae25eb0-6239-43b2-946d-d2b2d95d75cf None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:06.425461 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ca61a-1dd5-465f-bea1-73708af5806c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:06.426143 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ca61a-1dd5-465f-bea1-73708af5806c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:06.426462 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ca61a-1dd5-465f-bea1-73708af5806c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:06.426742 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ca61a-1dd5-465f-bea1-73708af5806c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:06.450578 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ca0d9-2caf-46f9-b19a-9a06976cb9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:06.451820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca0d9-2caf-46f9-b19a-9a06976cb9f9 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:06.452508 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca0d9-2caf-46f9-b19a-9a06976cb9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:06.452948 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca0d9-2caf-46f9-b19a-9a06976cb9f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:06.454376 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ca0d9-2caf-46f9-b19a-9a06976cb9f9 None admin] RBAC: Adding query filter params (group.id=5a9c1d404b51489eb1e36f1cc3a722de, role.id=666cfd1181184612ba462b8eefcad6a7, scope.project.id=3022d4bdd09249328ba6a2f60404faf0) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:06.454861 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ca0d9-2caf-46f9-b19a-9a06976cb9f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:06.457084 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ca0d9-2caf-46f9-b19a-9a06976cb9f9 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:06.466066 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 184/368] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:18:06 2022] GET /identity/v3/role_assignments?group.id=5a9c1d404b51489eb1e36f1cc3a722de&scope.project.id=3022d4bdd09249328ba6a2f60404faf0&role.id=666cfd1181184612ba462b8eefcad6a7 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:06.467210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ca0d9-2caf-46f9-b19a-9a06976cb9f9 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:06.531003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:06.573581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ca61a-1dd5-465f-bea1-73708af5806c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:06.575214 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ca61a-1dd5-465f-bea1-73708af5806c None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:06.591717 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ca0d9-2caf-46f9-b19a-9a06976cb9f9 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:06.634785 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad51668-088e-4772-a6ba-56e3d44e1045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:06.635968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad51668-088e-4772-a6ba-56e3d44e1045 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:06.636423 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad51668-088e-4772-a6ba-56e3d44e1045 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:06.636880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad51668-088e-4772-a6ba-56e3d44e1045 None admin] PATH_INFO: `/v3/users/f8570fdd261d477eaeeccb94ae4c9192` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:06.640268 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad51668-088e-4772-a6ba-56e3d44e1045 None admin] RBAC: Authorizing `identity:get_user(user_id=f8570fdd261d477eaeeccb94ae4c9192)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:06.641992 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad51668-088e-4772-a6ba-56e3d44e1045 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:06.644701 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 185/369] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:18:06 2022] GET /identity/v3/users/f8570fdd261d477eaeeccb94ae4c9192 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:06.645697 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad51668-088e-4772-a6ba-56e3d44e1045 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:06.771950 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad51668-088e-4772-a6ba-56e3d44e1045 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:06.794023 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:06.814625 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 185/370] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:06.815525 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ca61a-1dd5-465f-bea1-73708af5806c None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:06.832931 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba668e-5b3a-4c5d-8fb6-5adc20a1de7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:06.833758 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba668e-5b3a-4c5d-8fb6-5adc20a1de7c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:06.834041 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba668e-5b3a-4c5d-8fb6-5adc20a1de7c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:06.834342 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba668e-5b3a-4c5d-8fb6-5adc20a1de7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:06.835032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:06.855742 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba668e-5b3a-4c5d-8fb6-5adc20a1de7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-deba668e-5b3a-4c5d-8fb6-5adc20a1de7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:06.863968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:06.865008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:06.866038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:06.866038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:06.866038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:06.866038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:06.866038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:06.866038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:06.866038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:06.866038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:06.866038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:06.866038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:06.866038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:06.866038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:06.866038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 186/371] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:18:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:06.867172 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba668e-5b3a-4c5d-8fb6-5adc20a1de7c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:06.941151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ca61a-1dd5-465f-bea1-73708af5806c None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:06.990420 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a2ad4-df0a-428c-abce-79be1f8ba5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:06.991669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a2ad4-df0a-428c-abce-79be1f8ba5db None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:06.991798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a2ad4-df0a-428c-abce-79be1f8ba5db None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:06.992056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a2ad4-df0a-428c-abce-79be1f8ba5db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:06.993489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a2ad4-df0a-428c-abce-79be1f8ba5db None admin] RBAC: Adding query filter params (role.id=086ec02a15114cc3b858b231d55e19aa, user.id=f8570fdd261d477eaeeccb94ae4c9192, scope.system=all) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:06.994084 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a2ad4-df0a-428c-abce-79be1f8ba5db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:06.996042 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a2ad4-df0a-428c-abce-79be1f8ba5db None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:06.996696 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba668e-5b3a-4c5d-8fb6-5adc20a1de7c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:07.011825 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 186/372] 10.222.0.84 () {56 vars in 1308 bytes} [Thu May 19 20:18:06 2022] GET /identity/v3/role_assignments?user.id=f8570fdd261d477eaeeccb94ae4c9192&scope.system=all&role.id=086ec02a15114cc3b858b231d55e19aa => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:07.018286 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a2ad4-df0a-428c-abce-79be1f8ba5db None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:07.111136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61108ad-86c4-4df6-9369-216b6edd208b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:07.111386 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:07.112635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61108ad-86c4-4df6-9369-216b6edd208b None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:07.112968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61108ad-86c4-4df6-9369-216b6edd208b None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:07.113355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61108ad-86c4-4df6-9369-216b6edd208b None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:07.119293 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61108ad-86c4-4df6-9369-216b6edd208b None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:07.120888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61108ad-86c4-4df6-9369-216b6edd208b None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:07.123383 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 187/373] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:18:06 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:07.135387 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61108ad-86c4-4df6-9369-216b6edd208b None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:07.140537 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a2ad4-df0a-428c-abce-79be1f8ba5db None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:07.190154 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402e52f-ebb1-4870-994c-faeebc114418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:07.191880 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402e52f-ebb1-4870-994c-faeebc114418 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:07.192395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402e52f-ebb1-4870-994c-faeebc114418 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:07.192877 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402e52f-ebb1-4870-994c-faeebc114418 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:07.194172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402e52f-ebb1-4870-994c-faeebc114418 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:07.194705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402e52f-ebb1-4870-994c-faeebc114418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:07.197315 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402e52f-ebb1-4870-994c-faeebc114418 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:07.212463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 187/374] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:07.222528 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402e52f-ebb1-4870-994c-faeebc114418 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:07.276310 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:07.360554 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402e52f-ebb1-4870-994c-faeebc114418 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:07.396702 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61108ad-86c4-4df6-9369-216b6edd208b None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:07.404485 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2a39b-70e0-4f8f-9e9a-1b949db5929f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:07.405774 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2a39b-70e0-4f8f-9e9a-1b949db5929f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:07.407267 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2a39b-70e0-4f8f-9e9a-1b949db5929f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:07.407761 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2a39b-70e0-4f8f-9e9a-1b949db5929f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:07.440314 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:07.495055 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2a39b-70e0-4f8f-9e9a-1b949db5929f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:07.498113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2a39b-70e0-4f8f-9e9a-1b949db5929f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:07.502994 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e14247-c83b-403b-972a-ffafc15b6d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:07.504523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e14247-c83b-403b-972a-ffafc15b6d79 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:07.504523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e14247-c83b-403b-972a-ffafc15b6d79 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:07.504523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e14247-c83b-403b-972a-ffafc15b6d79 None admin] PATH_INFO: `/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:07.507144 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e14247-c83b-403b-972a-ffafc15b6d79 None admin] RBAC: Authorizing `identity:get_group(group_id=5a9c1d404b51489eb1e36f1cc3a722de)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:07.508681 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e14247-c83b-403b-972a-ffafc15b6d79 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:07.510924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 188/375] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:18:07 2022] GET /identity/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de => generated 226 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:07.519193 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e14247-c83b-403b-972a-ffafc15b6d79 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:07.624480 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 188/376] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:18:07.630374 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2a39b-70e0-4f8f-9e9a-1b949db5929f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:07.682757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:07.688091 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e14247-c83b-403b-972a-ffafc15b6d79 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:07.705952 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:07.741156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172eb23-9ce8-4d18-8681-db217daafe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:07.742197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172eb23-9ce8-4d18-8681-db217daafe02 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:07.742742 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172eb23-9ce8-4d18-8681-db217daafe02 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:07.743289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172eb23-9ce8-4d18-8681-db217daafe02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:07.744344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172eb23-9ce8-4d18-8681-db217daafe02 None admin] RBAC: Adding query filter params (group.id=5a9c1d404b51489eb1e36f1cc3a722de, role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2, scope.project.id=3022d4bdd09249328ba6a2f60404faf0) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:07.744884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172eb23-9ce8-4d18-8681-db217daafe02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:07.746603 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172eb23-9ce8-4d18-8681-db217daafe02 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:07.752998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 189/377] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:18:07 2022] GET /identity/v3/role_assignments?group.id=5a9c1d404b51489eb1e36f1cc3a722de&scope.project.id=3022d4bdd09249328ba6a2f60404faf0&role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:07.764901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2a39b-70e0-4f8f-9e9a-1b949db5929f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:07.804596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464cc03-512e-45a1-9b76-fdc4208eda67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:07.805475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464cc03-512e-45a1-9b76-fdc4208eda67 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:07.805801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464cc03-512e-45a1-9b76-fdc4208eda67 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:07.806113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464cc03-512e-45a1-9b76-fdc4208eda67 None admin] PATH_INFO: `/v3/projects/8fd7be0b818c4867be2daed29f53ab77` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:07.807306 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464cc03-512e-45a1-9b76-fdc4208eda67 None admin] RBAC: Authorizing `identity:get_project(project_id=8fd7be0b818c4867be2daed29f53ab77)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:07.815322 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464cc03-512e-45a1-9b76-fdc4208eda67 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:07.815322 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 189/378] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:18:07 2022] GET /identity/v3/projects/8fd7be0b818c4867be2daed29f53ab77 => generated 344 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:07.817197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172eb23-9ce8-4d18-8681-db217daafe02 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:07.971746 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172eb23-9ce8-4d18-8681-db217daafe02 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:08.019468 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414674a-5689-47d7-80f7-efbf51949dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:08.020477 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414674a-5689-47d7-80f7-efbf51949dfd None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:08.020760 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414674a-5689-47d7-80f7-efbf51949dfd None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:08.021033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414674a-5689-47d7-80f7-efbf51949dfd None admin] PATH_INFO: `/v3/groups/b9d9d93453fd4f4d9de67be02d52ac4d` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:08.032907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414674a-5689-47d7-80f7-efbf51949dfd None admin] RBAC: Authorizing `identity:get_group(group_id=b9d9d93453fd4f4d9de67be02d52ac4d)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:08.034482 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414674a-5689-47d7-80f7-efbf51949dfd None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:08.036480 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 190/379] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:18:07 2022] GET /identity/v3/groups/b9d9d93453fd4f4d9de67be02d52ac4d => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:08.043925 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464cc03-512e-45a1-9b76-fdc4208eda67 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:08.189991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464cc03-512e-45a1-9b76-fdc4208eda67 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:08.192956 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:08.234559 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52a113-bedd-4245-b904-01b95820f9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:08.235161 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52a113-bedd-4245-b904-01b95820f9e9 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:08.235455 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52a113-bedd-4245-b904-01b95820f9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:08.235741 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52a113-bedd-4245-b904-01b95820f9e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:08.236611 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52a113-bedd-4245-b904-01b95820f9e9 None admin] RBAC: Adding query filter params (group.id=b9d9d93453fd4f4d9de67be02d52ac4d, role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=8fd7be0b818c4867be2daed29f53ab77) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:08.236953 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52a113-bedd-4245-b904-01b95820f9e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:08.238487 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52a113-bedd-4245-b904-01b95820f9e9 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:08.245412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 190/380] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:18:08 2022] GET /identity/v3/role_assignments?group.id=b9d9d93453fd4f4d9de67be02d52ac4d&scope.project.id=8fd7be0b818c4867be2daed29f53ab77&role.id=b239f941cb724331bf28dc489b70ad1d => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:08.728765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fc3f4-0ecd-4951-a498-0f2e6ca81f57 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:08.729402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fc3f4-0ecd-4951-a498-0f2e6ca81f57 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:08.729402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fc3f4-0ecd-4951-a498-0f2e6ca81f57 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:08.731123 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 191/381] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:08.737385 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014044c-18b4-4a44-971f-668ad440df75 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:08.737829 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014044c-18b4-4a44-971f-668ad440df75 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:08.738025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014044c-18b4-4a44-971f-668ad440df75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:08.759493 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-9014044c-18b4-4a44-971f-668ad440df75 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:08.760934 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-9014044c-18b4-4a44-971f-668ad440df75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:08.854615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 191/382] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:08.884638 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8fc3f4-0ecd-4951-a498-0f2e6ca81f57 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:08.938093 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:09.015901 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8fc3f4-0ecd-4951-a498-0f2e6ca81f57 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:09.053874 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c815d5-394b-4bf2-8177-dffee3f0d061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:09.054733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c815d5-394b-4bf2-8177-dffee3f0d061 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:09.055018 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c815d5-394b-4bf2-8177-dffee3f0d061 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:09.055293 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c815d5-394b-4bf2-8177-dffee3f0d061 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:09.073424 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c815d5-394b-4bf2-8177-dffee3f0d061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-93c815d5-394b-4bf2-8177-dffee3f0d061 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:09.085454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:09.087450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:09.088721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:09.088721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:09.088721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:09.088721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:09.088721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:09.088721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:09.088721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:09.088721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:09.088721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:09.088721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:09.088721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:18:09.088721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:09.088721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 192/383] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:09.093674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014044c-18b4-4a44-971f-668ad440df75 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:09.215044 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014044c-18b4-4a44-971f-668ad440df75 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:09.223030 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:09.265095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44504277-a120-4d20-839c-c588dd8c7058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:09.265805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44504277-a120-4d20-839c-c588dd8c7058 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:09.266142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44504277-a120-4d20-839c-c588dd8c7058 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:09.266402 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44504277-a120-4d20-839c-c588dd8c7058 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:09.267187 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44504277-a120-4d20-839c-c588dd8c7058 None admin] RBAC: Adding query filter params (name=member) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:09.267497 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44504277-a120-4d20-839c-c588dd8c7058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:09.268962 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44504277-a120-4d20-839c-c588dd8c7058 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:09.282100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 192/384] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:18:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:09.288008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c815d5-394b-4bf2-8177-dffee3f0d061 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:09.289074 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35510a8-fbb1-4ef0-909a-a254f6d257af None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:09.289361 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35510a8-fbb1-4ef0-909a-a254f6d257af None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:09.289643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35510a8-fbb1-4ef0-909a-a254f6d257af None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:09.291380 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 193/385] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:09.298872 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841dfac-a7cb-4f81-bee2-3d2adf9c9453 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:09.299163 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841dfac-a7cb-4f81-bee2-3d2adf9c9453 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:09.299425 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841dfac-a7cb-4f81-bee2-3d2adf9c9453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:09.331488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-f841dfac-a7cb-4f81-bee2-3d2adf9c9453 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:09.332772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-f841dfac-a7cb-4f81-bee2-3d2adf9c9453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:09.409789 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c815d5-394b-4bf2-8177-dffee3f0d061 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:09.411495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:09.454592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 194/386] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:18:09.484926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db55385-0407-4a00-980e-e581a8da1f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:09.485826 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db55385-0407-4a00-980e-e581a8da1f63 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:09.486115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db55385-0407-4a00-980e-e581a8da1f63 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:09.486422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db55385-0407-4a00-980e-e581a8da1f63 None admin] PATH_INFO: `/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:09.490003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db55385-0407-4a00-980e-e581a8da1f63 None admin] RBAC: Authorizing `identity:get_group(group_id=5a9c1d404b51489eb1e36f1cc3a722de)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:09.491703 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db55385-0407-4a00-980e-e581a8da1f63 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:09.493812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 193/387] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:18:09 2022] GET /identity/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:09.499440 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841dfac-a7cb-4f81-bee2-3d2adf9c9453 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:09.504427 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db55385-0407-4a00-980e-e581a8da1f63 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:09.606641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:09.622802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841dfac-a7cb-4f81-bee2-3d2adf9c9453 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:09.667207 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb9b34-e5a1-42db-a85b-732428ac6832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:09.668090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb9b34-e5a1-42db-a85b-732428ac6832 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:09.668711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb9b34-e5a1-42db-a85b-732428ac6832 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:09.668990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb9b34-e5a1-42db-a85b-732428ac6832 None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:09.670196 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb9b34-e5a1-42db-a85b-732428ac6832 None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:09.671703 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb9b34-e5a1-42db-a85b-732428ac6832 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:09.673735 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 195/388] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:18:09 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:09.674973 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f913a-e5c5-49cb-8850-c9c6f92abc62 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:09.675271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f913a-e5c5-49cb-8850-c9c6f92abc62 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:09.675540 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f913a-e5c5-49cb-8850-c9c6f92abc62 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:09.680853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 196/389] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:09.680853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f913a-e5c5-49cb-8850-c9c6f92abc62 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:09.690313 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db55385-0407-4a00-980e-e581a8da1f63 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:09.738339 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728ee49-b81a-418c-b5a1-d12f6d297aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:09.740261 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728ee49-b81a-418c-b5a1-d12f6d297aff None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:09.741287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728ee49-b81a-418c-b5a1-d12f6d297aff None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:09.741821 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728ee49-b81a-418c-b5a1-d12f6d297aff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:09.759297 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728ee49-b81a-418c-b5a1-d12f6d297aff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-9728ee49-b81a-418c-b5a1-d12f6d297aff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:09.771473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:09.772728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:09.773915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:09.773915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:09.773915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:09.773915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:09.773915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:09.773915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:09.773915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:09.773915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:09.773915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:09.773915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:09.773915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:18:09.773915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:09.776617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 194/390] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:18:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:09.778740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c90e6c-a541-444a-8aab-79106f1156aa None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:09.779215 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c90e6c-a541-444a-8aab-79106f1156aa None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:09.779654 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c90e6c-a541-444a-8aab-79106f1156aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:09.803861 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-01c90e6c-a541-444a-8aab-79106f1156aa None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:09.805645 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-01c90e6c-a541-444a-8aab-79106f1156aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:09.808142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f913a-e5c5-49cb-8850-c9c6f92abc62 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:09.834389 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:09.852935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63db1d-63bf-4acb-b2f7-51800550391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:09.853960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63db1d-63bf-4acb-b2f7-51800550391d None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:09.854243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63db1d-63bf-4acb-b2f7-51800550391d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:09.854547 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63db1d-63bf-4acb-b2f7-51800550391d None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151/groups/5a9c1d404b51489eb1e36f1cc3a722de/roles/666cfd1181184612ba462b8eefcad6a7` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:09.856724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63db1d-63bf-4acb-b2f7-51800550391d None admin] RBAC: Authorizing `identity:create_grant(project_id=ba315bae61f943debd8460a5fa9dd151, group_id=5a9c1d404b51489eb1e36f1cc3a722de, role_id=666cfd1181184612ba462b8eefcad6a7)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:09.858150 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63db1d-63bf-4acb-b2f7-51800550391d None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:09.878455 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 197/391] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 20:18:09 2022] PUT /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151/groups/5a9c1d404b51489eb1e36f1cc3a722de/roles/666cfd1181184612ba462b8eefcad6a7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:18:09.879508 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63db1d-63bf-4acb-b2f7-51800550391d None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:09.896620 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 195/392] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:09.922932 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:09.939865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c90e6c-a541-444a-8aab-79106f1156aa None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:10.078525 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63db1d-63bf-4acb-b2f7-51800550391d None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:10.126855 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a29ff6-61f2-4643-8f35-bc639f9f401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:10.127744 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a29ff6-61f2-4643-8f35-bc639f9f401d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:10.128037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a29ff6-61f2-4643-8f35-bc639f9f401d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:10.128325 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a29ff6-61f2-4643-8f35-bc639f9f401d None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:10.129350 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a29ff6-61f2-4643-8f35-bc639f9f401d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:10.129662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a29ff6-61f2-4643-8f35-bc639f9f401d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:10.131661 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a29ff6-61f2-4643-8f35-bc639f9f401d None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:10.141618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c90e6c-a541-444a-8aab-79106f1156aa None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:10.146293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 198/393] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:10.153658 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a29ff6-61f2-4643-8f35-bc639f9f401d None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:10.194682 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea9914-a600-487c-940e-3e1fbdc0acd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:10.195558 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea9914-a600-487c-940e-3e1fbdc0acd5 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:10.195762 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea9914-a600-487c-940e-3e1fbdc0acd5 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:10.196120 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea9914-a600-487c-940e-3e1fbdc0acd5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:10.208300 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea9914-a600-487c-940e-3e1fbdc0acd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-62ea9914-a600-487c-940e-3e1fbdc0acd5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:10.222436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:10.223765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:10.224968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:10.224968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:10.224968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:10.224968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:10.224968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:10.224968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:10.224968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:10.224968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:10.224968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:10.224968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:10.224968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:18:10.224968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:10.224968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 196/394] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:10.226356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea9914-a600-487c-940e-3e1fbdc0acd5 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:10.250931 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:10.287505 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:10.322686 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a29ff6-61f2-4643-8f35-bc639f9f401d None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:10.338296 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea9914-a600-487c-940e-3e1fbdc0acd5 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:10.376300 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46309d-1965-471d-a623-e0a58b2ae219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:10.377051 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46309d-1965-471d-a623-e0a58b2ae219 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:10.377270 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46309d-1965-471d-a623-e0a58b2ae219 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:10.377544 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46309d-1965-471d-a623-e0a58b2ae219 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:10.378314 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46309d-1965-471d-a623-e0a58b2ae219 None admin] RBAC: Adding query filter params (name=member) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:10.378623 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46309d-1965-471d-a623-e0a58b2ae219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:10.380013 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46309d-1965-471d-a623-e0a58b2ae219 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:10.391770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f8637-367d-411e-b02a-7d40e48a2fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:10.392653 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f8637-367d-411e-b02a-7d40e48a2fcf None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:10.392935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f8637-367d-411e-b02a-7d40e48a2fcf None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:10.393215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f8637-367d-411e-b02a-7d40e48a2fcf None admin] PATH_INFO: `/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:10.394525 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f8637-367d-411e-b02a-7d40e48a2fcf None admin] RBAC: Authorizing `identity:get_group(group_id=5a9c1d404b51489eb1e36f1cc3a722de)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:10.395997 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f8637-367d-411e-b02a-7d40e48a2fcf None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:10.398152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 199/395] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:18:10 2022] GET /identity/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:10.406357 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 197/396] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:18:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:10.406566 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f8637-367d-411e-b02a-7d40e48a2fcf None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:10.408222 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46309d-1965-471d-a623-e0a58b2ae219 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:10.525652 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f8637-367d-411e-b02a-7d40e48a2fcf None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:10.576293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888524a3-d81b-4a13-9b7b-0e88f9bb9e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:10.577596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888524a3-d81b-4a13-9b7b-0e88f9bb9e4e None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:10.578090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888524a3-d81b-4a13-9b7b-0e88f9bb9e4e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:10.578602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888524a3-d81b-4a13-9b7b-0e88f9bb9e4e None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:10.580024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888524a3-d81b-4a13-9b7b-0e88f9bb9e4e None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:10.581729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888524a3-d81b-4a13-9b7b-0e88f9bb9e4e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:10.590105 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 200/397] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:18:10 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:10.590105 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649f53a-2e19-43af-949b-439d4b640551 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:10.590105 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649f53a-2e19-43af-949b-439d4b640551 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:10.590105 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649f53a-2e19-43af-949b-439d4b640551 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:10.590105 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 201/398] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:10.591987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649f53a-2e19-43af-949b-439d4b640551 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:10.609054 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:10.612619 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:10.638240 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46309d-1965-471d-a623-e0a58b2ae219 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:10.678003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86791690-d802-4581-8982-4fe1fb8327d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:10.679420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86791690-d802-4581-8982-4fe1fb8327d9 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:10.679946 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86791690-d802-4581-8982-4fe1fb8327d9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:10.680429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86791690-d802-4581-8982-4fe1fb8327d9 None admin] PATH_INFO: `/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:10.682094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86791690-d802-4581-8982-4fe1fb8327d9 None admin] RBAC: Authorizing `identity:get_group(group_id=5a9c1d404b51489eb1e36f1cc3a722de)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:10.683911 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86791690-d802-4581-8982-4fe1fb8327d9 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:10.686280 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 198/399] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:18:10 2022] GET /identity/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:10.687835 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98712451-a30c-40b1-9960-ce694bdd1638 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:10.688342 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98712451-a30c-40b1-9960-ce694bdd1638 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:10.688894 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98712451-a30c-40b1-9960-ce694bdd1638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:10.704273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649f53a-2e19-43af-949b-439d4b640551 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:10.708190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-98712451-a30c-40b1-9960-ce694bdd1638 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:10.709630 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-98712451-a30c-40b1-9960-ce694bdd1638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:10.740302 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a0a9a-ec52-496e-bcb3-86910ae297e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:10.741122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a0a9a-ec52-496e-bcb3-86910ae297e1 None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:10.741405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a0a9a-ec52-496e-bcb3-86910ae297e1 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:10.741815 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a0a9a-ec52-496e-bcb3-86910ae297e1 None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151/groups/5a9c1d404b51489eb1e36f1cc3a722de/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:10.744190 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a0a9a-ec52-496e-bcb3-86910ae297e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba315bae61f943debd8460a5fa9dd151, group_id=5a9c1d404b51489eb1e36f1cc3a722de, role_id=6d25238fbc0e4d11bac1bcdc7e50d2e2)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:10.745713 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a0a9a-ec52-496e-bcb3-86910ae297e1 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:10.770300 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 202/400] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 20:18:10 2022] PUT /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151/groups/5a9c1d404b51489eb1e36f1cc3a722de/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:18:10.770300 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a0a9a-ec52-496e-bcb3-86910ae297e1 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:10.817164 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 199/401] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:10.853489 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98712451-a30c-40b1-9960-ce694bdd1638 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:10.881665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:10.909159 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a0a9a-ec52-496e-bcb3-86910ae297e1 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:10.947076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae521863-b945-4d05-ba2f-4ae85f69fa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:10.948334 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae521863-b945-4d05-ba2f-4ae85f69fa41 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:10.948781 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae521863-b945-4d05-ba2f-4ae85f69fa41 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:10.949208 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae521863-b945-4d05-ba2f-4ae85f69fa41 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:10.950680 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae521863-b945-4d05-ba2f-4ae85f69fa41 None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:10.952381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae521863-b945-4d05-ba2f-4ae85f69fa41 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:10.954819 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 203/402] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:18:10 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:10.960670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae521863-b945-4d05-ba2f-4ae85f69fa41 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:10.990960 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:11.016037 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98712451-a30c-40b1-9960-ce694bdd1638 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:11.057784 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaeb7f7-3bcc-4883-a569-0acc0238887d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:11.058842 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaeb7f7-3bcc-4883-a569-0acc0238887d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:11.059252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaeb7f7-3bcc-4883-a569-0acc0238887d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:11.059615 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaeb7f7-3bcc-4883-a569-0acc0238887d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:11.072887 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaeb7f7-3bcc-4883-a569-0acc0238887d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:11.078851 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae521863-b945-4d05-ba2f-4ae85f69fa41 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-0eaeb7f7-3bcc-4883-a569-0acc0238887d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:11.081776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:11.082997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:11.084311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:11.084311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:11.084311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:11.084311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:11.084311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:11.084311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:11.084311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:11.084311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:11.084311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:11.084311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:11.084311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:18:11.084311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:11.084311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 200/403] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:18:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:11.085105 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2216f8-44b3-4821-8028-0c5da5156db9 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:11.085105 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2216f8-44b3-4821-8028-0c5da5156db9 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:11.085252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2216f8-44b3-4821-8028-0c5da5156db9 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:11.090879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 201/404] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:11.092983 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2216f8-44b3-4821-8028-0c5da5156db9 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:11.124631 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4c1f5-a7e1-43c7-bde9-66c67d8ea786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:11.124631 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4c1f5-a7e1-43c7-bde9-66c67d8ea786 None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:11.124631 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4c1f5-a7e1-43c7-bde9-66c67d8ea786 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:11.124631 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4c1f5-a7e1-43c7-bde9-66c67d8ea786 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0/groups/5a9c1d404b51489eb1e36f1cc3a722de/roles/666cfd1181184612ba462b8eefcad6a7` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:11.125853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4c1f5-a7e1-43c7-bde9-66c67d8ea786 None admin] RBAC: Authorizing `identity:create_grant(project_id=3022d4bdd09249328ba6a2f60404faf0, group_id=5a9c1d404b51489eb1e36f1cc3a722de, role_id=666cfd1181184612ba462b8eefcad6a7)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:11.128305 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4c1f5-a7e1-43c7-bde9-66c67d8ea786 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:11.153540 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 204/405] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 20:18:10 2022] PUT /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0/groups/5a9c1d404b51489eb1e36f1cc3a722de/roles/666cfd1181184612ba462b8eefcad6a7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:18:11.161782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd199f-aad0-433c-b1f2-a065c70fefaa None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:11.162132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd199f-aad0-433c-b1f2-a065c70fefaa None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:11.162694 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd199f-aad0-433c-b1f2-a065c70fefaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:11.173117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:11.191906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-addd199f-aad0-433c-b1f2-a065c70fefaa None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:11.193225 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-addd199f-aad0-433c-b1f2-a065c70fefaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:11.228455 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2216f8-44b3-4821-8028-0c5da5156db9 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:11.272216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf342b-eb3c-46c5-9c59-77ff8f3b149f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:11.272907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf342b-eb3c-46c5-9c59-77ff8f3b149f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:11.273237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf342b-eb3c-46c5-9c59-77ff8f3b149f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:11.273799 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf342b-eb3c-46c5-9c59-77ff8f3b149f None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:11.275128 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf342b-eb3c-46c5-9c59-77ff8f3b149f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:11.275688 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf342b-eb3c-46c5-9c59-77ff8f3b149f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:11.277125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf342b-eb3c-46c5-9c59-77ff8f3b149f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:11.289096 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 202/406] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:11.296032 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf342b-eb3c-46c5-9c59-77ff8f3b149f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:11.300077 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:11.373160 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 205/407] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:11.410629 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd199f-aad0-433c-b1f2-a065c70fefaa None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:11.425628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf342b-eb3c-46c5-9c59-77ff8f3b149f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:11.468340 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008f637-8bf3-45d5-b22b-abfba1df354f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:11.469334 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008f637-8bf3-45d5-b22b-abfba1df354f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:11.469685 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008f637-8bf3-45d5-b22b-abfba1df354f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:11.469965 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008f637-8bf3-45d5-b22b-abfba1df354f None admin] PATH_INFO: `/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:11.471277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008f637-8bf3-45d5-b22b-abfba1df354f None admin] RBAC: Authorizing `identity:get_group(group_id=5a9c1d404b51489eb1e36f1cc3a722de)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:11.472774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008f637-8bf3-45d5-b22b-abfba1df354f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:11.474933 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 203/408] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:18:11 2022] GET /identity/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:11.481040 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008f637-8bf3-45d5-b22b-abfba1df354f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:11.570830 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:11.582310 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd199f-aad0-433c-b1f2-a065c70fefaa None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:11.629425 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008f637-8bf3-45d5-b22b-abfba1df354f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:11.649757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254edf5-76ef-475b-8676-c8342c8a99e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:11.650694 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254edf5-76ef-475b-8676-c8342c8a99e8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:11.650979 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254edf5-76ef-475b-8676-c8342c8a99e8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:11.651263 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254edf5-76ef-475b-8676-c8342c8a99e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:11.654871 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:11.674496 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254edf5-76ef-475b-8676-c8342c8a99e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-b254edf5-76ef-475b-8676-c8342c8a99e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:11.685465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:11.686754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:11.688123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:11.688123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:11.688123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:11.688123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:11.688123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:11.688123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:11.688123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:11.688123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:11.688123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:11.688123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:11.688123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:11.688123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:11.688123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 206/409] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:18:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:11.693462 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254edf5-76ef-475b-8676-c8342c8a99e8 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:11.703866 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90154e8-c047-45ca-b56e-5da397e976e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:11.704758 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90154e8-c047-45ca-b56e-5da397e976e9 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:11.705042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90154e8-c047-45ca-b56e-5da397e976e9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:11.710193 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90154e8-c047-45ca-b56e-5da397e976e9 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:11.711622 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90154e8-c047-45ca-b56e-5da397e976e9 None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:11.713223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90154e8-c047-45ca-b56e-5da397e976e9 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:11.715588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 204/410] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:18:11 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:11.722691 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90154e8-c047-45ca-b56e-5da397e976e9 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:11.882438 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254edf5-76ef-475b-8676-c8342c8a99e8 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:11.890310 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90154e8-c047-45ca-b56e-5da397e976e9 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:11.920337 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb545b66-8e10-4231-ba6c-7b72bbf867e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:11.921026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb545b66-8e10-4231-ba6c-7b72bbf867e4 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:11.921314 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb545b66-8e10-4231-ba6c-7b72bbf867e4 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:11.921598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb545b66-8e10-4231-ba6c-7b72bbf867e4 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:11.922403 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb545b66-8e10-4231-ba6c-7b72bbf867e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:11.922714 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb545b66-8e10-4231-ba6c-7b72bbf867e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:11.924265 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb545b66-8e10-4231-ba6c-7b72bbf867e4 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:11.937652 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 207/411] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:11.944271 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb545b66-8e10-4231-ba6c-7b72bbf867e4 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:11.952451 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b0d93-39eb-4ad4-b0a0-81fbe42a13f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:11.953562 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b0d93-39eb-4ad4-b0a0-81fbe42a13f1 None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:11.954013 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b0d93-39eb-4ad4-b0a0-81fbe42a13f1 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:11.954473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b0d93-39eb-4ad4-b0a0-81fbe42a13f1 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0/groups/5a9c1d404b51489eb1e36f1cc3a722de/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:11.956860 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b0d93-39eb-4ad4-b0a0-81fbe42a13f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=3022d4bdd09249328ba6a2f60404faf0, group_id=5a9c1d404b51489eb1e36f1cc3a722de, role_id=6d25238fbc0e4d11bac1bcdc7e50d2e2)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:11.958523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b0d93-39eb-4ad4-b0a0-81fbe42a13f1 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:11.970125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:11.975943 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 205/412] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 20:18:11 2022] PUT /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0/groups/5a9c1d404b51489eb1e36f1cc3a722de/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:18:12.055763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:12.114709 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb545b66-8e10-4231-ba6c-7b72bbf867e4 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:12.153281 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ed1e9-b9b6-4dbf-ad7c-2aac9268faa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:12.154203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ed1e9-b9b6-4dbf-ad7c-2aac9268faa9 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:12.154518 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ed1e9-b9b6-4dbf-ad7c-2aac9268faa9 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:12.154799 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ed1e9-b9b6-4dbf-ad7c-2aac9268faa9 None admin] PATH_INFO: `/v3/groups/b9d9d93453fd4f4d9de67be02d52ac4d` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:12.156075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ed1e9-b9b6-4dbf-ad7c-2aac9268faa9 None admin] RBAC: Authorizing `identity:get_group(group_id=b9d9d93453fd4f4d9de67be02d52ac4d)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:12.157604 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ed1e9-b9b6-4dbf-ad7c-2aac9268faa9 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:12.159662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 208/413] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:18:11 2022] GET /identity/v3/groups/b9d9d93453fd4f4d9de67be02d52ac4d => generated 229 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:12.165336 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b0d93-39eb-4ad4-b0a0-81fbe42a13f1 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:12.290831 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b0d93-39eb-4ad4-b0a0-81fbe42a13f1 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:12.327487 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885fe60-532a-42c5-8588-ef711c410bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:12.328392 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885fe60-532a-42c5-8588-ef711c410bb1 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:12.328674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885fe60-532a-42c5-8588-ef711c410bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:12.328950 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885fe60-532a-42c5-8588-ef711c410bb1 None admin] PATH_INFO: `/v3/projects/8fd7be0b818c4867be2daed29f53ab77` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:12.330172 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885fe60-532a-42c5-8588-ef711c410bb1 None admin] RBAC: Authorizing `identity:get_project(project_id=8fd7be0b818c4867be2daed29f53ab77)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:12.331687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885fe60-532a-42c5-8588-ef711c410bb1 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:12.333719 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 206/414] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:18:12 2022] GET /identity/v3/projects/8fd7be0b818c4867be2daed29f53ab77 => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:12.340224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ed1e9-b9b6-4dbf-ad7c-2aac9268faa9 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:12.479140 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ed1e9-b9b6-4dbf-ad7c-2aac9268faa9 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:12.506780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:12.515723 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1d304-6e99-4b4b-8242-c78f1310e3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:12.516643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1d304-6e99-4b4b-8242-c78f1310e3f7 None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:12.516930 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1d304-6e99-4b4b-8242-c78f1310e3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:12.517259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1d304-6e99-4b4b-8242-c78f1310e3f7 None admin] PATH_INFO: `/v3/projects/8fd7be0b818c4867be2daed29f53ab77/groups/b9d9d93453fd4f4d9de67be02d52ac4d/roles/b239f941cb724331bf28dc489b70ad1d` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:12.519989 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1d304-6e99-4b4b-8242-c78f1310e3f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fd7be0b818c4867be2daed29f53ab77, group_id=b9d9d93453fd4f4d9de67be02d52ac4d, role_id=b239f941cb724331bf28dc489b70ad1d)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:12.522057 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1d304-6e99-4b4b-8242-c78f1310e3f7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:12.539116 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 209/415] 10.222.0.84 () {58 vars in 1442 bytes} [Thu May 19 20:18:12 2022] PUT /identity/v3/projects/8fd7be0b818c4867be2daed29f53ab77/groups/b9d9d93453fd4f4d9de67be02d52ac4d/roles/b239f941cb724331bf28dc489b70ad1d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:18:12.888568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a636a8-1748-48e7-aed6-172a93a75ef8 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:12.889047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a636a8-1748-48e7-aed6-172a93a75ef8 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:12.889340 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a636a8-1748-48e7-aed6-172a93a75ef8 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:12.891049 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 207/416] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:12.897505 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a8a14-467e-4f14-8fd0-18375c9f7942 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:12.897950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a8a14-467e-4f14-8fd0-18375c9f7942 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:12.898139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a8a14-467e-4f14-8fd0-18375c9f7942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:12.916281 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-9f2a8a14-467e-4f14-8fd0-18375c9f7942 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:12.917500 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-9f2a8a14-467e-4f14-8fd0-18375c9f7942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:12.991460 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 210/417] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:13.015620 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a636a8-1748-48e7-aed6-172a93a75ef8 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:13.105643 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:13.150734 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a636a8-1748-48e7-aed6-172a93a75ef8 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:13.185932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5105a39-cbff-43db-9f38-9056c827062e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:13.186782 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5105a39-cbff-43db-9f38-9056c827062e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:13.187068 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5105a39-cbff-43db-9f38-9056c827062e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:13.187347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5105a39-cbff-43db-9f38-9056c827062e None admin] PATH_INFO: `/v3/roles/member` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:13.199069 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5105a39-cbff-43db-9f38-9056c827062e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-b5105a39-cbff-43db-9f38-9056c827062e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:13.207103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:13.208121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:13.209046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:13.209046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:13.209046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:13.209046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:13.209046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:13.209046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:13.209046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:13.209046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:13.209046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:13.209046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:13.209046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:18:13.209046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:13.209046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 208/418] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:13.215075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a8a14-467e-4f14-8fd0-18375c9f7942 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:13.323222 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a8a14-467e-4f14-8fd0-18375c9f7942 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:13.340809 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:13.363184 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2fc81-b61d-4278-8348-9e7a2c133dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:13.363871 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2fc81-b61d-4278-8348-9e7a2c133dbd None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:13.364157 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2fc81-b61d-4278-8348-9e7a2c133dbd None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:13.364438 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2fc81-b61d-4278-8348-9e7a2c133dbd None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:13.365200 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2fc81-b61d-4278-8348-9e7a2c133dbd None admin] RBAC: Adding query filter params (name=member) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:13.365507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2fc81-b61d-4278-8348-9e7a2c133dbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:13.366979 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2fc81-b61d-4278-8348-9e7a2c133dbd None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:13.376747 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 211/419] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:18:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:13.382426 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5105a39-cbff-43db-9f38-9056c827062e None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:13.535546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5105a39-cbff-43db-9f38-9056c827062e None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:13.547340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a34f32-c020-4104-b53e-1adbc8740469 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:13.547340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a34f32-c020-4104-b53e-1adbc8740469 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:13.547340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a34f32-c020-4104-b53e-1adbc8740469 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:13.547340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 212/420] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:13.553300 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21a72c-4dbf-4e20-b96c-75550d48053f None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:13.553584 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21a72c-4dbf-4e20-b96c-75550d48053f None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:13.553861 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21a72c-4dbf-4e20-b96c-75550d48053f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:13.575507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:13.584172 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a6e95-a82a-4e9a-8148-691881e9ff83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:13.585131 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a6e95-a82a-4e9a-8148-691881e9ff83 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:13.585648 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a6e95-a82a-4e9a-8148-691881e9ff83 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:13.586139 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a6e95-a82a-4e9a-8148-691881e9ff83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:13.603878 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-ff21a72c-4dbf-4e20-b96c-75550d48053f None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:13.605462 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-ff21a72c-4dbf-4e20-b96c-75550d48053f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:13.719536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a6e95-a82a-4e9a-8148-691881e9ff83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:13.720924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a6e95-a82a-4e9a-8148-691881e9ff83 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:13.735130 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 213/421] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:18:13.767483 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21a72c-4dbf-4e20-b96c-75550d48053f None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:13.903886 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 209/422] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:13.909481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a6e95-a82a-4e9a-8148-691881e9ff83 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:13.940698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21a72c-4dbf-4e20-b96c-75550d48053f None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:13.954149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:13.961517 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:14.013382 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c2acf-9a36-4860-a50b-e77a59a4c7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:14.014375 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c2acf-9a36-4860-a50b-e77a59a4c7e8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:14.014660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c2acf-9a36-4860-a50b-e77a59a4c7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:14.014939 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c2acf-9a36-4860-a50b-e77a59a4c7e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:14.035868 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c2acf-9a36-4860-a50b-e77a59a4c7e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-b85c2acf-9a36-4860-a50b-e77a59a4c7e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:14.054372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:14.055475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:14.056490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:14.056490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:14.056490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:14.056490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:14.056490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:14.056490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:14.056490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:14.056490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:14.056490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:14.056490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:14.056490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:18:14.056490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:14.057195 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 214/423] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:18:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:14.063212 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c2acf-9a36-4860-a50b-e77a59a4c7e8 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:14.080805 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a6e95-a82a-4e9a-8148-691881e9ff83 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:14.119695 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073a4ce-6e0c-4902-b1b2-258af430e788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:14.121042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073a4ce-6e0c-4902-b1b2-258af430e788 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:14.121482 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073a4ce-6e0c-4902-b1b2-258af430e788 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:14.121929 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073a4ce-6e0c-4902-b1b2-258af430e788 None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:14.123502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073a4ce-6e0c-4902-b1b2-258af430e788 None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:14.125257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073a4ce-6e0c-4902-b1b2-258af430e788 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:14.127776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 210/424] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:18:13 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:14.133735 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073a4ce-6e0c-4902-b1b2-258af430e788 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:14.242478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c2acf-9a36-4860-a50b-e77a59a4c7e8 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:14.259297 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:14.267474 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073a4ce-6e0c-4902-b1b2-258af430e788 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:14.281571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fe68c-b491-413c-a0d4-007063204a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:14.282460 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fe68c-b491-413c-a0d4-007063204a06 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:14.282844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fe68c-b491-413c-a0d4-007063204a06 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:14.283220 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fe68c-b491-413c-a0d4-007063204a06 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:14.284077 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fe68c-b491-413c-a0d4-007063204a06 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:14.284479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fe68c-b491-413c-a0d4-007063204a06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:14.285977 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fe68c-b491-413c-a0d4-007063204a06 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:14.295196 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 215/425] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:14.296688 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce7d7e-36a0-48b1-aa8f-d9556f0800af None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:14.297077 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce7d7e-36a0-48b1-aa8f-d9556f0800af None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:14.297473 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce7d7e-36a0-48b1-aa8f-d9556f0800af None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:14.298831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 216/426] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:14.303877 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce7d7e-36a0-48b1-aa8f-d9556f0800af None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:14.306759 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354490c-d46d-442d-9db1-66a285deaa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:14.307793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354490c-d46d-442d-9db1-66a285deaa2d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:14.308175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354490c-d46d-442d-9db1-66a285deaa2d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:14.308453 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354490c-d46d-442d-9db1-66a285deaa2d None admin] PATH_INFO: `/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:14.309889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354490c-d46d-442d-9db1-66a285deaa2d None admin] RBAC: Authorizing `identity:get_group(group_id=5a9c1d404b51489eb1e36f1cc3a722de)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:14.311391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354490c-d46d-442d-9db1-66a285deaa2d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:14.313378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 211/427] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:18:14 2022] GET /identity/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:14.314851 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbacae7-ee26-476f-98d3-029629db8ab7 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:14.315133 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbacae7-ee26-476f-98d3-029629db8ab7 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:14.315412 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbacae7-ee26-476f-98d3-029629db8ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:14.333950 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-5cbacae7-ee26-476f-98d3-029629db8ab7 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:14.334673 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:14.335821 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-5cbacae7-ee26-476f-98d3-029629db8ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:14.412455 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce7d7e-36a0-48b1-aa8f-d9556f0800af None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:14.412796 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 212/428] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:14.413304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbacae7-ee26-476f-98d3-029629db8ab7 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:14.463097 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e7841-4ac9-4229-86ea-f7db0e04d99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:14.464349 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e7841-4ac9-4229-86ea-f7db0e04d99f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:14.464786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e7841-4ac9-4229-86ea-f7db0e04d99f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:14.465235 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e7841-4ac9-4229-86ea-f7db0e04d99f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:14.515641 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:14.526274 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbacae7-ee26-476f-98d3-029629db8ab7 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:14.541022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e7841-4ac9-4229-86ea-f7db0e04d99f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:14.542936 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e7841-4ac9-4229-86ea-f7db0e04d99f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:14.563365 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6064301-0a31-44dd-be34-92fad8120bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:14.564371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6064301-0a31-44dd-be34-92fad8120bb5 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:14.564883 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6064301-0a31-44dd-be34-92fad8120bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:14.565454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6064301-0a31-44dd-be34-92fad8120bb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:14.566474 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6064301-0a31-44dd-be34-92fad8120bb5 None admin] RBAC: Adding query filter params (group.id=5a9c1d404b51489eb1e36f1cc3a722de, role.id=666cfd1181184612ba462b8eefcad6a7, scope.project.id=ba315bae61f943debd8460a5fa9dd151) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:14.567099 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6064301-0a31-44dd-be34-92fad8120bb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:14.569024 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6064301-0a31-44dd-be34-92fad8120bb5 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:14.572079 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:14.576904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 213/429] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:18:14 2022] GET /identity/v3/role_assignments?group.id=5a9c1d404b51489eb1e36f1cc3a722de&scope.project.id=ba315bae61f943debd8460a5fa9dd151&role.id=666cfd1181184612ba462b8eefcad6a7 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:14.578118 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6064301-0a31-44dd-be34-92fad8120bb5 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:14.685328 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 217/430] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:14.690756 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e7841-4ac9-4229-86ea-f7db0e04d99f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:14.768365 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6064301-0a31-44dd-be34-92fad8120bb5 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:14.828126 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb26c4-561d-4460-a61d-47d0e3db769d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:14.829321 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb26c4-561d-4460-a61d-47d0e3db769d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:14.829772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb26c4-561d-4460-a61d-47d0e3db769d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:14.830229 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb26c4-561d-4460-a61d-47d0e3db769d None admin] PATH_INFO: `/v3/roles/member` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:14.846020 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb26c4-561d-4460-a61d-47d0e3db769d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:14.850891 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e7841-4ac9-4229-86ea-f7db0e04d99f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-89eb26c4-561d-4460-a61d-47d0e3db769d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:14.855057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:14.856372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:14.857335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:14.857335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:14.857335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:14.857335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:14.857335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:14.857335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:14.857335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:14.857335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:14.857335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:14.857335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:14.857335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:18:14.857335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:14.860105 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 214/431] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:14.868196 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb26c4-561d-4460-a61d-47d0e3db769d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:14.894915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:14.907590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:14.917942 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aaa29d-1a22-4c5d-a4ff-fcd9eb8a7539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:14.918853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aaa29d-1a22-4c5d-a4ff-fcd9eb8a7539 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:14.919151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aaa29d-1a22-4c5d-a4ff-fcd9eb8a7539 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:14.919418 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aaa29d-1a22-4c5d-a4ff-fcd9eb8a7539 None admin] PATH_INFO: `/v3/projects/ba315bae61f943debd8460a5fa9dd151` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:14.921554 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aaa29d-1a22-4c5d-a4ff-fcd9eb8a7539 None admin] RBAC: Authorizing `identity:get_project(project_id=ba315bae61f943debd8460a5fa9dd151)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:14.923134 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aaa29d-1a22-4c5d-a4ff-fcd9eb8a7539 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:14.928355 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 218/432] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:18:14 2022] GET /identity/v3/projects/ba315bae61f943debd8460a5fa9dd151 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:14.928355 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad4b9d-d0a3-4366-8448-440a16bf9ff7 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:14.928355 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad4b9d-d0a3-4366-8448-440a16bf9ff7 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:14.928355 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad4b9d-d0a3-4366-8448-440a16bf9ff7 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:14.928737 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 219/433] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:14.934440 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad4b9d-d0a3-4366-8448-440a16bf9ff7 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:15.009823 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb26c4-561d-4460-a61d-47d0e3db769d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:15.056012 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0705cc-b188-48bd-b01d-7492cd9fb1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:15.057015 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0705cc-b188-48bd-b01d-7492cd9fb1b7 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:15.057873 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0705cc-b188-48bd-b01d-7492cd9fb1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:15.058686 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0705cc-b188-48bd-b01d-7492cd9fb1b7 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:15.059932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0705cc-b188-48bd-b01d-7492cd9fb1b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:15.060435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0705cc-b188-48bd-b01d-7492cd9fb1b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:15.062517 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0705cc-b188-48bd-b01d-7492cd9fb1b7 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:15.070903 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad4b9d-d0a3-4366-8448-440a16bf9ff7 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:15.073727 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 215/434] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:18:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:15.075562 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b77ff-7a62-4005-b56c-12d419085036 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:15.075859 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b77ff-7a62-4005-b56c-12d419085036 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:15.076141 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b77ff-7a62-4005-b56c-12d419085036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:15.097503 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-f80b77ff-7a62-4005-b56c-12d419085036 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:15.098662 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-f80b77ff-7a62-4005-b56c-12d419085036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:15.113677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3d472-5e75-4d1e-9f1e-6c621654c67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:15.114607 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3d472-5e75-4d1e-9f1e-6c621654c67e None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:15.114888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3d472-5e75-4d1e-9f1e-6c621654c67e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:15.115208 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3d472-5e75-4d1e-9f1e-6c621654c67e None admin] PATH_INFO: `/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:15.116442 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3d472-5e75-4d1e-9f1e-6c621654c67e None admin] RBAC: Authorizing `identity:get_group(group_id=5a9c1d404b51489eb1e36f1cc3a722de)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:15.118031 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3d472-5e75-4d1e-9f1e-6c621654c67e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:15.120100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 220/435] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:18:14 2022] GET /identity/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:15.121052 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3d472-5e75-4d1e-9f1e-6c621654c67e None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:15.176971 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 216/436] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:15.177878 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b77ff-7a62-4005-b56c-12d419085036 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:15.191104 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:15.202952 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:15.247161 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3d472-5e75-4d1e-9f1e-6c621654c67e None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:15.284089 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6537265-266d-4aed-86d5-fec0481f9a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:15.284780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6537265-266d-4aed-86d5-fec0481f9a20 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:15.285262 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6537265-266d-4aed-86d5-fec0481f9a20 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:15.285399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6537265-266d-4aed-86d5-fec0481f9a20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:15.288308 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b77ff-7a62-4005-b56c-12d419085036 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:15.324411 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e08df6-518e-4ebf-98bc-7a69c380f073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:15.325076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e08df6-518e-4ebf-98bc-7a69c380f073 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:15.325352 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e08df6-518e-4ebf-98bc-7a69c380f073 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:15.325620 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e08df6-518e-4ebf-98bc-7a69c380f073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:15.326476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e08df6-518e-4ebf-98bc-7a69c380f073 None admin] RBAC: Adding query filter params (group.id=5a9c1d404b51489eb1e36f1cc3a722de, role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2, scope.project.id=ba315bae61f943debd8460a5fa9dd151) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:15.326782 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e08df6-518e-4ebf-98bc-7a69c380f073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:15.328212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e08df6-518e-4ebf-98bc-7a69c380f073 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:15.334341 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 217/437] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:18:15 2022] GET /identity/v3/role_assignments?group.id=5a9c1d404b51489eb1e36f1cc3a722de&scope.project.id=ba315bae61f943debd8460a5fa9dd151&role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:15.335389 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e08df6-518e-4ebf-98bc-7a69c380f073 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:15.368153 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6537265-266d-4aed-86d5-fec0481f9a20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:15.371035 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6537265-266d-4aed-86d5-fec0481f9a20 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:15.416921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:15.448691 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:15.459189 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e08df6-518e-4ebf-98bc-7a69c380f073 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:15.497819 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 221/438] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:15.499329 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db101bb6-c6c0-4394-8683-597d5de3c4f0 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:15.499617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db101bb6-c6c0-4394-8683-597d5de3c4f0 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:15.499899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db101bb6-c6c0-4394-8683-597d5de3c4f0 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:15.501357 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 222/439] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:15.503403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b18111-ccd4-47a3-8a92-53ff1b556933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:15.504359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b18111-ccd4-47a3-8a92-53ff1b556933 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:15.504712 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b18111-ccd4-47a3-8a92-53ff1b556933 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:15.505059 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b18111-ccd4-47a3-8a92-53ff1b556933 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:15.506451 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db101bb6-c6c0-4394-8683-597d5de3c4f0 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:15.517716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b18111-ccd4-47a3-8a92-53ff1b556933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-d1b18111-ccd4-47a3-8a92-53ff1b556933 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:15.526395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:15.527481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:15.528408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:15.528408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:15.528408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:15.528408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:15.528408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:15.528408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:15.528408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:15.528408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:15.528408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:15.528408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:15.528408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:18:15.528408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:15.528408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 218/440] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:18:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:15.529182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeb348-40bb-4778-89f9-0e0cf4c4003f None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:15.529182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeb348-40bb-4778-89f9-0e0cf4c4003f None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:15.530221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeb348-40bb-4778-89f9-0e0cf4c4003f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:15.548694 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-15eeb348-40bb-4778-89f9-0e0cf4c4003f None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:15.549900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-15eeb348-40bb-4778-89f9-0e0cf4c4003f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:15.625557 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db101bb6-c6c0-4394-8683-597d5de3c4f0 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:15.630033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 219/441] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:15.630749 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeb348-40bb-4778-89f9-0e0cf4c4003f None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:15.668597 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9786558-606e-4219-906a-873cdbeef812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:15.670125 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9786558-606e-4219-906a-873cdbeef812 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:15.670447 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9786558-606e-4219-906a-873cdbeef812 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:15.670727 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9786558-606e-4219-906a-873cdbeef812 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:15.671941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9786558-606e-4219-906a-873cdbeef812 None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:15.673414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9786558-606e-4219-906a-873cdbeef812 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:15.675162 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 223/442] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:18:15 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:15.675937 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9786558-606e-4219-906a-873cdbeef812 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:15.704569 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:15.709035 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:15.755107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeb348-40bb-4778-89f9-0e0cf4c4003f None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:15.790635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8989b-22d8-4d02-99e8-501fd9038688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:15.790856 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9786558-606e-4219-906a-873cdbeef812 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:15.791312 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8989b-22d8-4d02-99e8-501fd9038688 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:15.791598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8989b-22d8-4d02-99e8-501fd9038688 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:15.791877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8989b-22d8-4d02-99e8-501fd9038688 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:15.792632 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe8989b-22d8-4d02-99e8-501fd9038688 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:15.792934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe8989b-22d8-4d02-99e8-501fd9038688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:15.794359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe8989b-22d8-4d02-99e8-501fd9038688 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:15.802858 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 220/443] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:15.803644 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8989b-22d8-4d02-99e8-501fd9038688 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:15.827537 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc50e3-4e27-4d1c-8fcd-cc30373c7d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:15.828340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc50e3-4e27-4d1c-8fcd-cc30373c7d1a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:15.828627 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc50e3-4e27-4d1c-8fcd-cc30373c7d1a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:15.828903 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc50e3-4e27-4d1c-8fcd-cc30373c7d1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:15.840383 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc50e3-4e27-4d1c-8fcd-cc30373c7d1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-6dbc50e3-4e27-4d1c-8fcd-cc30373c7d1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:15.848063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:15.849090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:15.850023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:15.850023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:15.850023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:15.850023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:15.850023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:15.850023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:15.850023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:15.850023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:15.850023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:15.850023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:15.850023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:15.850023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:15.854285 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 224/444] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:18:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:15.854993 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc50e3-4e27-4d1c-8fcd-cc30373c7d1a None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:15.908491 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8989b-22d8-4d02-99e8-501fd9038688 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:15.932885 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:15.944402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad9ae3-942e-401e-953a-1ad55d3fd347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:15.945274 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad9ae3-942e-401e-953a-1ad55d3fd347 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:15.945557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad9ae3-942e-401e-953a-1ad55d3fd347 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:15.945835 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad9ae3-942e-401e-953a-1ad55d3fd347 None admin] PATH_INFO: `/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:15.946604 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:15.947661 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad9ae3-942e-401e-953a-1ad55d3fd347 None admin] RBAC: Authorizing `identity:get_group(group_id=5a9c1d404b51489eb1e36f1cc3a722de)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:15.949310 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad9ae3-942e-401e-953a-1ad55d3fd347 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:15.951030 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 221/445] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:18:15 2022] GET /identity/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:15.956266 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad9ae3-942e-401e-953a-1ad55d3fd347 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:15.956430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc50e3-4e27-4d1c-8fcd-cc30373c7d1a None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:15.991886 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def70601-bd6f-44d2-9f3e-1bc3f3073134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:15.992537 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def70601-bd6f-44d2-9f3e-1bc3f3073134 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:15.992808 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def70601-bd6f-44d2-9f3e-1bc3f3073134 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:15.993128 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def70601-bd6f-44d2-9f3e-1bc3f3073134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:16.052339 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad9ae3-942e-401e-953a-1ad55d3fd347 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:16.062286 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def70601-bd6f-44d2-9f3e-1bc3f3073134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:16.063804 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def70601-bd6f-44d2-9f3e-1bc3f3073134 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:16.088488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e4535-2445-4310-8fbc-9ad162c27723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:16.089476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e4535-2445-4310-8fbc-9ad162c27723 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:16.089760 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e4535-2445-4310-8fbc-9ad162c27723 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:16.090019 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e4535-2445-4310-8fbc-9ad162c27723 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:16.090930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e4535-2445-4310-8fbc-9ad162c27723 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:16.091244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e4535-2445-4310-8fbc-9ad162c27723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:16.092619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e4535-2445-4310-8fbc-9ad162c27723 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:16.100833 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 222/446] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:16.101622 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e4535-2445-4310-8fbc-9ad162c27723 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:16.154752 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:16.159693 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:16.165397 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 225/447] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:16.166219 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def70601-bd6f-44d2-9f3e-1bc3f3073134 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:16.204660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e4535-2445-4310-8fbc-9ad162c27723 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:16.238019 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4af73-1787-4693-b0b7-f986bee361f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:16.238711 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4af73-1787-4693-b0b7-f986bee361f5 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:16.238990 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4af73-1787-4693-b0b7-f986bee361f5 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:16.239267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4af73-1787-4693-b0b7-f986bee361f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:16.240091 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4af73-1787-4693-b0b7-f986bee361f5 None admin] RBAC: Adding query filter params (group.id=5a9c1d404b51489eb1e36f1cc3a722de, role.id=666cfd1181184612ba462b8eefcad6a7, scope.project.id=3022d4bdd09249328ba6a2f60404faf0) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:16.240406 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4af73-1787-4693-b0b7-f986bee361f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:16.241874 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4af73-1787-4693-b0b7-f986bee361f5 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:16.247950 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 223/448] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:18:16 2022] GET /identity/v3/role_assignments?group.id=5a9c1d404b51489eb1e36f1cc3a722de&scope.project.id=3022d4bdd09249328ba6a2f60404faf0&role.id=666cfd1181184612ba462b8eefcad6a7 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:16.248759 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4af73-1787-4693-b0b7-f986bee361f5 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:16.286359 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def70601-bd6f-44d2-9f3e-1bc3f3073134 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:16.331447 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f9fe7-60fc-46d6-84a5-c7c431e5d8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:16.332117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f9fe7-60fc-46d6-84a5-c7c431e5d8b8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:16.332395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f9fe7-60fc-46d6-84a5-c7c431e5d8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:16.332672 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f9fe7-60fc-46d6-84a5-c7c431e5d8b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:16.363965 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4af73-1787-4693-b0b7-f986bee361f5 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:16.383308 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:16.402298 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:16.402749 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668621c7-f0b6-4ce4-91ec-b1735dbc9625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:16.403997 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668621c7-f0b6-4ce4-91ec-b1735dbc9625 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:16.404324 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668621c7-f0b6-4ce4-91ec-b1735dbc9625 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:16.404873 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668621c7-f0b6-4ce4-91ec-b1735dbc9625 None admin] PATH_INFO: `/v3/projects/3022d4bdd09249328ba6a2f60404faf0` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:16.406405 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668621c7-f0b6-4ce4-91ec-b1735dbc9625 None admin] RBAC: Authorizing `identity:get_project(project_id=3022d4bdd09249328ba6a2f60404faf0)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:16.407893 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668621c7-f0b6-4ce4-91ec-b1735dbc9625 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:16.410040 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 224/449] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:18:16 2022] GET /identity/v3/projects/3022d4bdd09249328ba6a2f60404faf0 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:16.414454 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f9fe7-60fc-46d6-84a5-c7c431e5d8b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:16.415437 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668621c7-f0b6-4ce4-91ec-b1735dbc9625 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:16.416281 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f9fe7-60fc-46d6-84a5-c7c431e5d8b8 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:16.529822 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668621c7-f0b6-4ce4-91ec-b1735dbc9625 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:16.538864 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 226/450] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:16.545514 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f9fe7-60fc-46d6-84a5-c7c431e5d8b8 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:16.572317 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a04e9f-3f3e-47e2-9a0d-e15a1b822328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:16.573648 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a04e9f-3f3e-47e2-9a0d-e15a1b822328 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:16.574069 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a04e9f-3f3e-47e2-9a0d-e15a1b822328 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:16.574525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a04e9f-3f3e-47e2-9a0d-e15a1b822328 None admin] PATH_INFO: `/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:16.576283 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a04e9f-3f3e-47e2-9a0d-e15a1b822328 None admin] RBAC: Authorizing `identity:get_group(group_id=5a9c1d404b51489eb1e36f1cc3a722de)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:16.578679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a04e9f-3f3e-47e2-9a0d-e15a1b822328 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:16.581184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 225/451] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:18:16 2022] GET /identity/v3/groups/5a9c1d404b51489eb1e36f1cc3a722de => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:16.591841 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a04e9f-3f3e-47e2-9a0d-e15a1b822328 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:16.654252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:16.665828 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:16.666344 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f9fe7-60fc-46d6-84a5-c7c431e5d8b8 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:16.701597 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d80127-7c4e-4fdf-8e3a-e3f9a72150ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:16.702485 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d80127-7c4e-4fdf-8e3a-e3f9a72150ef None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:16.702767 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d80127-7c4e-4fdf-8e3a-e3f9a72150ef None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:16.703041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d80127-7c4e-4fdf-8e3a-e3f9a72150ef None admin] PATH_INFO: `/v3/projects/8fd7be0b818c4867be2daed29f53ab77` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:16.704239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d80127-7c4e-4fdf-8e3a-e3f9a72150ef None admin] RBAC: Authorizing `identity:get_project(project_id=8fd7be0b818c4867be2daed29f53ab77)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:16.705707 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d80127-7c4e-4fdf-8e3a-e3f9a72150ef None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:16.705948 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a04e9f-3f3e-47e2-9a0d-e15a1b822328 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:16.707461 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 227/452] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:18:16 2022] GET /identity/v3/projects/8fd7be0b818c4867be2daed29f53ab77 => generated 344 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:16.713439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d80127-7c4e-4fdf-8e3a-e3f9a72150ef None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:16.740862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc81262-9b61-4c7d-9658-39f6365e6f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:16.741546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc81262-9b61-4c7d-9658-39f6365e6f54 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:16.741832 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc81262-9b61-4c7d-9658-39f6365e6f54 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:16.742108 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc81262-9b61-4c7d-9658-39f6365e6f54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:16.743108 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc81262-9b61-4c7d-9658-39f6365e6f54 None admin] RBAC: Adding query filter params (group.id=5a9c1d404b51489eb1e36f1cc3a722de, role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2, scope.project.id=3022d4bdd09249328ba6a2f60404faf0) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:16.743633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc81262-9b61-4c7d-9658-39f6365e6f54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:16.744917 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc81262-9b61-4c7d-9658-39f6365e6f54 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:16.751088 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 226/453] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:18:16 2022] GET /identity/v3/role_assignments?group.id=5a9c1d404b51489eb1e36f1cc3a722de&scope.project.id=3022d4bdd09249328ba6a2f60404faf0&role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:16.818805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d80127-7c4e-4fdf-8e3a-e3f9a72150ef None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:16.856615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dd9da-f48d-4af2-be26-d1e9a3b226f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:16.857498 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dd9da-f48d-4af2-be26-d1e9a3b226f7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:16.857779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dd9da-f48d-4af2-be26-d1e9a3b226f7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:16.858052 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dd9da-f48d-4af2-be26-d1e9a3b226f7 None admin] PATH_INFO: `/v3/groups/b9d9d93453fd4f4d9de67be02d52ac4d` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:16.859388 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774dd9da-f48d-4af2-be26-d1e9a3b226f7 None admin] RBAC: Authorizing `identity:get_group(group_id=b9d9d93453fd4f4d9de67be02d52ac4d)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:16.860868 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774dd9da-f48d-4af2-be26-d1e9a3b226f7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:16.862567 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 228/454] 10.222.0.84 () {56 vars in 1180 bytes} [Thu May 19 20:18:16 2022] GET /identity/v3/groups/b9d9d93453fd4f4d9de67be02d52ac4d => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:16.869140 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc81262-9b61-4c7d-9658-39f6365e6f54 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:16.979039 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc81262-9b61-4c7d-9658-39f6365e6f54 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:16.996766 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:17.015944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88816d2-4af2-44eb-8d78-8d43580f2fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:17.016615 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88816d2-4af2-44eb-8d78-8d43580f2fb9 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:17.016898 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88816d2-4af2-44eb-8d78-8d43580f2fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:17.017170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88816d2-4af2-44eb-8d78-8d43580f2fb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:17.018015 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88816d2-4af2-44eb-8d78-8d43580f2fb9 None admin] RBAC: Adding query filter params (group.id=b9d9d93453fd4f4d9de67be02d52ac4d, role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=8fd7be0b818c4867be2daed29f53ab77) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:17.018352 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88816d2-4af2-44eb-8d78-8d43580f2fb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:17.019798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88816d2-4af2-44eb-8d78-8d43580f2fb9 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:17.026081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 227/455] 10.222.0.84 () {56 vars in 1376 bytes} [Thu May 19 20:18:16 2022] GET /identity/v3/role_assignments?group.id=b9d9d93453fd4f4d9de67be02d52ac4d&scope.project.id=8fd7be0b818c4867be2daed29f53ab77&role.id=b239f941cb724331bf28dc489b70ad1d => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:19.660002 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b612d-1266-4a13-9439-8f556b562e2d None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:19.660904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b612d-1266-4a13-9439-8f556b562e2d None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:19.661336 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b612d-1266-4a13-9439-8f556b562e2d None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:19.663241 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 229/456] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:19.685752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e81a79-08ee-4b96-8e26-69e006780457 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:19.686906 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e81a79-08ee-4b96-8e26-69e006780457 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:19.687417 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e81a79-08ee-4b96-8e26-69e006780457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:19.710360 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-69e81a79-08ee-4b96-8e26-69e006780457 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:19.712732 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-69e81a79-08ee-4b96-8e26-69e006780457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:19.827877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 228/457] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:18:19.871796 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b612d-1266-4a13-9439-8f556b562e2d None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:19.915822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:20.004088 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b612d-1266-4a13-9439-8f556b562e2d None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:20.055948 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510088f-7f38-41fe-b755-462ffdeb86e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:20.056767 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510088f-7f38-41fe-b755-462ffdeb86e3 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:20.057052 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510088f-7f38-41fe-b755-462ffdeb86e3 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:20.057325 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510088f-7f38-41fe-b755-462ffdeb86e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:20.068377 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510088f-7f38-41fe-b755-462ffdeb86e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:20.069881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510088f-7f38-41fe-b755-462ffdeb86e3 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:20.098059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-3510088f-7f38-41fe-b755-462ffdeb86e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:20.098059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:20.098059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:20.098059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:20.098059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:20.098059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:20.098059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:20.098059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:20.098059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:20.098059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:20.098059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:20.098059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:20.098059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:20.098059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:20.098059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:20.099152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:20.100563 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:20.100563 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:20.100563 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:20.100563 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:20.100563 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 230/458] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:20.107090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e81a79-08ee-4b96-8e26-69e006780457 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:20.122001 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21123ccb-1586-452c-aedd-36561783994f None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:20.122361 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21123ccb-1586-452c-aedd-36561783994f None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:20.122597 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21123ccb-1586-452c-aedd-36561783994f None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:20.124239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 231/459] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:20.129727 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f437e-6e5d-422d-8922-ee047cd4eb42 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:20.130296 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f437e-6e5d-422d-8922-ee047cd4eb42 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:20.130809 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f437e-6e5d-422d-8922-ee047cd4eb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:20.160571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-6a9f437e-6e5d-422d-8922-ee047cd4eb42 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:20.163551 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-6a9f437e-6e5d-422d-8922-ee047cd4eb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:20.307005 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 232/460] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:18:20.312228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e81a79-08ee-4b96-8e26-69e006780457 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:20.350700 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:20.358099 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f437e-6e5d-422d-8922-ee047cd4eb42 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:20.378684 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:20.391947 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814aaf11-b62e-4782-8423-c7ddf50391f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:20.392638 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814aaf11-b62e-4782-8423-c7ddf50391f8 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:20.392927 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814aaf11-b62e-4782-8423-c7ddf50391f8 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:20.393209 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814aaf11-b62e-4782-8423-c7ddf50391f8 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:20.393944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814aaf11-b62e-4782-8423-c7ddf50391f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:20.394253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814aaf11-b62e-4782-8423-c7ddf50391f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:20.395668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814aaf11-b62e-4782-8423-c7ddf50391f8 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:20.413190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 229/461] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:20.415130 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a673ac-d7fc-4922-b6bc-604413b3afae None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:20.415429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a673ac-d7fc-4922-b6bc-604413b3afae None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:20.415827 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a673ac-d7fc-4922-b6bc-604413b3afae None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:20.419744 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 230/462] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:20.423421 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a673ac-d7fc-4922-b6bc-604413b3afae None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:20.497617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f437e-6e5d-422d-8922-ee047cd4eb42 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:20.531686 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a673ac-d7fc-4922-b6bc-604413b3afae None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:20.534354 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecd513-e568-4837-800f-42327595def2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:20.535472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecd513-e568-4837-800f-42327595def2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:20.535964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecd513-e568-4837-800f-42327595def2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:20.536442 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecd513-e568-4837-800f-42327595def2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:20.547821 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecd513-e568-4837-800f-42327595def2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:20.549444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecd513-e568-4837-800f-42327595def2 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:20.567427 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61531f01-0781-4561-91cd-233daa948fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:20.568006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-14ecd513-e568-4837-800f-42327595def2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:20.568006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:20.568006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:20.568006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:20.568006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:20.568006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:20.568006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:20.568006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:20.568006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:20.568006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:20.568006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:20.568006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:20.568006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:20.568006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:20.568006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:20.568964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:20.569989 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:20.569989 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:20.569989 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:20.569989 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:20.570244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61531f01-0781-4561-91cd-233daa948fe4 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:20.570244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61531f01-0781-4561-91cd-233daa948fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:20.570244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61531f01-0781-4561-91cd-233daa948fe4 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:20.570244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61531f01-0781-4561-91cd-233daa948fe4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:20.571073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 233/463] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:20.571151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61531f01-0781-4561-91cd-233daa948fe4 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:20.571249 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d2e3-cefd-46ac-8aa9-ed394f6957ba None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:20.571249 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d2e3-cefd-46ac-8aa9-ed394f6957ba None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:20.571404 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d2e3-cefd-46ac-8aa9-ed394f6957ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:20.589121 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-cb93d2e3-cefd-46ac-8aa9-ed394f6957ba None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:20.590360 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-cb93d2e3-cefd-46ac-8aa9-ed394f6957ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:20.602467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 231/464] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:18:20 2022] POST /identity/v3/users => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:18:20.604035 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31adc83-d373-4c65-adb1-73e03afb86c6 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:20.604323 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31adc83-d373-4c65-adb1-73e03afb86c6 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:20.604596 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31adc83-d373-4c65-adb1-73e03afb86c6 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:20.610294 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 232/465] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:20.610294 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31adc83-d373-4c65-adb1-73e03afb86c6 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:20.663993 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:20.667498 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:20.679602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 234/466] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:20.681395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df2e20-27f8-4212-91e6-d7d8bf3404fa None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:20.681804 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df2e20-27f8-4212-91e6-d7d8bf3404fa None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:20.682348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df2e20-27f8-4212-91e6-d7d8bf3404fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:20.700810 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-72df2e20-27f8-4212-91e6-d7d8bf3404fa None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:20.702240 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-72df2e20-27f8-4212-91e6-d7d8bf3404fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:20.783048 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31adc83-d373-4c65-adb1-73e03afb86c6 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:20.794490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 235/467] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:20.796299 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df2e20-27f8-4212-91e6-d7d8bf3404fa None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:20.827744 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19419338-edc8-43d5-8845-bf0dbc623e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:20.830059 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19419338-edc8-43d5-8845-bf0dbc623e59 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:20.830707 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19419338-edc8-43d5-8845-bf0dbc623e59 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:20.831142 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19419338-edc8-43d5-8845-bf0dbc623e59 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:20.832158 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19419338-edc8-43d5-8845-bf0dbc623e59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:20.832617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19419338-edc8-43d5-8845-bf0dbc623e59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:20.834175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19419338-edc8-43d5-8845-bf0dbc623e59 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:20.848319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 233/468] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:20.849433 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19419338-edc8-43d5-8845-bf0dbc623e59 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:20.944898 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df2e20-27f8-4212-91e6-d7d8bf3404fa None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:20.966636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:20.971120 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:20.979125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19419338-edc8-43d5-8845-bf0dbc623e59 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:20.984884 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6beb6a7-48b9-43c5-82e5-8fa9d24e81ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:20.985698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6beb6a7-48b9-43c5-82e5-8fa9d24e81ab None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:20.985980 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6beb6a7-48b9-43c5-82e5-8fa9d24e81ab None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:20.986282 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6beb6a7-48b9-43c5-82e5-8fa9d24e81ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:20.997748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6beb6a7-48b9-43c5-82e5-8fa9d24e81ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:20.999278 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6beb6a7-48b9-43c5-82e5-8fa9d24e81ab None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:21.018789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-e6beb6a7-48b9-43c5-82e5-8fa9d24e81ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:21.018789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:21.018789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:21.018789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:21.018789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:21.018789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:21.018789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:21.018789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:21.018789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:21.018789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:21.018789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:21.018789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:21.018789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:21.018789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:21.018789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:21.020075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:21.021250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:21.021250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:21.021250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:21.021250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:21.021250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 236/469] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:21.021250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6beb6a7-48b9-43c5-82e5-8fa9d24e81ab None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:21.026366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab821a-4f40-4771-b3df-73ab8dd9f2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:21.027473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab821a-4f40-4771-b3df-73ab8dd9f2e5 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:21.027925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab821a-4f40-4771-b3df-73ab8dd9f2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:21.028360 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab821a-4f40-4771-b3df-73ab8dd9f2e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:21.045469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab821a-4f40-4771-b3df-73ab8dd9f2e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:21.047978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab821a-4f40-4771-b3df-73ab8dd9f2e5 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:21.075476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-75ab821a-4f40-4771-b3df-73ab8dd9f2e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:21.075476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:21.075476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:21.075476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:21.075476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:21.075476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:21.075476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:21.075476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:21.075476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:21.075476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:21.075476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:21.075476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:21.075476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:21.075476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:21.075476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:21.076768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:21.078085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:21.078085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:21.078085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:21.078085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:21.079886 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 234/470] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:21.085185 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab821a-4f40-4771-b3df-73ab8dd9f2e5 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:21.164522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6beb6a7-48b9-43c5-82e5-8fa9d24e81ab None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:21.211566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab821a-4f40-4771-b3df-73ab8dd9f2e5 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:21.213230 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a2cb0-88dc-4d3b-8e5a-755f7d17b3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:21.214702 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a2cb0-88dc-4d3b-8e5a-755f7d17b3bb None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:21.215145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a2cb0-88dc-4d3b-8e5a-755f7d17b3bb None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:21.215608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a2cb0-88dc-4d3b-8e5a-755f7d17b3bb None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:21.217422 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a2cb0-88dc-4d3b-8e5a-755f7d17b3bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:21.219089 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a2cb0-88dc-4d3b-8e5a-755f7d17b3bb None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:21.244230 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 237/471] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:18:21 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:18:21.245347 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a2cb0-88dc-4d3b-8e5a-755f7d17b3bb None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:21.262318 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908abf6e-bef2-40f9-aae4-1a8e3e761d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:21.263653 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908abf6e-bef2-40f9-aae4-1a8e3e761d7f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:21.264061 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908abf6e-bef2-40f9-aae4-1a8e3e761d7f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:21.264435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908abf6e-bef2-40f9-aae4-1a8e3e761d7f None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:21.265633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908abf6e-bef2-40f9-aae4-1a8e3e761d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:21.266052 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908abf6e-bef2-40f9-aae4-1a8e3e761d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:21.268301 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908abf6e-bef2-40f9-aae4-1a8e3e761d7f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:21.279159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:21.283683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 235/472] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:21.285573 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ab0e5-1a90-4abc-9721-edfb313f62ac None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:21.286112 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ab0e5-1a90-4abc-9721-edfb313f62ac None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:21.286608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:21.286876 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ab0e5-1a90-4abc-9721-edfb313f62ac None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:21.288820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 236/473] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:21.290141 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ab0e5-1a90-4abc-9721-edfb313f62ac None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:21.382089 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a2cb0-88dc-4d3b-8e5a-755f7d17b3bb None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:21.416176 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ab0e5-1a90-4abc-9721-edfb313f62ac None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:21.425108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8be04-e8cc-4967-a6ee-2710433ebe79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:21.426176 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8be04-e8cc-4967-a6ee-2710433ebe79 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:21.426849 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8be04-e8cc-4967-a6ee-2710433ebe79 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:21.427196 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8be04-e8cc-4967-a6ee-2710433ebe79 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:21.428019 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8be04-e8cc-4967-a6ee-2710433ebe79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:21.428410 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8be04-e8cc-4967-a6ee-2710433ebe79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:21.430361 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8be04-e8cc-4967-a6ee-2710433ebe79 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:21.451970 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 238/474] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:21.457379 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7b272-e261-4883-acde-28134c944994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:21.458209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ec88d-49de-434a-bb5d-dab37b850a1a None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:21.458604 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ec88d-49de-434a-bb5d-dab37b850a1a None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:21.458909 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7b272-e261-4883-acde-28134c944994 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:21.459340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ec88d-49de-434a-bb5d-dab37b850a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:21.459568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7b272-e261-4883-acde-28134c944994 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:21.460030 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7b272-e261-4883-acde-28134c944994 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:21.461069 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7b272-e261-4883-acde-28134c944994 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:21.463020 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7b272-e261-4883-acde-28134c944994 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:21.480003 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-c64ec88d-49de-434a-bb5d-dab37b850a1a None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:21.481512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-c64ec88d-49de-434a-bb5d-dab37b850a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:21.497998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 237/475] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:18:21 2022] POST /identity/v3/users => generated 248 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:18:21.498205 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7b272-e261-4883-acde-28134c944994 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:21.578693 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 239/476] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:21.603832 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:21.614437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ec88d-49de-434a-bb5d-dab37b850a1a None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:21.644500 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:21.674994 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7b272-e261-4883-acde-28134c944994 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:21.729267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5a2e1-2fc0-4eec-8c78-0a9010016756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:21.729965 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5a2e1-2fc0-4eec-8c78-0a9010016756 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:21.730240 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5a2e1-2fc0-4eec-8c78-0a9010016756 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:21.730541 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5a2e1-2fc0-4eec-8c78-0a9010016756 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:21.731259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d5a2e1-2fc0-4eec-8c78-0a9010016756 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:21.732714 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d5a2e1-2fc0-4eec-8c78-0a9010016756 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:21.757456 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 238/477] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:18:21 2022] POST /identity/v3/users => generated 249 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:18:21.776688 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ec88d-49de-434a-bb5d-dab37b850a1a None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:21.816513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e9ca4-9737-4034-bd91-9cd913ec88e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:21.817214 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e9ca4-9737-4034-bd91-9cd913ec88e4 None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:21.817484 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e9ca4-9737-4034-bd91-9cd913ec88e4 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:21.817731 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e9ca4-9737-4034-bd91-9cd913ec88e4 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:21.818522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e9ca4-9737-4034-bd91-9cd913ec88e4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:21.819953 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e9ca4-9737-4034-bd91-9cd913ec88e4 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:21.828674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.sql.core [None req-171e9ca4-9737-4034-bd91-9cd913ec88e4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:18:21.828674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 20:18:21.828674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [parameters: {'id': 'c72ced30bef14627ad3f846e4274e487', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 20:18:21.828674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=277681) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:18:21.847071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-171e9ca4-9737-4034-bd91-9cd913ec88e4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 20:18:21.847071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:21.847071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:18:21.847071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:18:21.847071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:18:21.847071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:18:21.847071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:18:21.847071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:18:21.847071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:18:21.847071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application conn.query(q) May 19 20:18:21.847071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:18:21.847071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:18:21.847071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:18:21.847071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application result.read() May 19 20:18:21.847071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return ref.to_dict() May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:18:21.848166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application next(self.gen) May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application yield resource May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application next(self.gen) May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise self.value May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application session.commit() May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:18:21.849373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self.session.flush() May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application compat.raise_( May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:18:21.850421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:18:21.851446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application conn.query(q) May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application result.read() May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:18:21.852555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [parameters: {'id': 'c72ced30bef14627ad3f846e4274e487', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:21.853608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 20:18:21.860275 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:21.862524 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 240/478] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 20:18:21 2022] POST /identity/v3/roles => generated 144 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:21.864988 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5a2e1-2fc0-4eec-8c78-0a9010016756 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:21.997789 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5a2e1-2fc0-4eec-8c78-0a9010016756 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:22.015706 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:22.044760 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cbf4d-b20f-4208-881b-1b87d705f2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:22.045608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cbf4d-b20f-4208-881b-1b87d705f2d6 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:22.045894 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cbf4d-b20f-4208-881b-1b87d705f2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:22.046171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cbf4d-b20f-4208-881b-1b87d705f2d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:22.057800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cbf4d-b20f-4208-881b-1b87d705f2d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-415cbf4d-b20f-4208-881b-1b87d705f2d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:22.066047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:22.067134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:22.068074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:22.068074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:22.068074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:22.068074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:22.068074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:22.068074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:22.068074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:22.068074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:22.068074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:22.068074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:22.068074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:18:22.068074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:22.068074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 239/479] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:18:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:22.075239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e9ca4-9737-4034-bd91-9cd913ec88e4 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:22.173676 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:22.193508 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e9ca4-9737-4034-bd91-9cd913ec88e4 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:22.232505 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866785d-dc69-4608-8113-7dd3f26c8471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:22.233215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866785d-dc69-4608-8113-7dd3f26c8471 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:22.233498 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866785d-dc69-4608-8113-7dd3f26c8471 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:22.233857 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866785d-dc69-4608-8113-7dd3f26c8471 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:22.234664 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866785d-dc69-4608-8113-7dd3f26c8471 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:22.235032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866785d-dc69-4608-8113-7dd3f26c8471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:22.236490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866785d-dc69-4608-8113-7dd3f26c8471 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:22.245599 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 241/480] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:23.249553 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed14d6-62c6-4a0a-8107-340bea646d8a None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:23.251031 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed14d6-62c6-4a0a-8107-340bea646d8a None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:23.251727 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed14d6-62c6-4a0a-8107-340bea646d8a None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:23.254637 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 240/481] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:23.264636 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d05364-c02b-42f7-921e-802fefded97f None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:23.265932 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d05364-c02b-42f7-921e-802fefded97f None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:23.266693 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d05364-c02b-42f7-921e-802fefded97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:23.295387 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-79d05364-c02b-42f7-921e-802fefded97f None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:23.296953 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-79d05364-c02b-42f7-921e-802fefded97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:23.387983 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 242/482] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:23.427010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed14d6-62c6-4a0a-8107-340bea646d8a None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:23.586042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed14d6-62c6-4a0a-8107-340bea646d8a None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:23.631914 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a36a5-c336-4ae3-b6a4-d29e3e4ad87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:23.632735 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a36a5-c336-4ae3-b6a4-d29e3e4ad87b None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:23.633015 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a36a5-c336-4ae3-b6a4-d29e3e4ad87b None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:23.633320 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a36a5-c336-4ae3-b6a4-d29e3e4ad87b None admin] PATH_INFO: `/v3/roles/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:23.647291 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a36a5-c336-4ae3-b6a4-d29e3e4ad87b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-298a36a5-c336-4ae3-b6a4-d29e3e4ad87b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:23.655349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:23.656397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:23.657316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:23.657316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:23.657316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:23.657316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:23.657316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:23.657316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:23.657316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:23.657316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:23.657316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:23.657316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:23.657316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:23.657316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:23.661457 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 241/483] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:23.662510 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d05364-c02b-42f7-921e-802fefded97f None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:23.783273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:23.808309 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d05364-c02b-42f7-921e-802fefded97f None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:23.844793 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d15450-0a20-4575-8c50-a650457b4e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:23.845394 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d15450-0a20-4575-8c50-a650457b4e50 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:23.845726 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d15450-0a20-4575-8c50-a650457b4e50 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:23.846007 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d15450-0a20-4575-8c50-a650457b4e50 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:23.846847 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d15450-0a20-4575-8c50-a650457b4e50 None admin] RBAC: Adding query filter params (name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:23.847161 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d15450-0a20-4575-8c50-a650457b4e50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:23.848562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d15450-0a20-4575-8c50-a650457b4e50 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:23.861788 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 243/484] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:23.866922 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a36a5-c336-4ae3-b6a4-d29e3e4ad87b None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:23.929149 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:24.022814 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a36a5-c336-4ae3-b6a4-d29e3e4ad87b None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:24.080239 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b80bb-0fdd-461f-bd0e-edd03971f326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:24.081072 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b80bb-0fdd-461f-bd0e-edd03971f326 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:24.081355 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b80bb-0fdd-461f-bd0e-edd03971f326 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:24.081632 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b80bb-0fdd-461f-bd0e-edd03971f326 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:24.100511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b80bb-0fdd-461f-bd0e-edd03971f326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:24.101919 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b80bb-0fdd-461f-bd0e-edd03971f326 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:24.122214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-cb5b80bb-0fdd-461f-bd0e-edd03971f326 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:24.122214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:24.122214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:24.122214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:24.122214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:24.122214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:24.122214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:24.122214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:24.122214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:24.122214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:24.122214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:24.122214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:24.122214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:24.122214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:24.122214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:24.124416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:24.126682 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:24.126682 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:24.126682 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:24.126682 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:24.130294 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 242/485] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:24.130519 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d15450-0a20-4575-8c50-a650457b4e50 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:24.244045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d15450-0a20-4575-8c50-a650457b4e50 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:24.275267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe55ff-e5ec-4b6d-a001-91d11a5ce8ca None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:24.275920 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe55ff-e5ec-4b6d-a001-91d11a5ce8ca None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:24.275920 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe55ff-e5ec-4b6d-a001-91d11a5ce8ca None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:24.277211 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 243/486] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:24.284573 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3de588-f6ea-469d-8cbc-55c0affebfb2 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:24.284860 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3de588-f6ea-469d-8cbc-55c0affebfb2 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:24.285146 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3de588-f6ea-469d-8cbc-55c0affebfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:24.300605 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc3e7c-9adb-45fb-b727-06794a77f36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:24.301273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc3e7c-9adb-45fb-b727-06794a77f36e None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:24.301565 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc3e7c-9adb-45fb-b727-06794a77f36e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:24.301843 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc3e7c-9adb-45fb-b727-06794a77f36e None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:24.302601 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc3e7c-9adb-45fb-b727-06794a77f36e None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:24.302911 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc3e7c-9adb-45fb-b727-06794a77f36e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:24.304330 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc3e7c-9adb-45fb-b727-06794a77f36e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:24.312943 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-9c3de588-f6ea-469d-8cbc-55c0affebfb2 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:24.314154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-9c3de588-f6ea-469d-8cbc-55c0affebfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:24.316900 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:24.328832 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 244/487] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:24.333774 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc3e7c-9adb-45fb-b727-06794a77f36e None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:24.462543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 244/488] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:24.521566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3de588-f6ea-469d-8cbc-55c0affebfb2 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:24.541017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc3e7c-9adb-45fb-b727-06794a77f36e None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:24.548745 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:24.600411 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f26d9-71b3-4c7d-bd5a-40e1fa69877d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:24.601236 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f26d9-71b3-4c7d-bd5a-40e1fa69877d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:24.601518 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f26d9-71b3-4c7d-bd5a-40e1fa69877d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:24.601795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f26d9-71b3-4c7d-bd5a-40e1fa69877d None admin] PATH_INFO: `/v3/users/nova` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:24.611109 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f26d9-71b3-4c7d-bd5a-40e1fa69877d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:24.612936 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f26d9-71b3-4c7d-bd5a-40e1fa69877d None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-096f26d9-71b3-4c7d-bd5a-40e1fa69877d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:24.631770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:24.633113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:24.634536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:24.634536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:24.634536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:24.634536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:24.634536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:24.634536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:24.634536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:24.634536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:24.634536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:18:24.634536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:24.641464 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 245/489] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:18:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:24.643435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87cf70-2bca-498c-8cde-7c756cf13f1e None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:24.643979 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87cf70-2bca-498c-8cde-7c756cf13f1e None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:24.644447 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87cf70-2bca-498c-8cde-7c756cf13f1e None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:24.646319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 246/490] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:24.647261 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87cf70-2bca-498c-8cde-7c756cf13f1e None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:24.650545 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3de588-f6ea-469d-8cbc-55c0affebfb2 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:24.688205 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b69ad9-a5d4-4dc2-b7df-607529face3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:24.689290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b69ad9-a5d4-4dc2-b7df-607529face3a None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:24.689745 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b69ad9-a5d4-4dc2-b7df-607529face3a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:24.690186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b69ad9-a5d4-4dc2-b7df-607529face3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:24.707571 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b69ad9-a5d4-4dc2-b7df-607529face3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-98b69ad9-a5d4-4dc2-b7df-607529face3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:24.717868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:24.719228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:24.720563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:24.720563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:24.720563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:24.720563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:24.720563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:24.720563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:24.720563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:24.720563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:24.720563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:24.720563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:24.720563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:24.720563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:24.720563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 245/491] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:24.721537 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e07308-e69f-4e63-82d1-4aa0de815d14 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:24.721537 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e07308-e69f-4e63-82d1-4aa0de815d14 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:24.721537 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e07308-e69f-4e63-82d1-4aa0de815d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:24.729583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:24.746045 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-26e07308-e69f-4e63-82d1-4aa0de815d14 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:24.747305 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-26e07308-e69f-4e63-82d1-4aa0de815d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:24.778907 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87cf70-2bca-498c-8cde-7c756cf13f1e None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:24.816423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8895b-2582-423a-8bcb-2a064aecdd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:24.817111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8895b-2582-423a-8bcb-2a064aecdd2b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:24.817395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8895b-2582-423a-8bcb-2a064aecdd2b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:24.817669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8895b-2582-423a-8bcb-2a064aecdd2b None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:24.818578 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8895b-2582-423a-8bcb-2a064aecdd2b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:24.818889 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8895b-2582-423a-8bcb-2a064aecdd2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:24.820274 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 246/492] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:24.820462 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8895b-2582-423a-8bcb-2a064aecdd2b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:24.822296 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e07308-e69f-4e63-82d1-4aa0de815d14 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:24.829855 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:24.862780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 247/493] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:18:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:24.864288 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e50b2-366c-4ef0-abbc-34de9c27be69 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:24.864583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e50b2-366c-4ef0-abbc-34de9c27be69 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:24.864868 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e50b2-366c-4ef0-abbc-34de9c27be69 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:24.866435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 248/494] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:24.867162 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e50b2-366c-4ef0-abbc-34de9c27be69 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:24.931387 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e07308-e69f-4e63-82d1-4aa0de815d14 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:24.967062 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983a4f3-0a2a-4a15-9a20-448bd46db962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:24.967643 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983a4f3-0a2a-4a15-9a20-448bd46db962 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:24.967923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983a4f3-0a2a-4a15-9a20-448bd46db962 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:24.968203 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983a4f3-0a2a-4a15-9a20-448bd46db962 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:24.968936 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983a4f3-0a2a-4a15-9a20-448bd46db962 None admin] RBAC: Adding query filter params (name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:24.969232 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983a4f3-0a2a-4a15-9a20-448bd46db962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:24.970636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983a4f3-0a2a-4a15-9a20-448bd46db962 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:24.979614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 247/495] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:24.980229 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983a4f3-0a2a-4a15-9a20-448bd46db962 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:24.990542 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e50b2-366c-4ef0-abbc-34de9c27be69 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:25.009724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:25.030192 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987815d-37bc-4a38-a2a1-3862e20c6a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:25.031045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987815d-37bc-4a38-a2a1-3862e20c6a9b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:25.031375 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987815d-37bc-4a38-a2a1-3862e20c6a9b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:25.031595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987815d-37bc-4a38-a2a1-3862e20c6a9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:25.045573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987815d-37bc-4a38-a2a1-3862e20c6a9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:25.050862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-0987815d-37bc-4a38-a2a1-3862e20c6a9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:25.053780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:25.055561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:25.056968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:25.056968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:25.056968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:25.056968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:25.056968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:25.056968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:25.056968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:25.056968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:25.056968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:25.056968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:25.056968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:25.056968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:25.056968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 249/496] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:25.056968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cb00f-9fc2-48c7-a8e6-5912700374d3 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:25.057812 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cb00f-9fc2-48c7-a8e6-5912700374d3 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:25.057812 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cb00f-9fc2-48c7-a8e6-5912700374d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:25.076620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-a85cb00f-9fc2-48c7-a8e6-5912700374d3 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:25.077838 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-a85cb00f-9fc2-48c7-a8e6-5912700374d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:25.095834 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983a4f3-0a2a-4a15-9a20-448bd46db962 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:25.139173 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353187f-4493-4f38-8cd9-8f4737502bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:25.139839 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353187f-4493-4f38-8cd9-8f4737502bba None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:25.140116 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353187f-4493-4f38-8cd9-8f4737502bba None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:25.140388 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353187f-4493-4f38-8cd9-8f4737502bba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:25.158295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 250/497] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:25.158295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cb00f-9fc2-48c7-a8e6-5912700374d3 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:25.209626 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353187f-4493-4f38-8cd9-8f4737502bba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:25.211141 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353187f-4493-4f38-8cd9-8f4737502bba None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:25.268761 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cb00f-9fc2-48c7-a8e6-5912700374d3 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:25.275169 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:25.277441 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:25.304567 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74a9b8-f8cf-4dc1-82c4-57f9f559cb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:25.305414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74a9b8-f8cf-4dc1-82c4-57f9f559cb13 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:25.305698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74a9b8-f8cf-4dc1-82c4-57f9f559cb13 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:25.305980 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74a9b8-f8cf-4dc1-82c4-57f9f559cb13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:25.312838 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 248/498] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:25.313686 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353187f-4493-4f38-8cd9-8f4737502bba None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:25.316277 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74a9b8-f8cf-4dc1-82c4-57f9f559cb13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:25.317773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74a9b8-f8cf-4dc1-82c4-57f9f559cb13 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:25.335846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-2a74a9b8-f8cf-4dc1-82c4-57f9f559cb13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:25.335846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:25.335846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:25.335846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:25.335846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:25.335846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:25.335846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:25.335846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:25.335846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:25.335846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:25.335846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:25.335846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:25.335846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:25.335846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:25.335846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:25.337113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:25.338347 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:25.338347 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:25.338347 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:25.338347 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:25.338347 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 251/499] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:25.338347 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74a9b8-f8cf-4dc1-82c4-57f9f559cb13 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:25.412576 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353187f-4493-4f38-8cd9-8f4737502bba None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:25.442487 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74a9b8-f8cf-4dc1-82c4-57f9f559cb13 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:25.448622 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd203b-1076-4a76-b00b-3e900cccabe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:25.449303 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd203b-1076-4a76-b00b-3e900cccabe5 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:25.449584 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd203b-1076-4a76-b00b-3e900cccabe5 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:25.449860 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd203b-1076-4a76-b00b-3e900cccabe5 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:25.450643 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd203b-1076-4a76-b00b-3e900cccabe5 None admin] RBAC: Adding query filter params (name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:25.450950 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd203b-1076-4a76-b00b-3e900cccabe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:25.452343 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd203b-1076-4a76-b00b-3e900cccabe5 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:25.460334 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 249/500] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:25.461578 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9841f9-117d-4678-b4f0-9dc4d01d5e54 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:25.461864 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9841f9-117d-4678-b4f0-9dc4d01d5e54 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:25.462140 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9841f9-117d-4678-b4f0-9dc4d01d5e54 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:25.463404 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 250/501] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:25.464413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9841f9-117d-4678-b4f0-9dc4d01d5e54 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:25.479760 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319a637-1880-4646-a03c-ebd9d84b37e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:25.480571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319a637-1880-4646-a03c-ebd9d84b37e2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:25.480849 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319a637-1880-4646-a03c-ebd9d84b37e2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:25.481152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319a637-1880-4646-a03c-ebd9d84b37e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:25.491790 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319a637-1880-4646-a03c-ebd9d84b37e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:25.492455 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-7319a637-1880-4646-a03c-ebd9d84b37e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:25.500063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:25.501124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:25.502439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:25.502439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:25.502439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:25.502439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:25.502439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:25.502439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:25.502439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:25.502439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:25.502439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:25.502439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:25.502439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:25.502439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:25.502439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 252/502] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:25.502439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319a637-1880-4646-a03c-ebd9d84b37e2 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:25.520860 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:25.573068 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9841f9-117d-4678-b4f0-9dc4d01d5e54 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:25.604206 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319a637-1880-4646-a03c-ebd9d84b37e2 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:25.610561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bce249-0a5a-4d6d-b3b9-2f8ffc2ef86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:25.611679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bce249-0a5a-4d6d-b3b9-2f8ffc2ef86e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:25.611957 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bce249-0a5a-4d6d-b3b9-2f8ffc2ef86e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:25.612232 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bce249-0a5a-4d6d-b3b9-2f8ffc2ef86e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:25.623053 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bce249-0a5a-4d6d-b3b9-2f8ffc2ef86e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:25.624509 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bce249-0a5a-4d6d-b3b9-2f8ffc2ef86e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:25.638589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f393ba-00f6-4c47-b4b3-2e2efda670dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:25.639251 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f393ba-00f6-4c47-b4b3-2e2efda670dc None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:25.639536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f393ba-00f6-4c47-b4b3-2e2efda670dc None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:25.639815 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f393ba-00f6-4c47-b4b3-2e2efda670dc None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:25.640535 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f393ba-00f6-4c47-b4b3-2e2efda670dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:25.640846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f393ba-00f6-4c47-b4b3-2e2efda670dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:25.641723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-54bce249-0a5a-4d6d-b3b9-2f8ffc2ef86e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:25.641723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:25.641723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:25.641723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:25.641723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:25.641723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:25.641723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:25.641723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:25.641723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:25.641723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:25.641723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:25.641723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:25.641723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:25.641723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:25.641723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:25.642964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:25.644013 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:25.644013 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:25.644013 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:25.644013 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:25.644283 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f393ba-00f6-4c47-b4b3-2e2efda670dc None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:25.644361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 251/503] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:25.644361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bce249-0a5a-4d6d-b3b9-2f8ffc2ef86e None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:25.655467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 253/504] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:25.656772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead8809-07cb-4d48-82a8-24136cb5d3c8 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:25.657058 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead8809-07cb-4d48-82a8-24136cb5d3c8 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:25.657335 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead8809-07cb-4d48-82a8-24136cb5d3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:25.673385 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-2ead8809-07cb-4d48-82a8-24136cb5d3c8 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:25.674598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-2ead8809-07cb-4d48-82a8-24136cb5d3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:25.736317 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:25.742770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 254/505] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:25.743624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead8809-07cb-4d48-82a8-24136cb5d3c8 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:25.753332 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:25.769193 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bce249-0a5a-4d6d-b3b9-2f8ffc2ef86e None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:25.804248 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affb36c-69ca-4ab8-8d7d-bf40f211bdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:25.805062 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affb36c-69ca-4ab8-8d7d-bf40f211bdb3 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:25.805563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affb36c-69ca-4ab8-8d7d-bf40f211bdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:25.805948 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affb36c-69ca-4ab8-8d7d-bf40f211bdb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:25.816644 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affb36c-69ca-4ab8-8d7d-bf40f211bdb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:25.818109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affb36c-69ca-4ab8-8d7d-bf40f211bdb3 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:25.835905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-9affb36c-69ca-4ab8-8d7d-bf40f211bdb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:25.835905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:25.835905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:25.835905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:25.835905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:25.835905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:25.835905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:25.835905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:25.835905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:25.835905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:25.835905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:25.835905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:25.835905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:25.835905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:25.835905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:25.837256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:25.838303 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:25.838303 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:25.838303 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:25.838303 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:25.838303 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 252/506] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:25.839195 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affb36c-69ca-4ab8-8d7d-bf40f211bdb3 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:25.852473 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead8809-07cb-4d48-82a8-24136cb5d3c8 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:25.886872 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815a21f-6e82-407e-a18a-dd7ff39a0e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:25.887532 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815a21f-6e82-407e-a18a-dd7ff39a0e64 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:25.887810 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815a21f-6e82-407e-a18a-dd7ff39a0e64 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:25.888084 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815a21f-6e82-407e-a18a-dd7ff39a0e64 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:25.888859 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815a21f-6e82-407e-a18a-dd7ff39a0e64 None admin] RBAC: Adding query filter params (name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:25.889164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815a21f-6e82-407e-a18a-dd7ff39a0e64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:25.890590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815a21f-6e82-407e-a18a-dd7ff39a0e64 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:25.898732 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 255/507] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:25.899593 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815a21f-6e82-407e-a18a-dd7ff39a0e64 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:25.947448 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affb36c-69ca-4ab8-8d7d-bf40f211bdb3 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:25.972461 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:25.976482 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:25.982343 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b142001-896f-4b81-b214-6e0546d1721a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:25.982990 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b142001-896f-4b81-b214-6e0546d1721a None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:25.983272 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b142001-896f-4b81-b214-6e0546d1721a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:25.983545 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b142001-896f-4b81-b214-6e0546d1721a None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:25.984259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b142001-896f-4b81-b214-6e0546d1721a None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:25.984564 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b142001-896f-4b81-b214-6e0546d1721a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:25.985939 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b142001-896f-4b81-b214-6e0546d1721a None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:25.998543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 253/508] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:25.999254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b142001-896f-4b81-b214-6e0546d1721a None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:26.007965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815a21f-6e82-407e-a18a-dd7ff39a0e64 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:26.042323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06c39e-f0de-4a55-9344-8b29f4df91bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:26.043165 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06c39e-f0de-4a55-9344-8b29f4df91bc None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:26.043425 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06c39e-f0de-4a55-9344-8b29f4df91bc None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:26.043708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06c39e-f0de-4a55-9344-8b29f4df91bc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:26.051459 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06c39e-f0de-4a55-9344-8b29f4df91bc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:26.052941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06c39e-f0de-4a55-9344-8b29f4df91bc None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-8e06c39e-f0de-4a55-9344-8b29f4df91bc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:26.067398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:26.068489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:26.069573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:26.069573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:26.069573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:26.069573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:26.069573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:26.069573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:26.069573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:26.069573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:26.069573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:18:26.069573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:26.069573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 256/509] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:26.070240 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06c39e-f0de-4a55-9344-8b29f4df91bc None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:26.104715 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b142001-896f-4b81-b214-6e0546d1721a None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:26.140728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb1457-231e-467b-a2aa-2293fc74e2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:26.141507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb1457-231e-467b-a2aa-2293fc74e2f2 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:26.141784 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb1457-231e-467b-a2aa-2293fc74e2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:26.142060 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb1457-231e-467b-a2aa-2293fc74e2f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:26.152507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb1457-231e-467b-a2aa-2293fc74e2f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:26.153952 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb1457-231e-467b-a2aa-2293fc74e2f2 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:26.169869 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06c39e-f0de-4a55-9344-8b29f4df91bc None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:26.171154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-3fbb1457-231e-467b-a2aa-2293fc74e2f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:26.171154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:26.171154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:26.171154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:26.171154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:26.171154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:26.171154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:26.171154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:26.171154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:26.171154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:26.171154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:26.171154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:26.171154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:26.171154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:26.171154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:26.177067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:26.178414 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:26.178414 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:26.178414 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:26.178414 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:26.178414 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 254/510] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:26.178414 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb1457-231e-467b-a2aa-2293fc74e2f2 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:26.206796 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:26.213129 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc37859-981d-4654-9c74-0aa36e593528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:26.214142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc37859-981d-4654-9c74-0aa36e593528 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:26.214430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc37859-981d-4654-9c74-0aa36e593528 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:26.214716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc37859-981d-4654-9c74-0aa36e593528 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:26.215441 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc37859-981d-4654-9c74-0aa36e593528 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:26.215755 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc37859-981d-4654-9c74-0aa36e593528 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:26.217166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc37859-981d-4654-9c74-0aa36e593528 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:26.220026 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:26.229420 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 257/511] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:26.230236 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc37859-981d-4654-9c74-0aa36e593528 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:26.280776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb1457-231e-467b-a2aa-2293fc74e2f2 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:26.315755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bca56-b729-45ec-8a38-8d0bb4afc9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:26.316578 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bca56-b729-45ec-8a38-8d0bb4afc9fb None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:26.316882 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bca56-b729-45ec-8a38-8d0bb4afc9fb None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:26.317160 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bca56-b729-45ec-8a38-8d0bb4afc9fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:26.327970 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bca56-b729-45ec-8a38-8d0bb4afc9fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:26.329431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bca56-b729-45ec-8a38-8d0bb4afc9fb None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:26.335045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc37859-981d-4654-9c74-0aa36e593528 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:26.347817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-402bca56-b729-45ec-8a38-8d0bb4afc9fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:26.347817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:26.347817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:26.347817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:26.347817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:26.347817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:26.347817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:26.347817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:26.347817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:26.347817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:26.347817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:26.347817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:26.347817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:26.347817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:26.347817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:26.348795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:26.349805 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:26.349805 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:26.349805 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:26.349805 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:26.349805 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 255/512] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:26.349805 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bca56-b729-45ec-8a38-8d0bb4afc9fb None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:26.376857 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72cb36-ac5b-4895-a593-8acc415f82c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:26.377695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72cb36-ac5b-4895-a593-8acc415f82c9 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:26.377964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72cb36-ac5b-4895-a593-8acc415f82c9 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:26.378242 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72cb36-ac5b-4895-a593-8acc415f82c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:26.385572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72cb36-ac5b-4895-a593-8acc415f82c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:26.387068 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72cb36-ac5b-4895-a593-8acc415f82c9 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:26.401892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-ec72cb36-ac5b-4895-a593-8acc415f82c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:26.401892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:26.401892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:26.401892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:26.401892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:26.401892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:26.401892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:26.401892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:26.401892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:26.401892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:26.401892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:26.401892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:26.401892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:26.401892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:26.401892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:26.402923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:26.404024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:26.404024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:26.404024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:26.404024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:26.404024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:26.404024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:26.404024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:26.404024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:26.404024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 258/513] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:26.404024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72cb36-ac5b-4895-a593-8acc415f82c9 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:26.449633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:26.455716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bca56-b729-45ec-8a38-8d0bb4afc9fb None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:26.459840 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:26.490363 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc2916-a523-49f8-a3cb-605ddc06294c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:26.491056 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc2916-a523-49f8-a3cb-605ddc06294c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:26.491336 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc2916-a523-49f8-a3cb-605ddc06294c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:26.491609 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc2916-a523-49f8-a3cb-605ddc06294c None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:26.492433 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc2916-a523-49f8-a3cb-605ddc06294c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:26.492888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc2916-a523-49f8-a3cb-605ddc06294c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:26.494356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc2916-a523-49f8-a3cb-605ddc06294c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:26.511965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72cb36-ac5b-4895-a593-8acc415f82c9 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:26.531267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 256/514] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 20:18:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:26.532096 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc2916-a523-49f8-a3cb-605ddc06294c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:26.550902 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f222f-0dac-4ab5-8056-97203e5956d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:26.551581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f222f-0dac-4ab5-8056-97203e5956d0 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:26.551867 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f222f-0dac-4ab5-8056-97203e5956d0 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:26.552151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f222f-0dac-4ab5-8056-97203e5956d0 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:26.552886 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f222f-0dac-4ab5-8056-97203e5956d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:26.553212 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f222f-0dac-4ab5-8056-97203e5956d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:26.554628 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f222f-0dac-4ab5-8056-97203e5956d0 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:26.567006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 259/515] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:26.567818 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f222f-0dac-4ab5-8056-97203e5956d0 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:26.633970 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc2916-a523-49f8-a3cb-605ddc06294c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:26.670552 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f222f-0dac-4ab5-8056-97203e5956d0 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:26.670849 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc316c-02a2-45d1-8305-04475e8083bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:26.671637 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc316c-02a2-45d1-8305-04475e8083bd None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:26.671916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc316c-02a2-45d1-8305-04475e8083bd None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:26.672184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc316c-02a2-45d1-8305-04475e8083bd None admin] PATH_INFO: `/v3/users/glance` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:26.679555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc316c-02a2-45d1-8305-04475e8083bd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:26.681188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:26.681450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc316c-02a2-45d1-8305-04475e8083bd None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-d9dc316c-02a2-45d1-8305-04475e8083bd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:26.697932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:26.699042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:26.700090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:26.700090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:26.700090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:26.700090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:26.700090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:26.700090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:26.700090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:26.700090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:26.700090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:18:26.700090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:26.700090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 257/516] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:26.700090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc316c-02a2-45d1-8305-04475e8083bd None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:26.710928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b1414-1bdf-44d1-ab09-d702dec14005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:26.711661 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b1414-1bdf-44d1-ab09-d702dec14005 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:26.711943 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b1414-1bdf-44d1-ab09-d702dec14005 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:26.712495 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b1414-1bdf-44d1-ab09-d702dec14005 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:26.712966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b1414-1bdf-44d1-ab09-d702dec14005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:26.713278 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b1414-1bdf-44d1-ab09-d702dec14005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:26.715039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b1414-1bdf-44d1-ab09-d702dec14005 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:26.716425 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:26.726824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 260/517] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:26.727715 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b1414-1bdf-44d1-ab09-d702dec14005 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:26.807697 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc316c-02a2-45d1-8305-04475e8083bd None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:26.838132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b1414-1bdf-44d1-ab09-d702dec14005 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:26.843885 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b15fb-daa0-470b-881b-f96e897eb08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:26.844555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b15fb-daa0-470b-881b-f96e897eb08e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:26.844838 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b15fb-daa0-470b-881b-f96e897eb08e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:26.845334 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b15fb-daa0-470b-881b-f96e897eb08e None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:26.845892 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b15fb-daa0-470b-881b-f96e897eb08e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:26.846201 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b15fb-daa0-470b-881b-f96e897eb08e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:26.847714 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b15fb-daa0-470b-881b-f96e897eb08e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:26.865718 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 258/518] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:26.866547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b15fb-daa0-470b-881b-f96e897eb08e None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:26.873974 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde061d2-0d47-4930-bb49-4a38806884ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:26.874721 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde061d2-0d47-4930-bb49-4a38806884ee None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:26.874979 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde061d2-0d47-4930-bb49-4a38806884ee None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:26.875260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde061d2-0d47-4930-bb49-4a38806884ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:26.916397 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:26.944473 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde061d2-0d47-4930-bb49-4a38806884ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:26.946149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde061d2-0d47-4930-bb49-4a38806884ee None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:26.947399 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:26.973068 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b15fb-daa0-470b-881b-f96e897eb08e None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:27.010766 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee6b94-06c3-441d-b109-36b70c1044ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:27.011447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee6b94-06c3-441d-b109-36b70c1044ad None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:27.011731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee6b94-06c3-441d-b109-36b70c1044ad None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:27.012007 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee6b94-06c3-441d-b109-36b70c1044ad None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:27.012712 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee6b94-06c3-441d-b109-36b70c1044ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:27.014175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee6b94-06c3-441d-b109-36b70c1044ad None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:27.035493 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 259/519] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:18:26 2022] POST /identity/v3/users => generated 247 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:18:27.036304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee6b94-06c3-441d-b109-36b70c1044ad None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:27.069156 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 261/520] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:27.069156 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde061d2-0d47-4930-bb49-4a38806884ee None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:27.151070 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:27.163374 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee6b94-06c3-441d-b109-36b70c1044ad None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:27.188483 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde061d2-0d47-4930-bb49-4a38806884ee None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:27.197892 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:27.205193 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49114c50-5e15-40e8-b74b-e8eaab9884d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:27.205780 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49114c50-5e15-40e8-b74b-e8eaab9884d6 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:27.206071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49114c50-5e15-40e8-b74b-e8eaab9884d6 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:27.206733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49114c50-5e15-40e8-b74b-e8eaab9884d6 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:27.207994 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49114c50-5e15-40e8-b74b-e8eaab9884d6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:27.208551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49114c50-5e15-40e8-b74b-e8eaab9884d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:27.210394 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49114c50-5e15-40e8-b74b-e8eaab9884d6 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:27.233208 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714c2e7-c26f-4aa4-b527-17f1e2bb24a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:27.234054 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714c2e7-c26f-4aa4-b527-17f1e2bb24a2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:27.234383 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714c2e7-c26f-4aa4-b527-17f1e2bb24a2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:27.234668 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714c2e7-c26f-4aa4-b527-17f1e2bb24a2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:27.234774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 260/521] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 20:18:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:27.235735 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49114c50-5e15-40e8-b74b-e8eaab9884d6 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:27.243449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714c2e7-c26f-4aa4-b527-17f1e2bb24a2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:27.244965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714c2e7-c26f-4aa4-b527-17f1e2bb24a2 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-e714c2e7-c26f-4aa4-b527-17f1e2bb24a2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:27.262833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:27.263991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:27.265041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:27.265041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:27.265041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:27.265041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:27.265041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:27.265041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:27.265041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:27.265041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:27.265041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:18:27.265041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:27.265041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 262/522] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:27.265041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714c2e7-c26f-4aa4-b527-17f1e2bb24a2 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:27.358900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49114c50-5e15-40e8-b74b-e8eaab9884d6 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:27.375905 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714c2e7-c26f-4aa4-b527-17f1e2bb24a2 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:27.399856 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bad1d-f7f7-44a4-bc32-dc965eeeda51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:27.400684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bad1d-f7f7-44a4-bc32-dc965eeeda51 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:27.400971 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bad1d-f7f7-44a4-bc32-dc965eeeda51 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:27.401248 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bad1d-f7f7-44a4-bc32-dc965eeeda51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:27.402108 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bad1d-f7f7-44a4-bc32-dc965eeeda51 None admin] RBAC: Adding query filter params (role.id=c9206a4a0d49473f87a45d6c8abb2e51, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=edb1bcb781084f03a4ad2611a09bd77e) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:27.402596 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bad1d-f7f7-44a4-bc32-dc965eeeda51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:27.404366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bad1d-f7f7-44a4-bc32-dc965eeeda51 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:27.408665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:27.411532 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 261/523] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:18:27 2022] GET /identity/v3/role_assignments?user.id=edb1bcb781084f03a4ad2611a09bd77e&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=c9206a4a0d49473f87a45d6c8abb2e51 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:27.412914 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bad1d-f7f7-44a4-bc32-dc965eeeda51 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:27.416187 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a917b96-1335-463a-b23b-01873508a12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:27.417021 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a917b96-1335-463a-b23b-01873508a12f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:27.417311 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a917b96-1335-463a-b23b-01873508a12f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:27.417592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a917b96-1335-463a-b23b-01873508a12f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:27.438908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a917b96-1335-463a-b23b-01873508a12f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:27.440701 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a917b96-1335-463a-b23b-01873508a12f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:27.461254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-9a917b96-1335-463a-b23b-01873508a12f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:27.461254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:27.461254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:27.461254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:27.461254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:27.461254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:27.461254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:27.461254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:27.461254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:27.461254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:27.461254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:27.461254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:27.461254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:27.461254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:27.461254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:27.462665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:27.464206 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:27.464206 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:27.464206 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:27.464206 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:27.468252 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 263/524] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:27.469269 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a917b96-1335-463a-b23b-01873508a12f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:27.497439 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:27.562290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bad1d-f7f7-44a4-bc32-dc965eeeda51 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:27.598232 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a917b96-1335-463a-b23b-01873508a12f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:27.612168 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05205a-99c6-47b3-9ede-82ae334d48bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:27.613813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05205a-99c6-47b3-9ede-82ae334d48bd None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:27.614284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05205a-99c6-47b3-9ede-82ae334d48bd None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:27.614705 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05205a-99c6-47b3-9ede-82ae334d48bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:27.641305 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e63a4-c1cd-445a-8cd3-6b289146890e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:27.642565 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e63a4-c1cd-445a-8cd3-6b289146890e None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:27.642565 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e63a4-c1cd-445a-8cd3-6b289146890e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:27.642565 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e63a4-c1cd-445a-8cd3-6b289146890e None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:27.643429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e63a4-c1cd-445a-8cd3-6b289146890e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:27.643744 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e63a4-c1cd-445a-8cd3-6b289146890e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:27.645350 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e63a4-c1cd-445a-8cd3-6b289146890e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:27.663952 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 264/525] 10.222.0.84 () {56 vars in 1139 bytes} [Thu May 19 20:18:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:27.665043 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e63a4-c1cd-445a-8cd3-6b289146890e None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:27.702444 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05205a-99c6-47b3-9ede-82ae334d48bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:27.706324 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05205a-99c6-47b3-9ede-82ae334d48bd None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:27.713026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:27.770313 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:27.792381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e63a4-c1cd-445a-8cd3-6b289146890e None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:27.834822 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 262/526] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:27.834822 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05205a-99c6-47b3-9ede-82ae334d48bd None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:27.843429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6805e65-551e-4bbc-a8fb-8a3b4a042566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:27.844949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6805e65-551e-4bbc-a8fb-8a3b4a042566 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:27.845447 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6805e65-551e-4bbc-a8fb-8a3b4a042566 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:27.846313 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6805e65-551e-4bbc-a8fb-8a3b4a042566 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:27.847568 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6805e65-551e-4bbc-a8fb-8a3b4a042566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:27.848098 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6805e65-551e-4bbc-a8fb-8a3b4a042566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:27.850454 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6805e65-551e-4bbc-a8fb-8a3b4a042566 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:27.867871 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 265/527] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:27.869076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6805e65-551e-4bbc-a8fb-8a3b4a042566 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:27.946932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05205a-99c6-47b3-9ede-82ae334d48bd None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:27.984717 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05cf21-d56e-42ed-bcdf-83f12fb1373c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:27.985476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05cf21-d56e-42ed-bcdf-83f12fb1373c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:27.985877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05cf21-d56e-42ed-bcdf-83f12fb1373c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:27.986254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05cf21-d56e-42ed-bcdf-83f12fb1373c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:28.000638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6805e65-551e-4bbc-a8fb-8a3b4a042566 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:28.002747 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:28.013136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:28.047396 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8487347-3531-4880-9d94-e58613261cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:28.048985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8487347-3531-4880-9d94-e58613261cdc None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:28.049703 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8487347-3531-4880-9d94-e58613261cdc None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:28.050141 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8487347-3531-4880-9d94-e58613261cdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:28.065098 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8487347-3531-4880-9d94-e58613261cdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:28.065281 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05cf21-d56e-42ed-bcdf-83f12fb1373c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:28.066874 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05cf21-d56e-42ed-bcdf-83f12fb1373c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:28.067072 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8487347-3531-4880-9d94-e58613261cdc None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:28.086209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-d8487347-3531-4880-9d94-e58613261cdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:28.086209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:28.086209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:28.086209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:28.086209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:28.086209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:28.086209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:28.086209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:28.086209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:28.086209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:28.086209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:28.086209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:28.086209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:28.086209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:28.086209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:28.087641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:28.089333 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:28.089333 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:28.089333 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:28.089333 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:28.094884 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 266/528] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:28.094884 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8487347-3531-4880-9d94-e58613261cdc None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:28.193248 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 263/529] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:28.194713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05cf21-d56e-42ed-bcdf-83f12fb1373c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:28.234619 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8487347-3531-4880-9d94-e58613261cdc None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:28.255247 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:28.286704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1a49d-dd98-48ac-b96a-ad5bb74d8f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:28.288106 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1a49d-dd98-48ac-b96a-ad5bb74d8f16 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:28.288837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1a49d-dd98-48ac-b96a-ad5bb74d8f16 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:28.289336 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1a49d-dd98-48ac-b96a-ad5bb74d8f16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:28.300075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1a49d-dd98-48ac-b96a-ad5bb74d8f16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:28.302407 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1a49d-dd98-48ac-b96a-ad5bb74d8f16 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:28.305598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05cf21-d56e-42ed-bcdf-83f12fb1373c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:28.324149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-25d1a49d-dd98-48ac-b96a-ad5bb74d8f16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:28.324149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:28.324149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:28.324149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:28.324149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:28.324149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:28.324149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:28.324149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:28.324149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:28.324149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:28.324149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:28.324149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:28.324149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:28.324149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:28.324149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:28.325264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:28.326343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:28.326343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:28.326343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:28.326343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:28.326343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:28.326343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:28.326343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:28.326343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:28.332880 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 267/530] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:28.332880 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1a49d-dd98-48ac-b96a-ad5bb74d8f16 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:28.352940 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:28.354981 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e458b0-61d3-449b-95f6-bfe6b7157cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:28.355660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e458b0-61d3-449b-95f6-bfe6b7157cf8 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:28.355937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e458b0-61d3-449b-95f6-bfe6b7157cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:28.356209 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e458b0-61d3-449b-95f6-bfe6b7157cf8 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:28.356927 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e458b0-61d3-449b-95f6-bfe6b7157cf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:28.357396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e458b0-61d3-449b-95f6-bfe6b7157cf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:28.358822 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e458b0-61d3-449b-95f6-bfe6b7157cf8 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:28.375398 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 264/531] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:28.376277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e458b0-61d3-449b-95f6-bfe6b7157cf8 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:28.476413 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1a49d-dd98-48ac-b96a-ad5bb74d8f16 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:28.490979 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e458b0-61d3-449b-95f6-bfe6b7157cf8 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:28.509147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:28.524388 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c441c-1d94-48f1-a69b-ac38f46ef2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:28.526294 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c441c-1d94-48f1-a69b-ac38f46ef2c7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:28.526787 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c441c-1d94-48f1-a69b-ac38f46ef2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:28.527508 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c441c-1d94-48f1-a69b-ac38f46ef2c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:28.533138 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a9b72-268b-4893-898e-0ce5a78ebc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:28.533880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a9b72-268b-4893-898e-0ce5a78ebc4a None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:28.534227 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a9b72-268b-4893-898e-0ce5a78ebc4a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:28.534597 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a9b72-268b-4893-898e-0ce5a78ebc4a None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:28.535469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a9b72-268b-4893-898e-0ce5a78ebc4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:28.535844 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a9b72-268b-4893-898e-0ce5a78ebc4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:28.537475 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a9b72-268b-4893-898e-0ce5a78ebc4a None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:28.542408 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c441c-1d94-48f1-a69b-ac38f46ef2c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:28.544107 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c441c-1d94-48f1-a69b-ac38f46ef2c7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:28.556357 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 265/532] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:28.556357 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a9b72-268b-4893-898e-0ce5a78ebc4a None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:28.573102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-5a1c441c-1d94-48f1-a69b-ac38f46ef2c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:28.573102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:28.573102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:28.573102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:28.573102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:28.573102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:28.573102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:28.573102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:28.573102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:28.573102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:28.573102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:28.573102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:28.573102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:28.573102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:28.573102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:28.574511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:28.575797 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:28.575797 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:28.575797 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:28.575797 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:28.578198 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 268/533] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:28.579565 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c441c-1d94-48f1-a69b-ac38f46ef2c7 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:28.652178 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:28.674744 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a9b72-268b-4893-898e-0ce5a78ebc4a None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:28.691863 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c441c-1d94-48f1-a69b-ac38f46ef2c7 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:28.714297 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07be46-df1c-4393-8b22-e9a3404601f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:28.715349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07be46-df1c-4393-8b22-e9a3404601f0 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:28.715726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07be46-df1c-4393-8b22-e9a3404601f0 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:28.716097 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07be46-df1c-4393-8b22-e9a3404601f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:28.723935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07be46-df1c-4393-8b22-e9a3404601f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:28.725695 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07be46-df1c-4393-8b22-e9a3404601f0 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:28.730415 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c520e-2400-497b-bb8a-ff13b2c6d526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:28.731344 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c520e-2400-497b-bb8a-ff13b2c6d526 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:28.731805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c520e-2400-497b-bb8a-ff13b2c6d526 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:28.732249 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c520e-2400-497b-bb8a-ff13b2c6d526 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:28.733254 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c520e-2400-497b-bb8a-ff13b2c6d526 None admin] RBAC: Adding query filter params (role.id=c9206a4a0d49473f87a45d6c8abb2e51, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=3681f07e23694baba690033a991a23c6) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:28.733736 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c520e-2400-497b-bb8a-ff13b2c6d526 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:28.735394 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c520e-2400-497b-bb8a-ff13b2c6d526 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:28.750316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-6b07be46-df1c-4393-8b22-e9a3404601f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:28.750316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:28.750316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:28.750316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:28.750316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:28.750316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:28.750316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:28.750316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:28.750316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:28.750316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:28.750316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:28.750316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:28.750316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:28.750316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:28.750316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:28.751590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:28.752594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:28.752594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:28.752594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:28.752594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:28.752594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:28.752594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:28.752594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:28.752594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:28.752594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 266/535] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:28.752594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07be46-df1c-4393-8b22-e9a3404601f0 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:28.753235 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 269/535] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:18:28 2022] GET /identity/v3/role_assignments?user.id=3681f07e23694baba690033a991a23c6&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=c9206a4a0d49473f87a45d6c8abb2e51 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:28.753235 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c520e-2400-497b-bb8a-ff13b2c6d526 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:28.844131 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:28.945644 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07be46-df1c-4393-8b22-e9a3404601f0 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:28.986836 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bd02d-e7fc-4c1e-a594-9cd32c6c3506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:28.987917 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bd02d-e7fc-4c1e-a594-9cd32c6c3506 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:28.988425 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bd02d-e7fc-4c1e-a594-9cd32c6c3506 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:28.988922 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bd02d-e7fc-4c1e-a594-9cd32c6c3506 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:28.989888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bd02d-e7fc-4c1e-a594-9cd32c6c3506 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:28.990453 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bd02d-e7fc-4c1e-a594-9cd32c6c3506 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:28.992070 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bd02d-e7fc-4c1e-a594-9cd32c6c3506 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:28.995722 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c520e-2400-497b-bb8a-ff13b2c6d526 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:29.005747 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 267/536] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:29.014399 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bd02d-e7fc-4c1e-a594-9cd32c6c3506 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:29.030881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:29.046388 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad15c99-2f19-41a5-8ef8-ed52af280f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:29.047074 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad15c99-2f19-41a5-8ef8-ed52af280f4a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:29.047359 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad15c99-2f19-41a5-8ef8-ed52af280f4a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:29.047641 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad15c99-2f19-41a5-8ef8-ed52af280f4a None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:29.048444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad15c99-2f19-41a5-8ef8-ed52af280f4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:29.048761 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad15c99-2f19-41a5-8ef8-ed52af280f4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:29.050235 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad15c99-2f19-41a5-8ef8-ed52af280f4a None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:29.096438 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 270/537] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:29.102475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad15c99-2f19-41a5-8ef8-ed52af280f4a None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:29.235448 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:29.252661 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bd02d-e7fc-4c1e-a594-9cd32c6c3506 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:29.270215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad15c99-2f19-41a5-8ef8-ed52af280f4a None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:29.294773 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2960641-af70-4217-84c5-b4a5ac86f091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:29.295604 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2960641-af70-4217-84c5-b4a5ac86f091 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:29.295885 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2960641-af70-4217-84c5-b4a5ac86f091 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:29.296164 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2960641-af70-4217-84c5-b4a5ac86f091 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:29.310286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2960641-af70-4217-84c5-b4a5ac86f091 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:29.310286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2960641-af70-4217-84c5-b4a5ac86f091 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:29.319838 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366760a-dbad-4343-a87b-637735297171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:29.320526 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366760a-dbad-4343-a87b-637735297171 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:29.320889 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366760a-dbad-4343-a87b-637735297171 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:29.321339 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366760a-dbad-4343-a87b-637735297171 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:29.326229 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366760a-dbad-4343-a87b-637735297171 None admin] RBAC: Adding query filter params (role.id=c9206a4a0d49473f87a45d6c8abb2e51, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=319fb9f41afb48e68c34bdb1f4ba114b) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:29.326628 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366760a-dbad-4343-a87b-637735297171 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:29.328209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366760a-dbad-4343-a87b-637735297171 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:29.334733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-d2960641-af70-4217-84c5-b4a5ac86f091 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:29.334733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:29.334733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:29.334733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:29.334733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:29.334733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:29.334733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:29.334733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:29.334733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:29.334733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:29.334733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:29.334733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:29.334733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:29.334733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:29.334733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:29.335713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:29.337184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:29.337184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:29.337184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:29.337184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:29.337184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:29.337184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:29.337184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:29.337184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:29.342283 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 268/538] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:29.342518 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 271/539] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:18:29 2022] GET /identity/v3/role_assignments?user.id=319fb9f41afb48e68c34bdb1f4ba114b&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=c9206a4a0d49473f87a45d6c8abb2e51 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:29.344316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2960641-af70-4217-84c5-b4a5ac86f091 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:29.373460 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38caea-d8ee-4f0f-b848-134ef621d370 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:29.373798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38caea-d8ee-4f0f-b848-134ef621d370 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:29.374702 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38caea-d8ee-4f0f-b848-134ef621d370 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:29.376425 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 272/540] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:18:29.384914 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed123c9-c397-49d7-8832-72baf0e608a3 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:29.385359 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed123c9-c397-49d7-8832-72baf0e608a3 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:29.385749 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed123c9-c397-49d7-8832-72baf0e608a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:29.406044 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-8ed123c9-c397-49d7-8832-72baf0e608a3 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:29.407729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-8ed123c9-c397-49d7-8832-72baf0e608a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:29.509894 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:29.553555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 273/541] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:29.560681 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2960641-af70-4217-84c5-b4a5ac86f091 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:29.577212 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed123c9-c397-49d7-8832-72baf0e608a3 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:29.620551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64546c01-61a1-41b7-b7ce-b5789ebd2a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:29.621228 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64546c01-61a1-41b7-b7ce-b5789ebd2a6b None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:29.621507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64546c01-61a1-41b7-b7ce-b5789ebd2a6b None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:29.621781 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64546c01-61a1-41b7-b7ce-b5789ebd2a6b None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:29.622632 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64546c01-61a1-41b7-b7ce-b5789ebd2a6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:29.622943 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64546c01-61a1-41b7-b7ce-b5789ebd2a6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:29.624498 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64546c01-61a1-41b7-b7ce-b5789ebd2a6b None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:29.657900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 269/542] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:29.664154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64546c01-61a1-41b7-b7ce-b5789ebd2a6b None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:29.690044 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:29.747847 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed123c9-c397-49d7-8832-72baf0e608a3 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:29.783053 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457081b1-a79a-49e6-b886-6cb26e2348e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:29.783875 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457081b1-a79a-49e6-b886-6cb26e2348e1 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:29.784202 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457081b1-a79a-49e6-b886-6cb26e2348e1 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:29.784477 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457081b1-a79a-49e6-b886-6cb26e2348e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:29.795688 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457081b1-a79a-49e6-b886-6cb26e2348e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-457081b1-a79a-49e6-b886-6cb26e2348e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:29.803779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:29.804881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:29.806389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:29.806389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:29.806389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:29.806389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:29.806389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:29.806389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:29.806389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:29.806389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:29.806389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:29.806389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:29.806389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:29.806389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:29.806389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 274/543] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:29.815251 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457081b1-a79a-49e6-b886-6cb26e2348e1 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:29.834421 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64546c01-61a1-41b7-b7ce-b5789ebd2a6b None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:29.873805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:29.880076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d06240-b2fc-4b62-9d59-62043c6cf28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:29.880929 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d06240-b2fc-4b62-9d59-62043c6cf28e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:29.881218 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d06240-b2fc-4b62-9d59-62043c6cf28e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:29.881495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d06240-b2fc-4b62-9d59-62043c6cf28e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:29.882751 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d06240-b2fc-4b62-9d59-62043c6cf28e None admin] RBAC: Adding query filter params (role.id=c9206a4a0d49473f87a45d6c8abb2e51, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=3d63f8a58ef24de897ab7fd0828ec467) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:29.883140 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d06240-b2fc-4b62-9d59-62043c6cf28e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:29.884857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d06240-b2fc-4b62-9d59-62043c6cf28e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:29.894313 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 270/544] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:18:29 2022] GET /identity/v3/role_assignments?user.id=3d63f8a58ef24de897ab7fd0828ec467&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=c9206a4a0d49473f87a45d6c8abb2e51 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:29.900099 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f181d0-58bd-4935-a538-8ecb598161ef None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:29.900515 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f181d0-58bd-4935-a538-8ecb598161ef None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:29.901768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f181d0-58bd-4935-a538-8ecb598161ef None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:29.906320 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 271/545] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:29.910015 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a85551-d902-4637-82d6-c412ebf3f717 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:29.911136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a85551-d902-4637-82d6-c412ebf3f717 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:29.911426 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a85551-d902-4637-82d6-c412ebf3f717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:29.950181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-f6a85551-d902-4637-82d6-c412ebf3f717 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:29.951489 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-f6a85551-d902-4637-82d6-c412ebf3f717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:29.974216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457081b1-a79a-49e6-b886-6cb26e2348e1 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:30.019879 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff98a19-4768-4076-96ea-c1187d7bfa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:30.020558 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff98a19-4768-4076-96ea-c1187d7bfa24 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:30.020837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff98a19-4768-4076-96ea-c1187d7bfa24 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:30.021112 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff98a19-4768-4076-96ea-c1187d7bfa24 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:30.021874 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff98a19-4768-4076-96ea-c1187d7bfa24 None admin] RBAC: Adding query filter params (name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:30.022178 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff98a19-4768-4076-96ea-c1187d7bfa24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:30.023633 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff98a19-4768-4076-96ea-c1187d7bfa24 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:30.032704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 275/546] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:30.038193 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff98a19-4768-4076-96ea-c1187d7bfa24 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:30.064981 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 272/547] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:30.112816 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a85551-d902-4637-82d6-c412ebf3f717 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:30.127153 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:30.171916 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff98a19-4768-4076-96ea-c1187d7bfa24 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:30.177149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:30.217181 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c63b6-94f8-482d-9437-263c81e8e5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:30.217995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c63b6-94f8-482d-9437-263c81e8e5ed None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:30.218291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c63b6-94f8-482d-9437-263c81e8e5ed None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:30.218565 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c63b6-94f8-482d-9437-263c81e8e5ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:30.232934 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c63b6-94f8-482d-9437-263c81e8e5ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:30.234626 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c63b6-94f8-482d-9437-263c81e8e5ed None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:30.257045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-b38c63b6-94f8-482d-9437-263c81e8e5ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:30.257045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:30.257045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:30.257045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:30.257045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:30.257045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:30.257045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:30.257045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:30.257045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:30.257045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:30.257045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:30.257045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:30.257045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:30.257045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:30.257045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:30.258486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:30.259869 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:30.259869 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:30.259869 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:30.259869 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:30.259869 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 276/548] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:30.268279 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c63b6-94f8-482d-9437-263c81e8e5ed None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:30.349481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a85551-d902-4637-82d6-c412ebf3f717 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:30.428043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd5e0d-d509-4723-b6fe-dbfab4d17cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:30.428850 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd5e0d-d509-4723-b6fe-dbfab4d17cc3 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:30.429520 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd5e0d-d509-4723-b6fe-dbfab4d17cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:30.429520 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd5e0d-d509-4723-b6fe-dbfab4d17cc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:30.443733 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c63b6-94f8-482d-9437-263c81e8e5ed None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:30.452283 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd5e0d-d509-4723-b6fe-dbfab4d17cc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-a5cd5e0d-d509-4723-b6fe-dbfab4d17cc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:30.469391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:30.470763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:30.471907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:30.471907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:30.471907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:30.471907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:30.471907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:30.471907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:30.471907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:30.471907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:30.471907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:30.471907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:30.471907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:30.471907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:30.478967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 273/549] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:30.487422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd5e0d-d509-4723-b6fe-dbfab4d17cc3 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:30.499595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aea0cc-4076-4a3b-8fe4-35b2232dc77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:30.500653 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aea0cc-4076-4a3b-8fe4-35b2232dc77d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:30.501171 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aea0cc-4076-4a3b-8fe4-35b2232dc77d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:30.501657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aea0cc-4076-4a3b-8fe4-35b2232dc77d None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:30.502776 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aea0cc-4076-4a3b-8fe4-35b2232dc77d None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:30.507089 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aea0cc-4076-4a3b-8fe4-35b2232dc77d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:30.509702 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aea0cc-4076-4a3b-8fe4-35b2232dc77d None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:30.537250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 277/550] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:30.547629 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aea0cc-4076-4a3b-8fe4-35b2232dc77d None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:30.581946 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:30.597831 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:30.637405 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd5e0d-d509-4723-b6fe-dbfab4d17cc3 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:30.706363 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b551841-4105-4c40-b10c-8d43b03722e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:30.708463 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b551841-4105-4c40-b10c-8d43b03722e3 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:30.708963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b551841-4105-4c40-b10c-8d43b03722e3 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:30.709607 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b551841-4105-4c40-b10c-8d43b03722e3 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:30.712796 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b551841-4105-4c40-b10c-8d43b03722e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:30.713419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b551841-4105-4c40-b10c-8d43b03722e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:30.715399 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b551841-4105-4c40-b10c-8d43b03722e3 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:30.734761 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 274/551] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:30.741082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b551841-4105-4c40-b10c-8d43b03722e3 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:30.798602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aea0cc-4076-4a3b-8fe4-35b2232dc77d None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:30.875666 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9fb87-b840-4fe6-b4e6-4ff79590f9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:30.876497 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9fb87-b840-4fe6-b4e6-4ff79590f9c4 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:30.876779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9fb87-b840-4fe6-b4e6-4ff79590f9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:30.877055 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9fb87-b840-4fe6-b4e6-4ff79590f9c4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:30.888771 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9fb87-b840-4fe6-b4e6-4ff79590f9c4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:30.890542 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9fb87-b840-4fe6-b4e6-4ff79590f9c4 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-73f9fb87-b840-4fe6-b4e6-4ff79590f9c4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:30.915319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:30.916901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:30.917978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:30.917978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:30.917978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:30.917978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:30.917978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:30.917978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:30.917978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:30.917978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:30.917978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:18:30.917978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:30.917978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 278/552] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:18:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:30.925997 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9fb87-b840-4fe6-b4e6-4ff79590f9c4 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:30.942676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b551841-4105-4c40-b10c-8d43b03722e3 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:30.978823 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:30.987278 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff309b-3f5e-489b-836a-4e39509f40e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:30.988444 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff309b-3f5e-489b-836a-4e39509f40e4 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:30.988931 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff309b-3f5e-489b-836a-4e39509f40e4 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:30.989435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff309b-3f5e-489b-836a-4e39509f40e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:31.003649 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff309b-3f5e-489b-836a-4e39509f40e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:31.005554 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff309b-3f5e-489b-836a-4e39509f40e4 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:31.027253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-1eff309b-3f5e-489b-836a-4e39509f40e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:31.027253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:31.027253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:31.027253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:31.027253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:31.027253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:31.027253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:31.027253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:31.027253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:31.027253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:31.027253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:31.027253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:31.027253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:31.027253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:31.027253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:31.028566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:31.030137 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:31.030137 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:31.030137 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:31.030137 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:31.033057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 275/553] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:31.043819 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff309b-3f5e-489b-836a-4e39509f40e4 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:31.087818 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:31.114874 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9fb87-b840-4fe6-b4e6-4ff79590f9c4 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:31.159514 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643640d-ddeb-4ecd-891f-520a003e9ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:31.160665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643640d-ddeb-4ecd-891f-520a003e9ac2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:31.161189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643640d-ddeb-4ecd-891f-520a003e9ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:31.161683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643640d-ddeb-4ecd-891f-520a003e9ac2 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:31.162866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643640d-ddeb-4ecd-891f-520a003e9ac2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:31.163459 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643640d-ddeb-4ecd-891f-520a003e9ac2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:31.165261 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643640d-ddeb-4ecd-891f-520a003e9ac2 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:31.186929 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 279/554] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 20:18:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:31.195380 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643640d-ddeb-4ecd-891f-520a003e9ac2 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:31.292503 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff309b-3f5e-489b-836a-4e39509f40e4 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:31.310073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643640d-ddeb-4ecd-891f-520a003e9ac2 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:31.351287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c5b3f-9133-451d-893c-28426ef37394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:31.352038 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c5b3f-9133-451d-893c-28426ef37394 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:31.352388 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c5b3f-9133-451d-893c-28426ef37394 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:31.352746 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c5b3f-9133-451d-893c-28426ef37394 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:31.358251 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:31.372627 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2373b-f1fb-4bf1-af24-6e22d9b90276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:31.373330 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2373b-f1fb-4bf1-af24-6e22d9b90276 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:31.373646 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2373b-f1fb-4bf1-af24-6e22d9b90276 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:31.373924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2373b-f1fb-4bf1-af24-6e22d9b90276 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:31.374724 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2373b-f1fb-4bf1-af24-6e22d9b90276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:31.375043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2373b-f1fb-4bf1-af24-6e22d9b90276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:31.376454 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2373b-f1fb-4bf1-af24-6e22d9b90276 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:31.401730 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 276/555] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:31.407453 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2373b-f1fb-4bf1-af24-6e22d9b90276 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:31.480815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:31.486777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c5b3f-9133-451d-893c-28426ef37394 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:31.492881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c5b3f-9133-451d-893c-28426ef37394 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:31.569483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2373b-f1fb-4bf1-af24-6e22d9b90276 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:31.614941 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46baa06d-e55d-47e9-90aa-d0e98dd78a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:31.615751 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46baa06d-e55d-47e9-90aa-d0e98dd78a1d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:31.616034 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46baa06d-e55d-47e9-90aa-d0e98dd78a1d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:31.616308 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46baa06d-e55d-47e9-90aa-d0e98dd78a1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:31.626416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46baa06d-e55d-47e9-90aa-d0e98dd78a1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:31.627926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46baa06d-e55d-47e9-90aa-d0e98dd78a1d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:31.628354 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 280/556] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:31.630599 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb0c08-5b2d-4d60-b756-da8142612e51 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:31.631705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb0c08-5b2d-4d60-b756-da8142612e51 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:31.632198 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb0c08-5b2d-4d60-b756-da8142612e51 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:31.634093 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 281/557] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:31.635061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb0c08-5b2d-4d60-b756-da8142612e51 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-46baa06d-e55d-47e9-90aa-d0e98dd78a1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:31.650692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:31.651956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:31.653003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:31.653003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:31.653003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:31.653003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:31.653003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:31.653003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:31.653003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:31.653003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:31.653003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:18:31.653003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:31.657100 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 277/558] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:18:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:31.658748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9e54e-d330-4af9-b395-741ecf0c7190 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:31.659036 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9e54e-d330-4af9-b395-741ecf0c7190 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:31.659364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9e54e-d330-4af9-b395-741ecf0c7190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:31.680638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:31.681874 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-a6b9e54e-d330-4af9-b395-741ecf0c7190 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:31.683991 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-a6b9e54e-d330-4af9-b395-741ecf0c7190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:31.764814 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb0c08-5b2d-4d60-b756-da8142612e51 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:31.774400 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 278/559] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:31.775555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9e54e-d330-4af9-b395-741ecf0c7190 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:31.816658 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce090fa-3a58-4736-a05b-66d18d4f2d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:31.817356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce090fa-3a58-4736-a05b-66d18d4f2d49 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:31.817636 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce090fa-3a58-4736-a05b-66d18d4f2d49 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:31.817906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce090fa-3a58-4736-a05b-66d18d4f2d49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:31.819953 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:31.831576 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce090fa-3a58-4736-a05b-66d18d4f2d49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:31.832888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce090fa-3a58-4736-a05b-66d18d4f2d49 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:31.853260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-cce090fa-3a58-4736-a05b-66d18d4f2d49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:31.853260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:31.853260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:31.853260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:31.853260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:31.853260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:31.853260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:31.853260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:31.853260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:31.853260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:31.853260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:31.853260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:31.853260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:31.853260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:31.853260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:31.854423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:31.855424 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:31.855424 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:31.855424 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:31.855424 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:31.855424 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 282/560] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:31.855734 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce090fa-3a58-4736-a05b-66d18d4f2d49 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:31.895934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9e54e-d330-4af9-b395-741ecf0c7190 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:31.935735 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc1684-b820-4a50-af61-08486a2f7e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:31.936875 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc1684-b820-4a50-af61-08486a2f7e7a None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:31.937369 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc1684-b820-4a50-af61-08486a2f7e7a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:31.937856 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc1684-b820-4a50-af61-08486a2f7e7a None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:31.938957 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc1684-b820-4a50-af61-08486a2f7e7a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:31.939490 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc1684-b820-4a50-af61-08486a2f7e7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:31.941184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc1684-b820-4a50-af61-08486a2f7e7a None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:31.953846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:31.960430 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 279/561] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:18:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:31.961671 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc1684-b820-4a50-af61-08486a2f7e7a None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:31.972422 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce090fa-3a58-4736-a05b-66d18d4f2d49 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:32.010089 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01752d5-deae-445d-b822-fe6bbd7aaaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:32.010968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01752d5-deae-445d-b822-fe6bbd7aaaca None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:32.011283 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01752d5-deae-445d-b822-fe6bbd7aaaca None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:32.011849 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01752d5-deae-445d-b822-fe6bbd7aaaca None admin] PATH_INFO: `/v3/roles/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:32.023138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01752d5-deae-445d-b822-fe6bbd7aaaca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-b01752d5-deae-445d-b822-fe6bbd7aaaca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:32.031610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:32.032729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:32.033667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:32.033667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:32.033667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:32.033667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:32.033667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:32.033667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:32.033667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:32.033667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:32.033667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:32.033667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:32.033667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:32.033667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:32.037650 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 283/562] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:32.038615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01752d5-deae-445d-b822-fe6bbd7aaaca None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:32.058722 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:32.071496 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc1684-b820-4a50-af61-08486a2f7e7a None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:32.106599 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd55ca-dac8-4a92-81ce-fd01b3d9a2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:32.107268 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd55ca-dac8-4a92-81ce-fd01b3d9a2ce None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:32.107559 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd55ca-dac8-4a92-81ce-fd01b3d9a2ce None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:32.107810 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd55ca-dac8-4a92-81ce-fd01b3d9a2ce None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:32.108525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd55ca-dac8-4a92-81ce-fd01b3d9a2ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:32.108993 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd55ca-dac8-4a92-81ce-fd01b3d9a2ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:32.110194 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd55ca-dac8-4a92-81ce-fd01b3d9a2ce None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:32.121567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 280/563] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:32.122396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd55ca-dac8-4a92-81ce-fd01b3d9a2ce None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:32.156126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01752d5-deae-445d-b822-fe6bbd7aaaca None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:32.193259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ba6dd-c9ee-4042-892f-a087ab205ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:32.194602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ba6dd-c9ee-4042-892f-a087ab205ca0 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:32.194602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ba6dd-c9ee-4042-892f-a087ab205ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:32.194602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ba6dd-c9ee-4042-892f-a087ab205ca0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:32.206331 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ba6dd-c9ee-4042-892f-a087ab205ca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:32.207827 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ba6dd-c9ee-4042-892f-a087ab205ca0 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:32.217024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:32.227480 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd55ca-dac8-4a92-81ce-fd01b3d9a2ce None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:32.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-384ba6dd-c9ee-4042-892f-a087ab205ca0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:32.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:32.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:32.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:32.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:32.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:32.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:32.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:32.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:32.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:32.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:32.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:32.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:32.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:32.227991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:32.228966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:32.229978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:32.229978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:32.229978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:32.229978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:32.229978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 284/564] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:32.230860 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d3bdb-4aff-42f7-8672-69fa1382c992 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:32.230860 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d3bdb-4aff-42f7-8672-69fa1382c992 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:32.231151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d3bdb-4aff-42f7-8672-69fa1382c992 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:32.236511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 285/565] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:32.237180 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d3bdb-4aff-42f7-8672-69fa1382c992 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:32.269074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46549a-50b6-47d7-b769-dc41858b2e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:32.269748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46549a-50b6-47d7-b769-dc41858b2e28 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:32.270032 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46549a-50b6-47d7-b769-dc41858b2e28 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:32.270342 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46549a-50b6-47d7-b769-dc41858b2e28 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:32.271089 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46549a-50b6-47d7-b769-dc41858b2e28 None admin] RBAC: Adding query filter params (name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:32.271395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46549a-50b6-47d7-b769-dc41858b2e28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:32.272781 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46549a-50b6-47d7-b769-dc41858b2e28 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:32.281743 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 281/566] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:32.282582 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46549a-50b6-47d7-b769-dc41858b2e28 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:32.299068 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:32.352157 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d3bdb-4aff-42f7-8672-69fa1382c992 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:32.388817 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f531-152f-4f10-b0af-0ba85a24db71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:32.389616 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f531-152f-4f10-b0af-0ba85a24db71 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:32.389861 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f531-152f-4f10-b0af-0ba85a24db71 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:32.390133 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f531-152f-4f10-b0af-0ba85a24db71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:32.391205 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46549a-50b6-47d7-b769-dc41858b2e28 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:32.397374 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535f531-152f-4f10-b0af-0ba85a24db71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:32.398958 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535f531-152f-4f10-b0af-0ba85a24db71 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:32.413801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-d535f531-152f-4f10-b0af-0ba85a24db71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:32.413801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:32.413801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:32.413801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:32.413801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:32.413801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:32.413801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:32.413801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:32.413801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:32.413801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:32.413801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:32.413801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:32.413801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:32.413801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:32.413801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:32.415025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:32.416012 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:32.416012 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:32.416012 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:32.416012 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:32.416012 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:32.416012 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:32.416012 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:32.416012 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:32.416012 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 286/567] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:32.416990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f261ac-484a-4859-a90f-8cc4d185aa75 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:32.416990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f261ac-484a-4859-a90f-8cc4d185aa75 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:32.416990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f261ac-484a-4859-a90f-8cc4d185aa75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:32.435406 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a081956-085a-4175-bea0-b89460d01ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:32.436088 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a081956-085a-4175-bea0-b89460d01ad3 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:32.436372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a081956-085a-4175-bea0-b89460d01ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:32.436648 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a081956-085a-4175-bea0-b89460d01ad3 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:32.437434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a081956-085a-4175-bea0-b89460d01ad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:32.437679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a081956-085a-4175-bea0-b89460d01ad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:32.438378 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-63f261ac-484a-4859-a90f-8cc4d185aa75 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:32.439091 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a081956-085a-4175-bea0-b89460d01ad3 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:32.439584 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-63f261ac-484a-4859-a90f-8cc4d185aa75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:32.452059 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 282/568] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:32.453803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a081956-085a-4175-bea0-b89460d01ad3 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:32.505715 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:32.522457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 287/569] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:32.523575 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f261ac-484a-4859-a90f-8cc4d185aa75 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:32.547689 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:32.583213 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a081956-085a-4175-bea0-b89460d01ad3 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:32.620096 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bad48-62f1-4a57-9d23-8ffecc1af526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:32.621621 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bad48-62f1-4a57-9d23-8ffecc1af526 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:32.621906 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bad48-62f1-4a57-9d23-8ffecc1af526 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:32.622179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bad48-62f1-4a57-9d23-8ffecc1af526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:32.629437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f261ac-484a-4859-a90f-8cc4d185aa75 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:32.632356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bad48-62f1-4a57-9d23-8ffecc1af526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:32.633845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bad48-62f1-4a57-9d23-8ffecc1af526 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:32.651877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-246bad48-62f1-4a57-9d23-8ffecc1af526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:32.651877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:32.651877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:32.651877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:32.651877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:32.651877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:32.651877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:32.651877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:32.651877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:32.651877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:32.651877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:32.651877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:32.651877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:32.651877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:32.651877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:32.652926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:32.653942 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:32.653942 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:32.653942 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:32.653942 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:32.657832 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 283/570] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:32.658698 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bad48-62f1-4a57-9d23-8ffecc1af526 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:32.668622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb787d8-4282-43cf-bd43-574acdf3c09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:32.669498 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb787d8-4282-43cf-bd43-574acdf3c09a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:32.669646 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb787d8-4282-43cf-bd43-574acdf3c09a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:32.669904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb787d8-4282-43cf-bd43-574acdf3c09a None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:32.670750 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb787d8-4282-43cf-bd43-574acdf3c09a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:32.671309 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb787d8-4282-43cf-bd43-574acdf3c09a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:32.672535 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb787d8-4282-43cf-bd43-574acdf3c09a None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:32.684711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 288/571] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:32.685385 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb787d8-4282-43cf-bd43-574acdf3c09a None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:32.745355 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:32.769519 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bad48-62f1-4a57-9d23-8ffecc1af526 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:32.799449 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:32.801887 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb787d8-4282-43cf-bd43-574acdf3c09a None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:32.818572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dbbf4-84a9-4222-9b38-c1616057af1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:32.819828 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dbbf4-84a9-4222-9b38-c1616057af1e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:32.820118 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dbbf4-84a9-4222-9b38-c1616057af1e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:32.820378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dbbf4-84a9-4222-9b38-c1616057af1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:32.827907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049dbbf4-84a9-4222-9b38-c1616057af1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:32.829374 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049dbbf4-84a9-4222-9b38-c1616057af1e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:32.842128 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b27df-80db-48f1-91c2-e17ed765a77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:32.843024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b27df-80db-48f1-91c2-e17ed765a77c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:32.843309 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b27df-80db-48f1-91c2-e17ed765a77c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:32.843582 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b27df-80db-48f1-91c2-e17ed765a77c None admin] PATH_INFO: `/v3/roles/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:32.844259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-049dbbf4-84a9-4222-9b38-c1616057af1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:32.844259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:32.844259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:32.844259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:32.844259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:32.844259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:32.844259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:32.844259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:32.844259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:32.844259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:32.844259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:32.844259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:32.844259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:32.844259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:32.844259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:32.845182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:32.846726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:32.846726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:32.846726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:32.846726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:32.846726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:32.846726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:32.846726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:32.846726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:32.846726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 284/572] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:32.846726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dbbf4-84a9-4222-9b38-c1616057af1e None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:32.859486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b27df-80db-48f1-91c2-e17ed765a77c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-e14b27df-80db-48f1-91c2-e17ed765a77c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:32.875117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:32.876253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:32.877215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:32.877215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:32.877215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:32.877215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:32.877215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:32.877215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:32.877215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:32.877215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:32.877215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:32.877215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:32.877215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:32.877215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:32.880915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 289/573] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:32.880915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1593b6-23ee-4953-9700-7226a7d7b350 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:32.880915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1593b6-23ee-4953-9700-7226a7d7b350 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:32.880915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1593b6-23ee-4953-9700-7226a7d7b350 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:32.882157 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 290/574] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:32.886469 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1593b6-23ee-4953-9700-7226a7d7b350 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:32.958132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dbbf4-84a9-4222-9b38-c1616057af1e None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:32.994856 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369df2d3-2269-4ea3-a660-6990574b2d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:32.995533 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369df2d3-2269-4ea3-a660-6990574b2d80 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:32.995819 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369df2d3-2269-4ea3-a660-6990574b2d80 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:32.996100 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369df2d3-2269-4ea3-a660-6990574b2d80 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:32.996823 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369df2d3-2269-4ea3-a660-6990574b2d80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:32.997136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369df2d3-2269-4ea3-a660-6990574b2d80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:32.998800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369df2d3-2269-4ea3-a660-6990574b2d80 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:33.000729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1593b6-23ee-4953-9700-7226a7d7b350 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:33.010478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:33.016280 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 285/575] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:33.016280 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369df2d3-2269-4ea3-a660-6990574b2d80 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:33.037074 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7652b-c1f7-43b3-b172-1388682d011e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:33.037735 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7652b-c1f7-43b3-b172-1388682d011e None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:33.038014 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7652b-c1f7-43b3-b172-1388682d011e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:33.038312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7652b-c1f7-43b3-b172-1388682d011e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:33.039138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7652b-c1f7-43b3-b172-1388682d011e None admin] RBAC: Adding query filter params (role.id=c9206a4a0d49473f87a45d6c8abb2e51, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=54395ae480884de38fc811a9e82c1f9e) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:33.039558 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7652b-c1f7-43b3-b172-1388682d011e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:33.041007 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7652b-c1f7-43b3-b172-1388682d011e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:33.047133 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 291/576] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:18:32 2022] GET /identity/v3/role_assignments?user.id=54395ae480884de38fc811a9e82c1f9e&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=c9206a4a0d49473f87a45d6c8abb2e51 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:33.047903 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7652b-c1f7-43b3-b172-1388682d011e None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:33.070439 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:33.137723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369df2d3-2269-4ea3-a660-6990574b2d80 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:33.162522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7652b-c1f7-43b3-b172-1388682d011e None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:33.175811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1137e-dfa1-4291-8cd4-e69be4c18999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:33.176605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1137e-dfa1-4291-8cd4-e69be4c18999 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:33.176883 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1137e-dfa1-4291-8cd4-e69be4c18999 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:33.177258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1137e-dfa1-4291-8cd4-e69be4c18999 None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:33.178063 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1137e-dfa1-4291-8cd4-e69be4c18999 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:33.178408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1137e-dfa1-4291-8cd4-e69be4c18999 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:33.180078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1137e-dfa1-4291-8cd4-e69be4c18999 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:33.196916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 286/577] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:33.198433 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de09a0-36ad-4bc4-937a-bf599c7a5a1a None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:33.198722 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de09a0-36ad-4bc4-937a-bf599c7a5a1a None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:33.198995 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de09a0-36ad-4bc4-937a-bf599c7a5a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:33.203125 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca4644-f490-482b-8d6c-fa7e56aa7d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:33.203825 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca4644-f490-482b-8d6c-fa7e56aa7d63 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:33.204107 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca4644-f490-482b-8d6c-fa7e56aa7d63 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:33.204380 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca4644-f490-482b-8d6c-fa7e56aa7d63 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:33.205126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca4644-f490-482b-8d6c-fa7e56aa7d63 None admin] RBAC: Adding query filter params (name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:33.205452 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca4644-f490-482b-8d6c-fa7e56aa7d63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:33.206921 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca4644-f490-482b-8d6c-fa7e56aa7d63 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:33.216558 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 292/578] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:33.217424 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca4644-f490-482b-8d6c-fa7e56aa7d63 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:33.222365 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-81de09a0-36ad-4bc4-937a-bf599c7a5a1a None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:33.223585 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-81de09a0-36ad-4bc4-937a-bf599c7a5a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:33.258840 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:33.295764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 287/579] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:33.296926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de09a0-36ad-4bc4-937a-bf599c7a5a1a None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:33.329161 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:33.341406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca4644-f490-482b-8d6c-fa7e56aa7d63 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:33.378482 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a3ad1-92f9-43ff-8925-4e268e489cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:33.379576 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a3ad1-92f9-43ff-8925-4e268e489cb4 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:33.380026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a3ad1-92f9-43ff-8925-4e268e489cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:33.380555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a3ad1-92f9-43ff-8925-4e268e489cb4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:33.389320 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a3ad1-92f9-43ff-8925-4e268e489cb4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:33.391250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a3ad1-92f9-43ff-8925-4e268e489cb4 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:33.399965 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de09a0-36ad-4bc4-937a-bf599c7a5a1a None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-875a3ad1-92f9-43ff-8925-4e268e489cb4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:33.407149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:33.408435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:33.409695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:33.409695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:33.409695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:33.409695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:33.409695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:33.409695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:33.409695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:33.409695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:33.409695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:18:33.409695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:33.416281 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 293/580] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:33.416281 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a3ad1-92f9-43ff-8925-4e268e489cb4 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:33.449295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042636f-a90f-48c7-a8ca-1febfb2df616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:33.450410 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042636f-a90f-48c7-a8ca-1febfb2df616 None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:33.450980 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042636f-a90f-48c7-a8ca-1febfb2df616 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:33.451475 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042636f-a90f-48c7-a8ca-1febfb2df616 None admin] PATH_INFO: `/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/edb1bcb781084f03a4ad2611a09bd77e/roles/c9206a4a0d49473f87a45d6c8abb2e51` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:33.468345 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042636f-a90f-48c7-a8ca-1febfb2df616 None admin] RBAC: Authorizing `identity:create_grant(project_id=59555e7adac54eddb47d1ef18ef870c4, user_id=edb1bcb781084f03a4ad2611a09bd77e, role_id=c9206a4a0d49473f87a45d6c8abb2e51)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:33.469949 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042636f-a90f-48c7-a8ca-1febfb2df616 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:33.494295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 288/581] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:18:33 2022] PUT /identity/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/edb1bcb781084f03a4ad2611a09bd77e/roles/c9206a4a0d49473f87a45d6c8abb2e51 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:18:33.494295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042636f-a90f-48c7-a8ca-1febfb2df616 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:33.532936 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:33.541746 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a3ad1-92f9-43ff-8925-4e268e489cb4 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:33.595299 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c9c9d-b5a2-49e5-897d-f9eebdbb68de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:33.596126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c9c9d-b5a2-49e5-897d-f9eebdbb68de None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:33.596403 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c9c9d-b5a2-49e5-897d-f9eebdbb68de None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:33.596675 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c9c9d-b5a2-49e5-897d-f9eebdbb68de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:33.602957 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:33.613270 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c9c9d-b5a2-49e5-897d-f9eebdbb68de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:33.615250 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c9c9d-b5a2-49e5-897d-f9eebdbb68de None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:33.627124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042636f-a90f-48c7-a8ca-1febfb2df616 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:33.641194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-083c9c9d-b5a2-49e5-897d-f9eebdbb68de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:33.641194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:33.641194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:33.641194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:33.641194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:33.641194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:33.641194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:33.641194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:33.641194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:33.641194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:33.641194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:33.641194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:33.641194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:33.641194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:33.641194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:33.642579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:33.643605 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:33.643605 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:33.643605 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:33.643605 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:33.643605 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 294/582] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:33.645057 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c9c9d-b5a2-49e5-897d-f9eebdbb68de None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:33.683031 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb33720-e9bc-4285-a7f8-a43a819d64cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:33.684311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb33720-e9bc-4285-a7f8-a43a819d64cb None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:33.684601 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb33720-e9bc-4285-a7f8-a43a819d64cb None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:33.684878 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb33720-e9bc-4285-a7f8-a43a819d64cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:33.701069 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb33720-e9bc-4285-a7f8-a43a819d64cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-2cb33720-e9bc-4285-a7f8-a43a819d64cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:33.710889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:33.712186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:33.713125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:33.713125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:33.713125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:33.713125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:33.713125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:33.713125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:33.713125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:33.713125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:33.713125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:33.713125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:33.713125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:33.713125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:33.713125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 289/583] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:33.714235 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb33720-e9bc-4285-a7f8-a43a819d64cb None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:33.775778 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c9c9d-b5a2-49e5-897d-f9eebdbb68de None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:33.824489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e3801-5e3c-4313-a589-48ff8a3db1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:33.826133 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e3801-5e3c-4313-a589-48ff8a3db1bd None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:33.826664 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e3801-5e3c-4313-a589-48ff8a3db1bd None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:33.827439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e3801-5e3c-4313-a589-48ff8a3db1bd None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:33.828764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e3801-5e3c-4313-a589-48ff8a3db1bd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:33.829705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e3801-5e3c-4313-a589-48ff8a3db1bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:33.832548 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e3801-5e3c-4313-a589-48ff8a3db1bd None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:33.838230 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb33720-e9bc-4285-a7f8-a43a819d64cb None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:33.866128 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 295/584] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 20:18:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:33.867069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e3801-5e3c-4313-a589-48ff8a3db1bd None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:33.877382 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:33.890895 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9386e6-0732-4669-86b2-c1bbf5f1767d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:33.892829 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9386e6-0732-4669-86b2-c1bbf5f1767d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:33.893286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9386e6-0732-4669-86b2-c1bbf5f1767d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:33.893730 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9386e6-0732-4669-86b2-c1bbf5f1767d None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:33.894657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9386e6-0732-4669-86b2-c1bbf5f1767d None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:33.895145 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9386e6-0732-4669-86b2-c1bbf5f1767d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:33.897564 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9386e6-0732-4669-86b2-c1bbf5f1767d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:33.919075 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 290/585] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:33.919075 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9386e6-0732-4669-86b2-c1bbf5f1767d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:33.922174 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:33.993778 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e3801-5e3c-4313-a589-48ff8a3db1bd None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:34.037425 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38a1a4-0e84-4b03-91d9-58234036a50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:34.037781 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9386e6-0732-4669-86b2-c1bbf5f1767d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:34.038785 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38a1a4-0e84-4b03-91d9-58234036a50f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:34.039819 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38a1a4-0e84-4b03-91d9-58234036a50f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:34.041207 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38a1a4-0e84-4b03-91d9-58234036a50f None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:34.042203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38a1a4-0e84-4b03-91d9-58234036a50f None admin] RBAC: Adding query filter params (name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:34.042744 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38a1a4-0e84-4b03-91d9-58234036a50f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:34.044358 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38a1a4-0e84-4b03-91d9-58234036a50f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:34.057022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 296/586] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:34.058185 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38a1a4-0e84-4b03-91d9-58234036a50f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:34.079243 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3e65c-244b-4ae8-afb1-cba05f04430a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:34.079766 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3e65c-244b-4ae8-afb1-cba05f04430a None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:34.080046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3e65c-244b-4ae8-afb1-cba05f04430a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:34.080317 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3e65c-244b-4ae8-afb1-cba05f04430a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:34.092102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3e65c-244b-4ae8-afb1-cba05f04430a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:34.093564 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3e65c-244b-4ae8-afb1-cba05f04430a None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:34.112157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-97b3e65c-244b-4ae8-afb1-cba05f04430a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:34.112157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:34.112157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:34.112157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:34.112157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:34.112157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:34.112157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:34.112157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:34.112157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:34.112157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:34.112157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:34.112157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:34.112157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:34.112157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:34.112157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:34.113210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:34.114189 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:34.114189 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:34.114189 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:34.114189 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:34.114189 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 291/587] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:34.115428 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3e65c-244b-4ae8-afb1-cba05f04430a None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:34.150088 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:34.178534 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:34.205650 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38a1a4-0e84-4b03-91d9-58234036a50f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:34.249389 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3e65c-244b-4ae8-afb1-cba05f04430a None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:34.254232 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8516f-9214-49d8-b1bf-da54014cae24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:34.255853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8516f-9214-49d8-b1bf-da54014cae24 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:34.256588 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8516f-9214-49d8-b1bf-da54014cae24 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:34.257298 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8516f-9214-49d8-b1bf-da54014cae24 None admin] PATH_INFO: `/v3/users/glance` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:34.274589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8516f-9214-49d8-b1bf-da54014cae24 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:34.278465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8516f-9214-49d8-b1bf-da54014cae24 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:34.287647 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b68a3-34c0-49b5-a942-952bbee579bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:34.288542 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b68a3-34c0-49b5-a942-952bbee579bc None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:34.288891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b68a3-34c0-49b5-a942-952bbee579bc None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:34.289344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b68a3-34c0-49b5-a942-952bbee579bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:34.300363 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b68a3-34c0-49b5-a942-952bbee579bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:34.301918 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b68a3-34c0-49b5-a942-952bbee579bc None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-60c8516f-9214-49d8-b1bf-da54014cae24 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:34.306888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:34.308545 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:34.310287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:34.310287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:34.310287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:34.310287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:34.310287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:34.310287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:34.310287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:34.310287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:34.310287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:18:34.310287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:34.318784 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 297/588] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:34.318784 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8516f-9214-49d8-b1bf-da54014cae24 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:34.330872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-904b68a3-34c0-49b5-a942-952bbee579bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:34.330872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:34.330872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:34.330872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:34.330872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:34.330872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:34.330872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:34.330872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:34.330872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:34.330872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:34.330872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:34.330872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:34.330872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:34.330872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:34.330872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:34.331935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:34.333028 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:34.333028 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:34.333028 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:34.333028 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:34.337197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 292/589] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:34.338327 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b68a3-34c0-49b5-a942-952bbee579bc None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:34.452444 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b68a3-34c0-49b5-a942-952bbee579bc None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:34.456122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8516f-9214-49d8-b1bf-da54014cae24 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:34.478625 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:34.489236 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:34.497303 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470b7d3-5f4f-450b-a509-0df41ad789e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:34.498060 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470b7d3-5f4f-450b-a509-0df41ad789e4 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:34.498451 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470b7d3-5f4f-450b-a509-0df41ad789e4 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:34.498787 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470b7d3-5f4f-450b-a509-0df41ad789e4 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:34.499684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470b7d3-5f4f-450b-a509-0df41ad789e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:34.500058 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470b7d3-5f4f-450b-a509-0df41ad789e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:34.501575 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470b7d3-5f4f-450b-a509-0df41ad789e4 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:34.502117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4569f-eb70-4823-a4a5-dac109b623b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:34.503025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4569f-eb70-4823-a4a5-dac109b623b0 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:34.503684 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4569f-eb70-4823-a4a5-dac109b623b0 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:34.504162 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4569f-eb70-4823-a4a5-dac109b623b0 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:34.505847 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4569f-eb70-4823-a4a5-dac109b623b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:34.506388 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4569f-eb70-4823-a4a5-dac109b623b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:34.508306 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4569f-eb70-4823-a4a5-dac109b623b0 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:34.525199 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 293/590] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 20:18:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:34.525199 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470b7d3-5f4f-450b-a509-0df41ad789e4 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:34.528392 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 298/591] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:34.530129 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4569f-eb70-4823-a4a5-dac109b623b0 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:34.641459 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4569f-eb70-4823-a4a5-dac109b623b0 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:34.660062 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470b7d3-5f4f-450b-a509-0df41ad789e4 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:34.680817 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61947aeb-caab-4971-9741-16d6ef4700d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:34.681677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61947aeb-caab-4971-9741-16d6ef4700d4 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:34.682074 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61947aeb-caab-4971-9741-16d6ef4700d4 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:34.682720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61947aeb-caab-4971-9741-16d6ef4700d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:34.693724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61947aeb-caab-4971-9741-16d6ef4700d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:34.695454 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61947aeb-caab-4971-9741-16d6ef4700d4 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:34.707427 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612114fd-0c38-4b86-ac81-4308ee9235fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:34.708336 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612114fd-0c38-4b86-ac81-4308ee9235fe None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:34.708803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612114fd-0c38-4b86-ac81-4308ee9235fe None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:34.709247 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612114fd-0c38-4b86-ac81-4308ee9235fe None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:34.710147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612114fd-0c38-4b86-ac81-4308ee9235fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:34.710645 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612114fd-0c38-4b86-ac81-4308ee9235fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:34.712655 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612114fd-0c38-4b86-ac81-4308ee9235fe None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:34.715295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-61947aeb-caab-4971-9741-16d6ef4700d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:34.715295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:34.715295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:34.715295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:34.715295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:34.715295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:34.715295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:34.715295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:34.715295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:34.715295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:34.715295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:34.715295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:34.715295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:34.715295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:34.715295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:34.716516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:34.717780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:34.717780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:34.717780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:34.717780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:34.723642 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 299/592] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:34.723642 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61947aeb-caab-4971-9741-16d6ef4700d4 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:34.737109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 294/593] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:34.738351 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612114fd-0c38-4b86-ac81-4308ee9235fe None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:34.771508 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:34.777362 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:34.840867 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61947aeb-caab-4971-9741-16d6ef4700d4 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:34.878738 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7ef18-d730-4337-95c0-64aa495831d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:34.879549 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7ef18-d730-4337-95c0-64aa495831d3 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:34.879900 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7ef18-d730-4337-95c0-64aa495831d3 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:34.880442 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7ef18-d730-4337-95c0-64aa495831d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:34.884581 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612114fd-0c38-4b86-ac81-4308ee9235fe None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:34.888986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7ef18-d730-4337-95c0-64aa495831d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:34.890684 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7ef18-d730-4337-95c0-64aa495831d3 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:34.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-4ca7ef18-d730-4337-95c0-64aa495831d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:34.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:34.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:34.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:34.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:34.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:34.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:34.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:34.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:34.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:34.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:34.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:34.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:34.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:34.905899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:34.906996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:34.908010 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:34.908010 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:34.908010 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:34.908010 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:34.908010 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:34.908010 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:34.908010 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:34.908010 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:34.908010 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 300/594] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:34.908570 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7ef18-d730-4337-95c0-64aa495831d3 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:34.932055 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289efec-2d55-4a74-81ef-ae9332566f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:34.932652 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289efec-2d55-4a74-81ef-ae9332566f3f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:34.932934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289efec-2d55-4a74-81ef-ae9332566f3f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:34.933210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289efec-2d55-4a74-81ef-ae9332566f3f None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:34.933940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289efec-2d55-4a74-81ef-ae9332566f3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:34.934362 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289efec-2d55-4a74-81ef-ae9332566f3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:34.935778 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289efec-2d55-4a74-81ef-ae9332566f3f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:34.954453 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 295/595] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:34.954453 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289efec-2d55-4a74-81ef-ae9332566f3f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:35.038023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:35.042858 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7ef18-d730-4337-95c0-64aa495831d3 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:35.052108 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:35.072152 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289efec-2d55-4a74-81ef-ae9332566f3f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:35.081574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a907dc3-c087-4296-8e84-bb71d3effeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:35.082215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a907dc3-c087-4296-8e84-bb71d3effeca None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:35.082528 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a907dc3-c087-4296-8e84-bb71d3effeca None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:35.082854 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a907dc3-c087-4296-8e84-bb71d3effeca None admin] PATH_INFO: `/v3/users/neutron` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:35.091882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a907dc3-c087-4296-8e84-bb71d3effeca None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:35.093487 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a907dc3-c087-4296-8e84-bb71d3effeca None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-6a907dc3-c087-4296-8e84-bb71d3effeca None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:35.109682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:35.111006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:35.112245 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:35.112245 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:35.112245 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:35.112245 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:35.112245 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:35.112245 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:35.112245 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:35.112245 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:35.112245 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:18:35.112245 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:35.112245 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 301/596] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:35.113127 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a907dc3-c087-4296-8e84-bb71d3effeca None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:35.116630 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994801f0-3f5b-4f68-95ca-cb8fd377d989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:35.117741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994801f0-3f5b-4f68-95ca-cb8fd377d989 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:35.118485 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994801f0-3f5b-4f68-95ca-cb8fd377d989 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:35.118930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994801f0-3f5b-4f68-95ca-cb8fd377d989 None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:35.119929 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994801f0-3f5b-4f68-95ca-cb8fd377d989 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:35.120413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994801f0-3f5b-4f68-95ca-cb8fd377d989 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:35.122058 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994801f0-3f5b-4f68-95ca-cb8fd377d989 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:35.137492 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 296/597] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:35.139057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994801f0-3f5b-4f68-95ca-cb8fd377d989 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:35.225223 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a907dc3-c087-4296-8e84-bb71d3effeca None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:35.255123 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994801f0-3f5b-4f68-95ca-cb8fd377d989 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:35.262301 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994353bc-6184-4ebd-ab57-891cc9ceb821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:35.263106 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994353bc-6184-4ebd-ab57-891cc9ceb821 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:35.263385 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994353bc-6184-4ebd-ab57-891cc9ceb821 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:35.263658 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994353bc-6184-4ebd-ab57-891cc9ceb821 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:35.271308 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994353bc-6184-4ebd-ab57-891cc9ceb821 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:35.273048 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994353bc-6184-4ebd-ab57-891cc9ceb821 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:35.288574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-994353bc-6184-4ebd-ab57-891cc9ceb821 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:35.288574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:35.288574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:35.288574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:35.288574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:35.288574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:35.288574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:35.288574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:35.288574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:35.288574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:35.288574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:35.288574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:35.288574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:35.288574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:35.288574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:35.290199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:35.291384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:35.291384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:35.291384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:35.291384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:35.291384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:35.291384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:35.291384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:35.291384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:35.295106 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 302/598] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:35.296085 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994353bc-6184-4ebd-ab57-891cc9ceb821 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:35.299308 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf307a-b728-4c91-8ff5-641e8bebd29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:35.300393 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf307a-b728-4c91-8ff5-641e8bebd29d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:35.300907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf307a-b728-4c91-8ff5-641e8bebd29d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:35.301399 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf307a-b728-4c91-8ff5-641e8bebd29d None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:35.302473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf307a-b728-4c91-8ff5-641e8bebd29d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:35.303014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf307a-b728-4c91-8ff5-641e8bebd29d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:35.304668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf307a-b728-4c91-8ff5-641e8bebd29d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:35.317667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:35.325130 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 297/599] 10.222.0.84 () {56 vars in 1139 bytes} [Thu May 19 20:18:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:35.325986 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf307a-b728-4c91-8ff5-641e8bebd29d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:35.328813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:35.411040 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994353bc-6184-4ebd-ab57-891cc9ceb821 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:35.441998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf307a-b728-4c91-8ff5-641e8bebd29d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:35.451102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a424971-31ef-42f5-8f1e-2e15353c90ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:35.452006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a424971-31ef-42f5-8f1e-2e15353c90ee None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:35.452286 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a424971-31ef-42f5-8f1e-2e15353c90ee None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:35.452559 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a424971-31ef-42f5-8f1e-2e15353c90ee None admin] PATH_INFO: `/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/3681f07e23694baba690033a991a23c6/roles/c9206a4a0d49473f87a45d6c8abb2e51` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:35.469948 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a424971-31ef-42f5-8f1e-2e15353c90ee None admin] RBAC: Authorizing `identity:create_grant(project_id=59555e7adac54eddb47d1ef18ef870c4, user_id=3681f07e23694baba690033a991a23c6, role_id=c9206a4a0d49473f87a45d6c8abb2e51)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:35.471466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a424971-31ef-42f5-8f1e-2e15353c90ee None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:35.480880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966ae57-6d68-4c44-99f4-573470123694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:35.481431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966ae57-6d68-4c44-99f4-573470123694 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:35.481709 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966ae57-6d68-4c44-99f4-573470123694 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:35.482014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966ae57-6d68-4c44-99f4-573470123694 None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:35.482840 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966ae57-6d68-4c44-99f4-573470123694 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:35.483124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966ae57-6d68-4c44-99f4-573470123694 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:35.484576 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966ae57-6d68-4c44-99f4-573470123694 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:35.490622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 303/600] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:18:35 2022] PUT /identity/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/3681f07e23694baba690033a991a23c6/roles/c9206a4a0d49473f87a45d6c8abb2e51 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:18:35.490622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d82f6c-9c81-415c-b060-47752323c283 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:35.490622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d82f6c-9c81-415c-b060-47752323c283 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:35.490622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d82f6c-9c81-415c-b060-47752323c283 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:35.490622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 304/601] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:35.490622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d82f6c-9c81-415c-b060-47752323c283 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:35.502811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 298/602] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:35.504181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47f1c9-8968-4f1c-8eef-d6607fb9fc86 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:35.504491 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47f1c9-8968-4f1c-8eef-d6607fb9fc86 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:35.504758 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47f1c9-8968-4f1c-8eef-d6607fb9fc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:35.528412 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-7b47f1c9-8968-4f1c-8eef-d6607fb9fc86 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:35.529875 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-7b47f1c9-8968-4f1c-8eef-d6607fb9fc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:35.591841 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:35.598024 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:35.618784 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 299/603] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:35.619808 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47f1c9-8968-4f1c-8eef-d6607fb9fc86 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:35.630085 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d82f6c-9c81-415c-b060-47752323c283 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:35.684670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb046b9-2e96-47da-8fd4-5184f411ad86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:35.687330 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb046b9-2e96-47da-8fd4-5184f411ad86 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:35.687798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb046b9-2e96-47da-8fd4-5184f411ad86 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:35.688242 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb046b9-2e96-47da-8fd4-5184f411ad86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:35.703358 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb046b9-2e96-47da-8fd4-5184f411ad86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:35.705838 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb046b9-2e96-47da-8fd4-5184f411ad86 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:35.731148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-5fb046b9-2e96-47da-8fd4-5184f411ad86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:35.731148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:35.731148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:35.731148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:35.731148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:35.731148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:35.731148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:35.731148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:35.731148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:35.731148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:35.731148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:35.731148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:35.731148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:35.731148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:35.731148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:35.732186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:35.733207 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:35.733207 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:35.733207 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:35.733207 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:35.733207 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 305/604] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:35.733540 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb046b9-2e96-47da-8fd4-5184f411ad86 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:35.770636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47f1c9-8968-4f1c-8eef-d6607fb9fc86 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:35.807841 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15ef72-3aa6-4390-b56b-63229a8097a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:35.808760 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15ef72-3aa6-4390-b56b-63229a8097a1 None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:35.809044 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15ef72-3aa6-4390-b56b-63229a8097a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:35.809324 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15ef72-3aa6-4390-b56b-63229a8097a1 None admin] PATH_INFO: `/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/319fb9f41afb48e68c34bdb1f4ba114b/roles/c9206a4a0d49473f87a45d6c8abb2e51` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:35.825273 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15ef72-3aa6-4390-b56b-63229a8097a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=59555e7adac54eddb47d1ef18ef870c4, user_id=319fb9f41afb48e68c34bdb1f4ba114b, role_id=c9206a4a0d49473f87a45d6c8abb2e51)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:35.826780 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15ef72-3aa6-4390-b56b-63229a8097a1 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:35.850325 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 300/605] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:18:35 2022] PUT /identity/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/319fb9f41afb48e68c34bdb1f4ba114b/roles/c9206a4a0d49473f87a45d6c8abb2e51 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:18:35.850325 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15ef72-3aa6-4390-b56b-63229a8097a1 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:35.863111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb046b9-2e96-47da-8fd4-5184f411ad86 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:35.895721 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:35.909401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:35.922348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea9607-aab9-4dd0-b47f-4f9af91876ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:35.923476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea9607-aab9-4dd0-b47f-4f9af91876ea None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:35.924456 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea9607-aab9-4dd0-b47f-4f9af91876ea None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:35.924911 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea9607-aab9-4dd0-b47f-4f9af91876ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:35.942520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea9607-aab9-4dd0-b47f-4f9af91876ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-95ea9607-aab9-4dd0-b47f-4f9af91876ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:35.953037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:35.954419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:35.955560 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:35.955560 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:35.955560 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:35.955560 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:35.955560 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:35.955560 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:35.955560 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:35.955560 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:35.955560 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:35.955560 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:35.955560 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:35.955560 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:35.957098 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 306/606] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:35.959968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf81d3-6e31-4ac0-804c-6e82b70baa8d None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:35.961133 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf81d3-6e31-4ac0-804c-6e82b70baa8d None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:35.961859 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf81d3-6e31-4ac0-804c-6e82b70baa8d None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:35.965669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 307/607] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:35.967651 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf81d3-6e31-4ac0-804c-6e82b70baa8d None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:35.987474 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15ef72-3aa6-4390-b56b-63229a8097a1 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:36.029573 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b458857-3f52-4f1b-9aca-f45758afe07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:36.030573 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b458857-3f52-4f1b-9aca-f45758afe07b None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:36.030840 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b458857-3f52-4f1b-9aca-f45758afe07b None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:36.031216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b458857-3f52-4f1b-9aca-f45758afe07b None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:36.032066 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b458857-3f52-4f1b-9aca-f45758afe07b None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:36.032477 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b458857-3f52-4f1b-9aca-f45758afe07b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:36.034039 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b458857-3f52-4f1b-9aca-f45758afe07b None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:36.053107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 301/608] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:36.053107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e039ca5-2748-47d6-87e5-0ba91eb5c062 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:36.053107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e039ca5-2748-47d6-87e5-0ba91eb5c062 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:36.053107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e039ca5-2748-47d6-87e5-0ba91eb5c062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:36.076648 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-8e039ca5-2748-47d6-87e5-0ba91eb5c062 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:36.078534 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-8e039ca5-2748-47d6-87e5-0ba91eb5c062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:36.107087 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf81d3-6e31-4ac0-804c-6e82b70baa8d None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:36.145068 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a53ef-1a2d-4b33-a390-cbd39fc351a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:36.145749 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a53ef-1a2d-4b33-a390-cbd39fc351a5 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:36.146034 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a53ef-1a2d-4b33-a390-cbd39fc351a5 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:36.146345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a53ef-1a2d-4b33-a390-cbd39fc351a5 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:36.147400 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a53ef-1a2d-4b33-a390-cbd39fc351a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:36.147546 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a53ef-1a2d-4b33-a390-cbd39fc351a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:36.149076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a53ef-1a2d-4b33-a390-cbd39fc351a5 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:36.157256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 302/609] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:36.158362 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e039ca5-2748-47d6-87e5-0ba91eb5c062 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:36.166294 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 308/610] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:36.176286 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a53ef-1a2d-4b33-a390-cbd39fc351a5 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:36.197268 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:36.233549 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:36.283117 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e039ca5-2748-47d6-87e5-0ba91eb5c062 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:36.312626 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a53ef-1a2d-4b33-a390-cbd39fc351a5 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:36.319006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c745f84-df4e-4aae-85b4-0c9fc681d408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:36.319836 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c745f84-df4e-4aae-85b4-0c9fc681d408 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:36.320118 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c745f84-df4e-4aae-85b4-0c9fc681d408 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:36.320425 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c745f84-df4e-4aae-85b4-0c9fc681d408 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:36.328123 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c745f84-df4e-4aae-85b4-0c9fc681d408 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:36.329635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c745f84-df4e-4aae-85b4-0c9fc681d408 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:36.344765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-9c745f84-df4e-4aae-85b4-0c9fc681d408 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:36.344765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:36.344765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:36.344765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:36.344765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:36.344765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:36.344765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:36.344765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:36.344765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:36.344765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:36.344765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:36.344765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:36.344765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:36.344765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:36.344765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:36.345989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:36.347132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:36.347132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:36.347132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:36.347132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:36.347132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:36.347132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:36.347132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:36.347132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:36.347132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 303/611] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:36.347132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c745f84-df4e-4aae-85b4-0c9fc681d408 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:36.358060 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35f8cc-8765-4d59-979b-ee2ac9e3ce88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:36.359154 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35f8cc-8765-4d59-979b-ee2ac9e3ce88 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:36.359591 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35f8cc-8765-4d59-979b-ee2ac9e3ce88 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:36.360379 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35f8cc-8765-4d59-979b-ee2ac9e3ce88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:36.374965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35f8cc-8765-4d59-979b-ee2ac9e3ce88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:36.377267 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35f8cc-8765-4d59-979b-ee2ac9e3ce88 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:36.401820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-1d35f8cc-8765-4d59-979b-ee2ac9e3ce88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:36.401820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:36.401820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:36.401820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:36.401820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:36.401820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:36.401820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:36.401820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:36.401820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:36.401820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:36.401820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:36.401820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:36.401820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:36.401820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:36.401820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:36.403145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:36.404269 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:36.404269 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:36.404269 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:36.404269 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:36.404269 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 309/612] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:36.404269 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35f8cc-8765-4d59-979b-ee2ac9e3ce88 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:36.459857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:36.470860 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c745f84-df4e-4aae-85b4-0c9fc681d408 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:36.506221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a6e6d-1059-423d-b5a7-4fac9bcd866b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:36.507218 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a6e6d-1059-423d-b5a7-4fac9bcd866b None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:36.507357 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a6e6d-1059-423d-b5a7-4fac9bcd866b None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:36.507596 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a6e6d-1059-423d-b5a7-4fac9bcd866b None admin] PATH_INFO: `/v3/roles/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:36.510104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:36.517504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35f8cc-8765-4d59-979b-ee2ac9e3ce88 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:36.518784 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a6e6d-1059-423d-b5a7-4fac9bcd866b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-017a6e6d-1059-423d-b5a7-4fac9bcd866b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:36.527082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:36.528371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:36.529530 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:36.529530 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:36.529530 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:36.529530 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:36.529530 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:36.529530 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:36.529530 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:36.529530 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:36.529530 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:36.529530 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:36.529530 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:36.529530 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:36.533108 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 304/613] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:36.534005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a6e6d-1059-423d-b5a7-4fac9bcd866b None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:36.559014 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b759b-dd13-437f-99d5-2257b81caa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:36.559944 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b759b-dd13-437f-99d5-2257b81caa29 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:36.560393 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b759b-dd13-437f-99d5-2257b81caa29 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:36.560830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b759b-dd13-437f-99d5-2257b81caa29 None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:36.561782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b759b-dd13-437f-99d5-2257b81caa29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:36.562243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b759b-dd13-437f-99d5-2257b81caa29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:36.563888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b759b-dd13-437f-99d5-2257b81caa29 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:36.576977 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 310/614] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:36.577983 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b759b-dd13-437f-99d5-2257b81caa29 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:36.640059 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a6e6d-1059-423d-b5a7-4fac9bcd866b None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:36.676582 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad489bd3-31a5-4292-9b16-5124364dea29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:36.677323 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad489bd3-31a5-4292-9b16-5124364dea29 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:36.677543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad489bd3-31a5-4292-9b16-5124364dea29 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:36.677822 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad489bd3-31a5-4292-9b16-5124364dea29 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:36.678618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad489bd3-31a5-4292-9b16-5124364dea29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:36.678926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad489bd3-31a5-4292-9b16-5124364dea29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:36.680531 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad489bd3-31a5-4292-9b16-5124364dea29 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:36.699819 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 305/615] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:36.699819 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad489bd3-31a5-4292-9b16-5124364dea29 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:36.700231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b759b-dd13-437f-99d5-2257b81caa29 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:36.717856 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:36.750206 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d1666-47ed-4a33-9307-0a0cb691c637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:36.751295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d1666-47ed-4a33-9307-0a0cb691c637 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:36.751950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d1666-47ed-4a33-9307-0a0cb691c637 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:36.752458 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d1666-47ed-4a33-9307-0a0cb691c637 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:36.753440 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d1666-47ed-4a33-9307-0a0cb691c637 None admin] RBAC: Adding query filter params (name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:36.753964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d1666-47ed-4a33-9307-0a0cb691c637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:36.755627 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d1666-47ed-4a33-9307-0a0cb691c637 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:36.765169 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 311/616] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:36.766377 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d1666-47ed-4a33-9307-0a0cb691c637 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:36.788448 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:36.815519 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad489bd3-31a5-4292-9b16-5124364dea29 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:36.852470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13166c94-ad5d-4dde-82d4-a47ec17b61a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:36.853382 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13166c94-ad5d-4dde-82d4-a47ec17b61a4 None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:36.853663 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13166c94-ad5d-4dde-82d4-a47ec17b61a4 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:36.853939 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13166c94-ad5d-4dde-82d4-a47ec17b61a4 None admin] PATH_INFO: `/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/3d63f8a58ef24de897ab7fd0828ec467/roles/c9206a4a0d49473f87a45d6c8abb2e51` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:36.869608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13166c94-ad5d-4dde-82d4-a47ec17b61a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=59555e7adac54eddb47d1ef18ef870c4, user_id=3d63f8a58ef24de897ab7fd0828ec467, role_id=c9206a4a0d49473f87a45d6c8abb2e51)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:36.871095 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13166c94-ad5d-4dde-82d4-a47ec17b61a4 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:36.879774 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d1666-47ed-4a33-9307-0a0cb691c637 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:36.889562 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 306/617] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:18:36 2022] PUT /identity/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/3d63f8a58ef24de897ab7fd0828ec467/roles/c9206a4a0d49473f87a45d6c8abb2e51 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:18:36.891132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13166c94-ad5d-4dde-82d4-a47ec17b61a4 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:36.954417 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f666629-eab9-4ba3-9c03-de71a328bac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:36.955265 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f666629-eab9-4ba3-9c03-de71a328bac2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:36.955558 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f666629-eab9-4ba3-9c03-de71a328bac2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:36.955833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f666629-eab9-4ba3-9c03-de71a328bac2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:36.975097 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f666629-eab9-4ba3-9c03-de71a328bac2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:36.976634 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f666629-eab9-4ba3-9c03-de71a328bac2 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:37.008388 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-4f666629-eab9-4ba3-9c03-de71a328bac2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:37.011080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:37.012677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:37.013998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:37.013998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:37.013998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:37.013998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:37.013998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:37.013998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:37.013998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:37.013998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:37.013998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:18:37.013998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:37.013998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 312/618] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:18:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:37.023353 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f666629-eab9-4ba3-9c03-de71a328bac2 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:37.081818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13166c94-ad5d-4dde-82d4-a47ec17b61a4 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:37.143013 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b08990-922b-4a25-84f3-ea59e537567f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:37.144027 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b08990-922b-4a25-84f3-ea59e537567f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:37.144411 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b08990-922b-4a25-84f3-ea59e537567f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:37.144787 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b08990-922b-4a25-84f3-ea59e537567f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:37.157945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b08990-922b-4a25-84f3-ea59e537567f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:37.159643 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b08990-922b-4a25-84f3-ea59e537567f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:37.166782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:37.180963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-02b08990-922b-4a25-84f3-ea59e537567f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:37.180963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:37.180963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:37.180963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:37.180963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:37.180963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:37.180963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:37.180963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:37.180963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:37.180963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:37.180963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:37.180963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:37.180963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:37.180963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:37.180963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:37.182244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:37.183568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:37.183568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:37.183568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:37.183568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:37.186284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 307/619] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:37.190500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b08990-922b-4a25-84f3-ea59e537567f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:37.195826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f666629-eab9-4ba3-9c03-de71a328bac2 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:37.232380 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e93249-ab77-4b75-a63f-279d480f2bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:37.233071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e93249-ab77-4b75-a63f-279d480f2bc5 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:37.233406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e93249-ab77-4b75-a63f-279d480f2bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:37.233691 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e93249-ab77-4b75-a63f-279d480f2bc5 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:37.239160 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e93249-ab77-4b75-a63f-279d480f2bc5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:37.239498 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e93249-ab77-4b75-a63f-279d480f2bc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:37.240995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e93249-ab77-4b75-a63f-279d480f2bc5 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:37.267132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 313/620] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 20:18:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:37.272609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e93249-ab77-4b75-a63f-279d480f2bc5 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:37.320125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b08990-922b-4a25-84f3-ea59e537567f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:37.365879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:37.382248 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e93249-ab77-4b75-a63f-279d480f2bc5 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:37.422831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147114e-0478-4dc1-83fd-1baf419e0d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:37.426300 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147114e-0478-4dc1-83fd-1baf419e0d31 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:37.426300 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147114e-0478-4dc1-83fd-1baf419e0d31 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:37.426300 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147114e-0478-4dc1-83fd-1baf419e0d31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:37.426620 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6c834-0442-466b-834d-992252fa8124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:37.426620 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6c834-0442-466b-834d-992252fa8124 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:37.426620 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6c834-0442-466b-834d-992252fa8124 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:37.426620 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6c834-0442-466b-834d-992252fa8124 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:37.426955 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:37.427128 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6c834-0442-466b-834d-992252fa8124 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:37.427598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6c834-0442-466b-834d-992252fa8124 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:37.429143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6c834-0442-466b-834d-992252fa8124 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:37.437911 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147114e-0478-4dc1-83fd-1baf419e0d31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:37.439273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147114e-0478-4dc1-83fd-1baf419e0d31 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:37.458520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-8147114e-0478-4dc1-83fd-1baf419e0d31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:37.458520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:37.458520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:37.458520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:37.458520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:37.458520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:37.458520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:37.458520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:37.458520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:37.458520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:37.458520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:37.458520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:37.458520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:37.458520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:37.458520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:37.459753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:37.460885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:37.460885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:37.460885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:37.460885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:37.460885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 314/621] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:37.465890 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147114e-0478-4dc1-83fd-1baf419e0d31 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:37.468533 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 308/622] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:37.479319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6c834-0442-466b-834d-992252fa8124 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:37.572617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147114e-0478-4dc1-83fd-1baf419e0d31 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:37.606659 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6c834-0442-466b-834d-992252fa8124 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:37.644952 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5894320-2ae3-412b-81ec-462c9cc11fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:37.645551 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5894320-2ae3-412b-81ec-462c9cc11fa8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:37.645831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5894320-2ae3-412b-81ec-462c9cc11fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:37.646105 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5894320-2ae3-412b-81ec-462c9cc11fa8 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:37.646867 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5894320-2ae3-412b-81ec-462c9cc11fa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:37.647172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5894320-2ae3-412b-81ec-462c9cc11fa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:37.648770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5894320-2ae3-412b-81ec-462c9cc11fa8 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:37.652054 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa0662-fc81-4254-b966-9b0dfae459c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:37.652867 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa0662-fc81-4254-b966-9b0dfae459c5 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:37.653144 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa0662-fc81-4254-b966-9b0dfae459c5 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:37.653415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa0662-fc81-4254-b966-9b0dfae459c5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:37.670684 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 315/623] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:37.674137 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa0662-fc81-4254-b966-9b0dfae459c5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:37.675722 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa0662-fc81-4254-b966-9b0dfae459c5 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:37.677097 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5894320-2ae3-412b-81ec-462c9cc11fa8 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-7afa0662-fc81-4254-b966-9b0dfae459c5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:37.699378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:37.700497 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:37.701553 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:37.701553 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:37.701553 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:37.701553 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:37.701553 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:37.701553 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:37.701553 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:37.701553 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:37.701553 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:18:37.701553 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:37.706416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 309/624] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:18:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:37.708467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa0662-fc81-4254-b966-9b0dfae459c5 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:37.753688 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:37.795036 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:37.854809 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5894320-2ae3-412b-81ec-462c9cc11fa8 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:37.893563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa0662-fc81-4254-b966-9b0dfae459c5 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:37.894232 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14415200-3dab-452f-85b7-486ce5dce0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:37.895336 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14415200-3dab-452f-85b7-486ce5dce0f9 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:37.895804 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14415200-3dab-452f-85b7-486ce5dce0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:37.896238 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14415200-3dab-452f-85b7-486ce5dce0f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:37.904862 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14415200-3dab-452f-85b7-486ce5dce0f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:37.906671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14415200-3dab-452f-85b7-486ce5dce0f9 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:37.923069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-14415200-3dab-452f-85b7-486ce5dce0f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:37.923069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:37.923069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:37.923069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:37.923069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:37.923069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:37.923069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:37.923069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:37.923069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:37.923069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:37.923069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:37.923069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:37.923069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:37.923069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:37.923069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:37.924381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:37.925620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:37.925620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:37.925620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:37.925620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:37.925620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:37.925620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:37.925620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:37.925620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:37.927885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 316/625] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:37.934165 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14415200-3dab-452f-85b7-486ce5dce0f9 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:37.999015 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12173a1-ce96-46ed-ae5d-7609ecda8eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:37.999722 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12173a1-ce96-46ed-ae5d-7609ecda8eff None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:38.000007 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12173a1-ce96-46ed-ae5d-7609ecda8eff None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:38.000346 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12173a1-ce96-46ed-ae5d-7609ecda8eff None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:38.001192 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12173a1-ce96-46ed-ae5d-7609ecda8eff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:38.001740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12173a1-ce96-46ed-ae5d-7609ecda8eff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:38.002989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12173a1-ce96-46ed-ae5d-7609ecda8eff None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:38.036237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 310/626] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:18:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:38.042552 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12173a1-ce96-46ed-ae5d-7609ecda8eff None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:38.054107 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14415200-3dab-452f-85b7-486ce5dce0f9 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:38.063379 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:38.095156 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca2a5c-6fc0-443b-8e3f-8248348899a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:38.096194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca2a5c-6fc0-443b-8e3f-8248348899a0 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:38.096635 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca2a5c-6fc0-443b-8e3f-8248348899a0 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:38.097172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca2a5c-6fc0-443b-8e3f-8248348899a0 None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:38.098146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca2a5c-6fc0-443b-8e3f-8248348899a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:38.098654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca2a5c-6fc0-443b-8e3f-8248348899a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:38.100330 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca2a5c-6fc0-443b-8e3f-8248348899a0 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:38.114851 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 317/627] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:38.121126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca2a5c-6fc0-443b-8e3f-8248348899a0 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:38.215058 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:38.225604 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12173a1-ce96-46ed-ae5d-7609ecda8eff None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:38.253361 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca2a5c-6fc0-443b-8e3f-8248348899a0 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:38.266900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456557eb-5343-40ae-942c-2cd98e8f26ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:38.267802 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456557eb-5343-40ae-942c-2cd98e8f26ca None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:38.268268 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456557eb-5343-40ae-942c-2cd98e8f26ca None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:38.268734 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456557eb-5343-40ae-942c-2cd98e8f26ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:38.298278 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ff504-3842-4b6a-95a6-455288bf665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:38.299446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ff504-3842-4b6a-95a6-455288bf665a None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:38.300109 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ff504-3842-4b6a-95a6-455288bf665a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:38.300401 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ff504-3842-4b6a-95a6-455288bf665a None admin] PATH_INFO: `/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/54395ae480884de38fc811a9e82c1f9e/roles/c9206a4a0d49473f87a45d6c8abb2e51` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:38.317090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ff504-3842-4b6a-95a6-455288bf665a None admin] RBAC: Authorizing `identity:create_grant(project_id=59555e7adac54eddb47d1ef18ef870c4, user_id=54395ae480884de38fc811a9e82c1f9e, role_id=c9206a4a0d49473f87a45d6c8abb2e51)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:38.318764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ff504-3842-4b6a-95a6-455288bf665a None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:38.333838 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 318/628] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:18:38 2022] PUT /identity/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/54395ae480884de38fc811a9e82c1f9e/roles/c9206a4a0d49473f87a45d6c8abb2e51 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:18:38.335248 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d03e0-f594-4007-a7ab-e12ededb1b2b None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:38.335535 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d03e0-f594-4007-a7ab-e12ededb1b2b None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:38.335814 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d03e0-f594-4007-a7ab-e12ededb1b2b None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:38.342129 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 319/629] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:38.342129 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fce51e-52f5-4e67-bcdb-0ef4ff09998c None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:38.342129 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fce51e-52f5-4e67-bcdb-0ef4ff09998c None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:38.342129 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fce51e-52f5-4e67-bcdb-0ef4ff09998c None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:38.346338 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 320/630] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:38.348038 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e5446-94df-4043-90cc-de723f4fa01d None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:38.348330 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e5446-94df-4043-90cc-de723f4fa01d None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:38.348612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e5446-94df-4043-90cc-de723f4fa01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:38.356169 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456557eb-5343-40ae-942c-2cd98e8f26ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:38.357726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456557eb-5343-40ae-942c-2cd98e8f26ca None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:38.368595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-175e5446-94df-4043-90cc-de723f4fa01d None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:38.370416 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-175e5446-94df-4043-90cc-de723f4fa01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:38.384160 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:38.478985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 321/631] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:38.480712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2a45e-8395-434b-a4de-34a61a2a39b8 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:38.481299 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2a45e-8395-434b-a4de-34a61a2a39b8 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:38.481804 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2a45e-8395-434b-a4de-34a61a2a39b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:38.503113 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:38.504358 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-6ff2a45e-8395-434b-a4de-34a61a2a39b8 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:38.505898 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-6ff2a45e-8395-434b-a4de-34a61a2a39b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:38.524985 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 311/632] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:38.525995 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456557eb-5343-40ae-942c-2cd98e8f26ca None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:38.596626 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 322/633] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:38.597320 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2a45e-8395-434b-a4de-34a61a2a39b8 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:38.645546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456557eb-5343-40ae-942c-2cd98e8f26ca None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:38.687111 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c8d85-52dd-4d83-b09e-b67e04a66643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:38.688298 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c8d85-52dd-4d83-b09e-b67e04a66643 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:38.688826 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c8d85-52dd-4d83-b09e-b67e04a66643 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:38.689339 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c8d85-52dd-4d83-b09e-b67e04a66643 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:38.693161 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:38.704890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c8d85-52dd-4d83-b09e-b67e04a66643 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-770c8d85-52dd-4d83-b09e-b67e04a66643 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:38.714171 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:38.715385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:38.716359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:38.716359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:38.716359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:38.716359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:38.716359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:38.716359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:38.716359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:38.716359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:38.716359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:38.716359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:38.716359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:38.716359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:38.719159 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2a45e-8395-434b-a4de-34a61a2a39b8 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:38.720470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 312/634] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:38.721560 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c8d85-52dd-4d83-b09e-b67e04a66643 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:38.768840 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2fbaf-1547-41c6-871e-232941b314f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:38.769566 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2fbaf-1547-41c6-871e-232941b314f9 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:38.769847 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2fbaf-1547-41c6-871e-232941b314f9 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:38.770123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2fbaf-1547-41c6-871e-232941b314f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:38.784271 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:38.784579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2fbaf-1547-41c6-871e-232941b314f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:38.787094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2fbaf-1547-41c6-871e-232941b314f9 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:38.812100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-d1c2fbaf-1547-41c6-871e-232941b314f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:38.812100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:38.812100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:38.812100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:38.812100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:38.812100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:38.812100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:38.812100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:38.812100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:38.812100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:38.812100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:38.812100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:38.812100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:38.812100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:38.812100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:38.813669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:38.819883 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:38.819883 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:38.819883 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:38.819883 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:38.819883 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 323/635] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:38.819883 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2fbaf-1547-41c6-871e-232941b314f9 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:38.847988 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c8d85-52dd-4d83-b09e-b67e04a66643 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:38.888467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94476a-367d-4bbb-8b9f-87d85df9c8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:38.889690 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94476a-367d-4bbb-8b9f-87d85df9c8eb None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:38.890143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94476a-367d-4bbb-8b9f-87d85df9c8eb None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:38.890596 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94476a-367d-4bbb-8b9f-87d85df9c8eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:38.902848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94476a-367d-4bbb-8b9f-87d85df9c8eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-9c94476a-367d-4bbb-8b9f-87d85df9c8eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:38.911923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:38.913223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:38.914443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:38.914443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:38.914443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:38.914443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:38.914443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:38.914443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:38.914443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:38.914443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:38.914443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:38.914443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:38.914443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:38.914443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:38.917260 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 313/636] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:38.918288 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94476a-367d-4bbb-8b9f-87d85df9c8eb None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:38.926067 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2fbaf-1547-41c6-871e-232941b314f9 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:38.958065 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:38.972481 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e3a4b-e3e5-4be7-b695-cd0c84008ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:38.973314 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e3a4b-e3e5-4be7-b695-cd0c84008ae0 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:38.973705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e3a4b-e3e5-4be7-b695-cd0c84008ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:38.974252 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e3a4b-e3e5-4be7-b695-cd0c84008ae0 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:38.975161 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e3a4b-e3e5-4be7-b695-cd0c84008ae0 None admin] RBAC: Adding query filter params (name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:38.975970 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e3a4b-e3e5-4be7-b695-cd0c84008ae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:38.978042 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e3a4b-e3e5-4be7-b695-cd0c84008ae0 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:38.994301 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 324/637] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:38.994301 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e3a4b-e3e5-4be7-b695-cd0c84008ae0 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:39.040968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94476a-367d-4bbb-8b9f-87d85df9c8eb None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:39.055778 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:39.080318 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325756ea-254b-4137-8807-7f871d468432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:39.081378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325756ea-254b-4137-8807-7f871d468432 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:39.081833 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325756ea-254b-4137-8807-7f871d468432 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:39.082290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325756ea-254b-4137-8807-7f871d468432 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:39.083204 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325756ea-254b-4137-8807-7f871d468432 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:39.083767 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325756ea-254b-4137-8807-7f871d468432 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:39.085403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325756ea-254b-4137-8807-7f871d468432 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:39.105564 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 314/638] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:39.106724 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325756ea-254b-4137-8807-7f871d468432 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:39.108263 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e3a4b-e3e5-4be7-b695-cd0c84008ae0 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:39.145594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ba983-8e2d-4926-a36d-9846aa02e044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:39.146484 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ba983-8e2d-4926-a36d-9846aa02e044 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:39.146873 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ba983-8e2d-4926-a36d-9846aa02e044 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:39.147251 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ba983-8e2d-4926-a36d-9846aa02e044 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:39.148110 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ba983-8e2d-4926-a36d-9846aa02e044 None admin] RBAC: Adding query filter params (name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:39.148683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ba983-8e2d-4926-a36d-9846aa02e044 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:39.150333 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ba983-8e2d-4926-a36d-9846aa02e044 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:39.160252 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 325/639] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:39.161056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ba983-8e2d-4926-a36d-9846aa02e044 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:39.219384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:39.221262 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325756ea-254b-4137-8807-7f871d468432 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:39.269436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbe1ea-d091-4eed-94d1-28f6592464ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:39.270708 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbe1ea-d091-4eed-94d1-28f6592464ad None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:39.271261 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbe1ea-d091-4eed-94d1-28f6592464ad None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:39.271545 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbe1ea-d091-4eed-94d1-28f6592464ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:39.273188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ba983-8e2d-4926-a36d-9846aa02e044 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:39.287331 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbe1ea-d091-4eed-94d1-28f6592464ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:39.289811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbe1ea-d091-4eed-94d1-28f6592464ad None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:39.313919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57daef4f-9531-4cb8-8c17-ba9923061212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:39.314829 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daef4f-9531-4cb8-8c17-ba9923061212 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:39.315217 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daef4f-9531-4cb8-8c17-ba9923061212 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:39.315596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daef4f-9531-4cb8-8c17-ba9923061212 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:39.315769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-c2cbe1ea-d091-4eed-94d1-28f6592464ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:39.315769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:39.315769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:39.315769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:39.315769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:39.315769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:39.315769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:39.315769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:39.315769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:39.315769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:39.315769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:39.315769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:39.315769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:39.315769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:39.315769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:39.316921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:39.318139 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:39.318139 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:39.318139 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:39.318139 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:39.318139 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 315/640] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:39.318576 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbe1ea-d091-4eed-94d1-28f6592464ad None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:39.327169 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57daef4f-9531-4cb8-8c17-ba9923061212 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:39.328926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57daef4f-9531-4cb8-8c17-ba9923061212 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:39.345340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-57daef4f-9531-4cb8-8c17-ba9923061212 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:39.345340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:39.345340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:39.345340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:39.345340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:39.345340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:39.345340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:39.345340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:39.345340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:39.345340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:39.345340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:39.345340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:39.345340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:39.345340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:39.345340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:39.346656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:39.347936 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:39.347936 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:39.347936 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:39.347936 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:39.347936 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:39.347936 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:39.347936 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:39.347936 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:39.349360 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:39.350662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 326/641] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:39.351766 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57daef4f-9531-4cb8-8c17-ba9923061212 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:39.455176 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbe1ea-d091-4eed-94d1-28f6592464ad None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:39.486674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57daef4f-9531-4cb8-8c17-ba9923061212 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:39.497633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c54edb-ae32-4718-8b26-9f4733ee439f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:39.498483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c54edb-ae32-4718-8b26-9f4733ee439f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:39.498761 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c54edb-ae32-4718-8b26-9f4733ee439f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:39.499034 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c54edb-ae32-4718-8b26-9f4733ee439f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:39.502727 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:39.510657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c54edb-ae32-4718-8b26-9f4733ee439f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:39.512117 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c54edb-ae32-4718-8b26-9f4733ee439f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:39.526533 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c82753-7e66-4050-9aae-a806872f6895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:39.527206 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c82753-7e66-4050-9aae-a806872f6895 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:39.527490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c82753-7e66-4050-9aae-a806872f6895 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:39.527763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c82753-7e66-4050-9aae-a806872f6895 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:39.528495 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c82753-7e66-4050-9aae-a806872f6895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:39.528800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c82753-7e66-4050-9aae-a806872f6895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:39.529608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-11c54edb-ae32-4718-8b26-9f4733ee439f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:39.529608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:39.529608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:39.529608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:39.529608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:39.529608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:39.529608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:39.529608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:39.529608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:39.529608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:39.529608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:39.529608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:39.529608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:39.529608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:39.529608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:39.531000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:39.531996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:39.531996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:39.531996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:39.531996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:39.532258 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c82753-7e66-4050-9aae-a806872f6895 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:39.532336 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 316/642] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:39.532336 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c54edb-ae32-4718-8b26-9f4733ee439f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:39.547736 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 327/643] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:39.547736 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e89332-8af8-4e1e-8619-2c418103c6c1 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:39.547736 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e89332-8af8-4e1e-8619-2c418103c6c1 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:39.547736 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e89332-8af8-4e1e-8619-2c418103c6c1 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:39.547736 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 328/644] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:39.547736 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e89332-8af8-4e1e-8619-2c418103c6c1 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:39.622169 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:39.650840 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c54edb-ae32-4718-8b26-9f4733ee439f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:39.663603 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e89332-8af8-4e1e-8619-2c418103c6c1 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:39.689005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b37a0-c102-4e99-9d64-fa6ea6c90f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:39.689644 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b37a0-c102-4e99-9d64-fa6ea6c90f74 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:39.689929 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b37a0-c102-4e99-9d64-fa6ea6c90f74 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:39.690240 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b37a0-c102-4e99-9d64-fa6ea6c90f74 None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:39.691063 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b37a0-c102-4e99-9d64-fa6ea6c90f74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:39.691377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b37a0-c102-4e99-9d64-fa6ea6c90f74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:39.692823 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b37a0-c102-4e99-9d64-fa6ea6c90f74 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:39.701792 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83a7d9-c853-411a-b421-e6618b5c9384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:39.702800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83a7d9-c853-411a-b421-e6618b5c9384 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:39.703293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83a7d9-c853-411a-b421-e6618b5c9384 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:39.703749 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83a7d9-c853-411a-b421-e6618b5c9384 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:39.704654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83a7d9-c853-411a-b421-e6618b5c9384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:39.705170 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83a7d9-c853-411a-b421-e6618b5c9384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:39.705291 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 317/645] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:39.706081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b37a0-c102-4e99-9d64-fa6ea6c90f74 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:39.707169 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83a7d9-c853-411a-b421-e6618b5c9384 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:39.722827 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 329/646] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:39.724372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33b103-f80b-4450-a0fb-59917500d3bc None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:39.724877 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33b103-f80b-4450-a0fb-59917500d3bc None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:39.725354 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33b103-f80b-4450-a0fb-59917500d3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:39.743006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-8b33b103-f80b-4450-a0fb-59917500d3bc None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:39.744448 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-8b33b103-f80b-4450-a0fb-59917500d3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:39.774707 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:39.817033 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 330/647] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:39.817881 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33b103-f80b-4450-a0fb-59917500d3bc None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:39.828233 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b37a0-c102-4e99-9d64-fa6ea6c90f74 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:39.867471 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60acad0-0585-4f89-9841-4efcb8f7e7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:39.868399 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60acad0-0585-4f89-9841-4efcb8f7e7ad None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:39.868683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60acad0-0585-4f89-9841-4efcb8f7e7ad None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:39.868963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60acad0-0585-4f89-9841-4efcb8f7e7ad None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:39.872508 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:39.878163 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60acad0-0585-4f89-9841-4efcb8f7e7ad None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:39.879721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60acad0-0585-4f89-9841-4efcb8f7e7ad None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-a60acad0-0585-4f89-9841-4efcb8f7e7ad None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:39.895502 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:39.896572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:39.897657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:39.897657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:39.897657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:39.897657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:39.897657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:39.897657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:39.897657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:39.897657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:39.897657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:18:39.897657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:39.901490 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 318/648] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:39.902394 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60acad0-0585-4f89-9841-4efcb8f7e7ad None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:39.926573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33b103-f80b-4450-a0fb-59917500d3bc None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:39.963333 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c09a2-bde8-416e-a921-b8078c1eeb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:39.964320 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c09a2-bde8-416e-a921-b8078c1eeb20 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:39.964602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c09a2-bde8-416e-a921-b8078c1eeb20 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:39.964879 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c09a2-bde8-416e-a921-b8078c1eeb20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:39.965704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c09a2-bde8-416e-a921-b8078c1eeb20 None admin] RBAC: Adding query filter params (role.id=c9206a4a0d49473f87a45d6c8abb2e51, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=edb1bcb781084f03a4ad2611a09bd77e) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:39.966014 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c09a2-bde8-416e-a921-b8078c1eeb20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:39.967590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c09a2-bde8-416e-a921-b8078c1eeb20 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:39.982294 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 331/649] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:18:39 2022] GET /identity/v3/role_assignments?user.id=edb1bcb781084f03a4ad2611a09bd77e&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=c9206a4a0d49473f87a45d6c8abb2e51 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:39.982294 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c09a2-bde8-416e-a921-b8078c1eeb20 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:40.004609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:40.030452 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60acad0-0585-4f89-9841-4efcb8f7e7ad None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:40.068970 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4e7e3-60a0-4601-8618-7b178dc26e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:40.069990 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4e7e3-60a0-4601-8618-7b178dc26e4d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:40.070547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4e7e3-60a0-4601-8618-7b178dc26e4d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:40.070918 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4e7e3-60a0-4601-8618-7b178dc26e4d None admin] PATH_INFO: `/v3/users/glance` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:40.080274 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4e7e3-60a0-4601-8618-7b178dc26e4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:40.081935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4e7e3-60a0-4601-8618-7b178dc26e4d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:40.087928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c09a2-bde8-416e-a921-b8078c1eeb20 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-37c4e7e3-60a0-4601-8618-7b178dc26e4d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:40.098657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:40.099798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:40.100862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:40.100862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:40.100862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:40.100862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:40.100862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:40.100862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:40.100862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:40.100862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:40.100862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:18:40.100862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:40.100862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 319/650] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:40.101563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4e7e3-60a0-4601-8618-7b178dc26e4d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:40.126622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665199b1-aa6f-4554-b2b8-f08cd64d24e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:40.127360 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665199b1-aa6f-4554-b2b8-f08cd64d24e6 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:40.127643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665199b1-aa6f-4554-b2b8-f08cd64d24e6 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:40.127981 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665199b1-aa6f-4554-b2b8-f08cd64d24e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:40.140100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665199b1-aa6f-4554-b2b8-f08cd64d24e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-665199b1-aa6f-4554-b2b8-f08cd64d24e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:40.148284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:40.149609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:40.150596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:40.150596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:40.150596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:40.150596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:40.150596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:40.150596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:40.150596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:40.150596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:40.150596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:40.150596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:40.150596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:40.150596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:40.154382 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 332/651] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:40.155387 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665199b1-aa6f-4554-b2b8-f08cd64d24e6 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:40.173445 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:40.230373 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4e7e3-60a0-4601-8618-7b178dc26e4d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:40.265116 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:40.275268 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8662f9-8646-4c3c-aef5-7c9b57203048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:40.276332 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8662f9-8646-4c3c-aef5-7c9b57203048 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:40.276756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8662f9-8646-4c3c-aef5-7c9b57203048 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:40.277432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8662f9-8646-4c3c-aef5-7c9b57203048 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:40.278806 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8662f9-8646-4c3c-aef5-7c9b57203048 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:40.279157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8662f9-8646-4c3c-aef5-7c9b57203048 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:40.280775 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8662f9-8646-4c3c-aef5-7c9b57203048 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:40.285042 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665199b1-aa6f-4554-b2b8-f08cd64d24e6 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:40.303161 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 320/652] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 20:18:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:40.303926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8662f9-8646-4c3c-aef5-7c9b57203048 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:40.333200 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc4c4b-202b-41fa-89fc-9ec530d21a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:40.334140 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc4c4b-202b-41fa-89fc-9ec530d21a05 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:40.334458 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc4c4b-202b-41fa-89fc-9ec530d21a05 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:40.335806 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc4c4b-202b-41fa-89fc-9ec530d21a05 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:40.336305 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc4c4b-202b-41fa-89fc-9ec530d21a05 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:40.339399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc4c4b-202b-41fa-89fc-9ec530d21a05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:40.339399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc4c4b-202b-41fa-89fc-9ec530d21a05 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:40.362885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 333/653] 10.222.0.84 () {56 vars in 1137 bytes} [Thu May 19 20:18:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:40.364065 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc4c4b-202b-41fa-89fc-9ec530d21a05 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:40.422633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8662f9-8646-4c3c-aef5-7c9b57203048 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:40.439984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:40.465267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ba098-a577-41f8-ac08-c0d56c159152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:40.466281 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ba098-a577-41f8-ac08-c0d56c159152 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:40.466723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ba098-a577-41f8-ac08-c0d56c159152 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:40.467169 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ba098-a577-41f8-ac08-c0d56c159152 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:40.468253 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ba098-a577-41f8-ac08-c0d56c159152 None admin] RBAC: Adding query filter params (name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:40.468717 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ba098-a577-41f8-ac08-c0d56c159152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:40.470300 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ba098-a577-41f8-ac08-c0d56c159152 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:40.477078 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc4c4b-202b-41fa-89fc-9ec530d21a05 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:40.480738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 321/654] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:40.481811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ba098-a577-41f8-ac08-c0d56c159152 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:40.513349 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357a782-18ce-4e14-96a6-ed3a420517fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:40.514317 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357a782-18ce-4e14-96a6-ed3a420517fb None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:40.514624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357a782-18ce-4e14-96a6-ed3a420517fb None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:40.514906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357a782-18ce-4e14-96a6-ed3a420517fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:40.535029 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:40.608046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ba098-a577-41f8-ac08-c0d56c159152 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:40.608311 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357a782-18ce-4e14-96a6-ed3a420517fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:40.609552 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357a782-18ce-4e14-96a6-ed3a420517fb None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:40.645285 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a183d7c-2ca7-453a-affa-26e091ab598e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:40.646432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a183d7c-2ca7-453a-affa-26e091ab598e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:40.646937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a183d7c-2ca7-453a-affa-26e091ab598e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:40.647501 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a183d7c-2ca7-453a-affa-26e091ab598e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:40.707617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:40.731005 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 334/655] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:40.731939 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357a782-18ce-4e14-96a6-ed3a420517fb None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:40.739298 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a183d7c-2ca7-453a-affa-26e091ab598e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:40.740773 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a183d7c-2ca7-453a-affa-26e091ab598e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:40.780483 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:40.843562 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 322/656] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:40.845409 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357a782-18ce-4e14-96a6-ed3a420517fb None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:40.849375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6f39b-43e0-4f73-bb42-640580c5cde5 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:40.849662 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6f39b-43e0-4f73-bb42-640580c5cde5 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:40.849934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6f39b-43e0-4f73-bb42-640580c5cde5 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:40.851485 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 323/657] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:40.852086 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6f39b-43e0-4f73-bb42-640580c5cde5 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:40.889236 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee423f4b-2d88-4402-bca9-e8015cf5a194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:40.890039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee423f4b-2d88-4402-bca9-e8015cf5a194 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:40.890345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee423f4b-2d88-4402-bca9-e8015cf5a194 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:40.890621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee423f4b-2d88-4402-bca9-e8015cf5a194 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:40.901439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee423f4b-2d88-4402-bca9-e8015cf5a194 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:40.902920 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee423f4b-2d88-4402-bca9-e8015cf5a194 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:40.920800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-ee423f4b-2d88-4402-bca9-e8015cf5a194 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:40.920800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:40.920800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:40.920800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:40.920800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:40.920800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:40.920800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:40.920800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:40.920800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:40.920800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:40.920800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:40.920800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:40.920800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:40.920800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:40.920800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:40.921831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:40.924313 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:40.924313 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:40.924313 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:40.924313 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:40.924313 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 335/658] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:40.924313 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee423f4b-2d88-4402-bca9-e8015cf5a194 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:40.938222 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:40.964786 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6f39b-43e0-4f73-bb42-640580c5cde5 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:41.000009 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733bf9c-ccd1-42bf-8dfa-5f6adb4bbb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:41.001225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733bf9c-ccd1-42bf-8dfa-5f6adb4bbb7a None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:41.001509 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733bf9c-ccd1-42bf-8dfa-5f6adb4bbb7a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:41.001786 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733bf9c-ccd1-42bf-8dfa-5f6adb4bbb7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:41.013690 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733bf9c-ccd1-42bf-8dfa-5f6adb4bbb7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:41.015203 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733bf9c-ccd1-42bf-8dfa-5f6adb4bbb7a None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:41.025409 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:41.032542 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee423f4b-2d88-4402-bca9-e8015cf5a194 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:41.033800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-4733bf9c-ccd1-42bf-8dfa-5f6adb4bbb7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:41.033800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:41.033800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:41.033800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:41.033800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:41.033800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:41.033800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:41.033800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:41.033800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:41.033800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:41.033800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:41.033800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:41.033800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:41.033800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:41.033800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:41.035238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:41.036215 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:41.036215 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:41.036215 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:41.036215 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:41.036215 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 324/659] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:41.037720 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823809cf-82a7-4a70-93d3-043203f74740 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:41.037720 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823809cf-82a7-4a70-93d3-043203f74740 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:41.037720 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823809cf-82a7-4a70-93d3-043203f74740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:41.054254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-823809cf-82a7-4a70-93d3-043203f74740 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:41.055500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-823809cf-82a7-4a70-93d3-043203f74740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:41.074701 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31049190-a4fc-40c3-9704-32da0916ac80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:41.075508 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31049190-a4fc-40c3-9704-32da0916ac80 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:41.075791 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31049190-a4fc-40c3-9704-32da0916ac80 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:41.076058 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31049190-a4fc-40c3-9704-32da0916ac80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:41.086879 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31049190-a4fc-40c3-9704-32da0916ac80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:41.088337 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31049190-a4fc-40c3-9704-32da0916ac80 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:41.107978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-31049190-a4fc-40c3-9704-32da0916ac80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:41.107978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:41.107978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:41.107978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:41.107978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:41.107978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:41.107978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:41.107978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:41.107978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:41.107978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:41.107978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:41.107978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:41.107978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:41.107978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:41.107978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:41.114414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:41.115458 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:41.115458 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:41.115458 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:41.115458 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:41.115458 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 336/660] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:41.115458 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31049190-a4fc-40c3-9704-32da0916ac80 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:41.131151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 325/661] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:41.143138 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823809cf-82a7-4a70-93d3-043203f74740 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:41.208310 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:41.219255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31049190-a4fc-40c3-9704-32da0916ac80 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:41.254711 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823809cf-82a7-4a70-93d3-043203f74740 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:41.257979 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d2b65-c811-4bc4-adc1-d203134e60e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:41.258688 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d2b65-c811-4bc4-adc1-d203134e60e4 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:41.258966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d2b65-c811-4bc4-adc1-d203134e60e4 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:41.259239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d2b65-c811-4bc4-adc1-d203134e60e4 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:41.259961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d2b65-c811-4bc4-adc1-d203134e60e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:41.260267 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d2b65-c811-4bc4-adc1-d203134e60e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:41.261666 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d2b65-c811-4bc4-adc1-d203134e60e4 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:41.278351 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 337/662] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:41.278351 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d2b65-c811-4bc4-adc1-d203134e60e4 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:41.291106 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:41.295215 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb16795-a45f-4699-9786-41ccf4391d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:41.295891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb16795-a45f-4699-9786-41ccf4391d01 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:41.296222 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb16795-a45f-4699-9786-41ccf4391d01 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:41.296427 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb16795-a45f-4699-9786-41ccf4391d01 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:41.297354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb16795-a45f-4699-9786-41ccf4391d01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:41.297470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb16795-a45f-4699-9786-41ccf4391d01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:41.298847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb16795-a45f-4699-9786-41ccf4391d01 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:41.314845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 326/663] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:41.314845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb16795-a45f-4699-9786-41ccf4391d01 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:41.393134 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d2b65-c811-4bc4-adc1-d203134e60e4 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:41.430172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a2c63-7ca4-4ad0-8549-817162280a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:41.430925 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a2c63-7ca4-4ad0-8549-817162280a47 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:41.431265 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a2c63-7ca4-4ad0-8549-817162280a47 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:41.431542 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a2c63-7ca4-4ad0-8549-817162280a47 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:41.432268 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a2c63-7ca4-4ad0-8549-817162280a47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:41.432571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a2c63-7ca4-4ad0-8549-817162280a47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:41.433914 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb16795-a45f-4699-9786-41ccf4391d01 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:41.434050 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a2c63-7ca4-4ad0-8549-817162280a47 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:41.452890 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 338/664] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:41.452890 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a2c63-7ca4-4ad0-8549-817162280a47 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:41.458495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:41.480419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f9984-dcae-4778-8eeb-13945e50f8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:41.481242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f9984-dcae-4778-8eeb-13945e50f8b9 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:41.481531 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f9984-dcae-4778-8eeb-13945e50f8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:41.481809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f9984-dcae-4778-8eeb-13945e50f8b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:41.492650 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f9984-dcae-4778-8eeb-13945e50f8b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-fe3f9984-dcae-4778-8eeb-13945e50f8b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:41.500755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:41.501798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:41.502777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:41.502777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:41.502777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:41.502777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:41.502777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:41.502777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:41.502777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:41.502777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:41.502777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:41.502777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:41.502777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:41.502777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:41.502777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 327/665] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:41.502777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f9984-dcae-4778-8eeb-13945e50f8b9 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:41.526613 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:41.565647 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a2c63-7ca4-4ad0-8549-817162280a47 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:41.601025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79903a-a45f-4bc5-b210-d20bfb46da76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:41.602286 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79903a-a45f-4bc5-b210-d20bfb46da76 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:41.602582 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79903a-a45f-4bc5-b210-d20bfb46da76 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:41.602864 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79903a-a45f-4bc5-b210-d20bfb46da76 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:41.610206 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79903a-a45f-4bc5-b210-d20bfb46da76 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:41.610572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f9984-dcae-4778-8eeb-13945e50f8b9 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:41.611694 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79903a-a45f-4bc5-b210-d20bfb46da76 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-da79903a-a45f-4bc5-b210-d20bfb46da76 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:41.628621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:41.629674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:41.631373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:41.631373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:41.631373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:41.631373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:41.631373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:41.631373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:41.631373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:41.631373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:41.631373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:18:41.631373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:41.631373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 339/666] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:41.631373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79903a-a45f-4bc5-b210-d20bfb46da76 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:41.650068 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ec2aa-1220-4aa2-881f-df85a36f81aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:41.650932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ec2aa-1220-4aa2-881f-df85a36f81aa None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:41.651214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ec2aa-1220-4aa2-881f-df85a36f81aa None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:41.651631 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ec2aa-1220-4aa2-881f-df85a36f81aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:41.658887 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ec2aa-1220-4aa2-881f-df85a36f81aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:41.660339 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ec2aa-1220-4aa2-881f-df85a36f81aa None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:41.675284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-965ec2aa-1220-4aa2-881f-df85a36f81aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:41.675284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:41.675284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:41.675284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:41.675284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:41.675284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:41.675284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:41.675284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:41.675284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:41.675284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:41.675284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:41.675284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:41.675284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:41.675284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:41.675284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:41.676488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:41.677739 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:41.677739 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:41.677739 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:41.677739 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:41.677739 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:41.677739 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:41.677739 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:41.677739 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:41.677739 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 328/667] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:41.677739 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ec2aa-1220-4aa2-881f-df85a36f81aa None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:41.717247 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:41.749985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79903a-a45f-4bc5-b210-d20bfb46da76 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:41.778193 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:41.779825 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ec2aa-1220-4aa2-881f-df85a36f81aa None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:41.786767 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13268590-fcdd-47aa-946a-d082ff1f5ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:41.787578 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13268590-fcdd-47aa-946a-d082ff1f5ac1 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:41.787863 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13268590-fcdd-47aa-946a-d082ff1f5ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:41.788138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13268590-fcdd-47aa-946a-d082ff1f5ac1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:41.796292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13268590-fcdd-47aa-946a-d082ff1f5ac1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:41.797762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13268590-fcdd-47aa-946a-d082ff1f5ac1 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:41.811654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-13268590-fcdd-47aa-946a-d082ff1f5ac1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:41.811654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:41.811654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:41.811654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:41.811654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:41.811654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:41.811654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:41.811654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:41.811654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:41.811654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:41.811654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:41.811654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:41.811654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:41.811654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:41.811654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:41.812683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:41.813670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:41.813670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:41.813670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:41.813670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:41.813670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:41.813670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:41.813670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:41.813670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:41.813670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 340/668] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:41.813670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13268590-fcdd-47aa-946a-d082ff1f5ac1 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:41.820773 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635b7aa-e58c-4980-8ca3-d0ce69f27b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:41.821781 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635b7aa-e58c-4980-8ca3-d0ce69f27b10 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:41.822064 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635b7aa-e58c-4980-8ca3-d0ce69f27b10 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:41.822372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635b7aa-e58c-4980-8ca3-d0ce69f27b10 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:41.823126 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635b7aa-e58c-4980-8ca3-d0ce69f27b10 None admin] RBAC: Adding query filter params (name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:41.823432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635b7aa-e58c-4980-8ca3-d0ce69f27b10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:41.824794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635b7aa-e58c-4980-8ca3-d0ce69f27b10 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:41.837780 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 329/669] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:41.837780 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635b7aa-e58c-4980-8ca3-d0ce69f27b10 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:41.933192 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13268590-fcdd-47aa-946a-d082ff1f5ac1 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:41.941666 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:41.944906 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635b7aa-e58c-4980-8ca3-d0ce69f27b10 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:41.971290 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86226b09-2480-40bc-8ce8-0fc05deeef3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:41.971970 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86226b09-2480-40bc-8ce8-0fc05deeef3e None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:41.972253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86226b09-2480-40bc-8ce8-0fc05deeef3e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:41.972513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86226b09-2480-40bc-8ce8-0fc05deeef3e None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:41.973345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86226b09-2480-40bc-8ce8-0fc05deeef3e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:41.973652 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86226b09-2480-40bc-8ce8-0fc05deeef3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:41.975165 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86226b09-2480-40bc-8ce8-0fc05deeef3e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:41.981236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c47d37-eef3-4c3c-9672-a7d17679475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:41.982162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c47d37-eef3-4c3c-9672-a7d17679475c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:41.982639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c47d37-eef3-4c3c-9672-a7d17679475c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:41.983065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c47d37-eef3-4c3c-9672-a7d17679475c None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:41.984002 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c47d37-eef3-4c3c-9672-a7d17679475c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:41.984462 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c47d37-eef3-4c3c-9672-a7d17679475c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:41.986082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c47d37-eef3-4c3c-9672-a7d17679475c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:41.994466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 341/670] 10.222.0.84 () {56 vars in 1139 bytes} [Thu May 19 20:18:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:41.995611 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86226b09-2480-40bc-8ce8-0fc05deeef3e None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:42.008073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 330/671] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:42.008073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c47d37-eef3-4c3c-9672-a7d17679475c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:42.048461 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:42.106477 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86226b09-2480-40bc-8ce8-0fc05deeef3e None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:42.113043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c47d37-eef3-4c3c-9672-a7d17679475c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:42.142089 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87648e6-e987-4839-b6ba-4139ab39d443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:42.142804 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87648e6-e987-4839-b6ba-4139ab39d443 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:42.143094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87648e6-e987-4839-b6ba-4139ab39d443 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:42.143378 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87648e6-e987-4839-b6ba-4139ab39d443 None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:42.144175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87648e6-e987-4839-b6ba-4139ab39d443 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:42.144488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87648e6-e987-4839-b6ba-4139ab39d443 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:42.145943 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87648e6-e987-4839-b6ba-4139ab39d443 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:42.151402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb68b71-c4da-4441-8c2d-ebb6105c1a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:42.152209 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb68b71-c4da-4441-8c2d-ebb6105c1a26 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:42.152492 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb68b71-c4da-4441-8c2d-ebb6105c1a26 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:42.152768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb68b71-c4da-4441-8c2d-ebb6105c1a26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:42.163544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 342/672] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:42.163544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87648e6-e987-4839-b6ba-4139ab39d443 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:42.168642 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb68b71-c4da-4441-8c2d-ebb6105c1a26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:42.170172 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb68b71-c4da-4441-8c2d-ebb6105c1a26 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:42.188292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-3eb68b71-c4da-4441-8c2d-ebb6105c1a26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:42.188292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:42.188292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:42.188292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:42.188292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:42.188292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:42.188292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:42.188292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:42.188292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:42.188292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:42.188292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:42.188292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:42.188292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:42.188292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:42.188292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:42.189347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:42.190384 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:42.190384 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:42.190384 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:42.190384 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:42.190384 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 331/673] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:42.190709 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb68b71-c4da-4441-8c2d-ebb6105c1a26 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:42.210371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:42.281614 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87648e6-e987-4839-b6ba-4139ab39d443 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:42.290689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:42.305777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb68b71-c4da-4441-8c2d-ebb6105c1a26 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:42.325444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a24f71-9d41-492a-b0e0-e2149785d145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:42.326127 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a24f71-9d41-492a-b0e0-e2149785d145 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:42.326449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a24f71-9d41-492a-b0e0-e2149785d145 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:42.326732 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a24f71-9d41-492a-b0e0-e2149785d145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:42.341226 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a68cf8-4c2d-433b-98fd-1864cf2e3b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:42.341987 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a68cf8-4c2d-433b-98fd-1864cf2e3b37 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:42.342290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a68cf8-4c2d-433b-98fd-1864cf2e3b37 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:42.342585 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a68cf8-4c2d-433b-98fd-1864cf2e3b37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:42.343414 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a68cf8-4c2d-433b-98fd-1864cf2e3b37 None admin] RBAC: Adding query filter params (role.id=c9206a4a0d49473f87a45d6c8abb2e51, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=3681f07e23694baba690033a991a23c6) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:42.343727 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a68cf8-4c2d-433b-98fd-1864cf2e3b37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:42.345167 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a68cf8-4c2d-433b-98fd-1864cf2e3b37 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:42.358995 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 332/674] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:18:42 2022] GET /identity/v3/role_assignments?user.id=3681f07e23694baba690033a991a23c6&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=c9206a4a0d49473f87a45d6c8abb2e51 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:42.360617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a68cf8-4c2d-433b-98fd-1864cf2e3b37 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:42.430690 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a24f71-9d41-492a-b0e0-e2149785d145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:42.432231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a24f71-9d41-492a-b0e0-e2149785d145 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:42.478895 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:42.506881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a68cf8-4c2d-433b-98fd-1864cf2e3b37 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:42.544569 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b8f7f-dff9-45f4-a7e4-217f28201698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:42.545409 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b8f7f-dff9-45f4-a7e4-217f28201698 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:42.545781 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b8f7f-dff9-45f4-a7e4-217f28201698 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:42.546127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b8f7f-dff9-45f4-a7e4-217f28201698 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:42.547003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b8f7f-dff9-45f4-a7e4-217f28201698 None admin] RBAC: Adding query filter params (role.id=c9206a4a0d49473f87a45d6c8abb2e51, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=319fb9f41afb48e68c34bdb1f4ba114b) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:42.547319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b8f7f-dff9-45f4-a7e4-217f28201698 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:42.548771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b8f7f-dff9-45f4-a7e4-217f28201698 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:42.555311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 333/675] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:18:42 2022] GET /identity/v3/role_assignments?user.id=319fb9f41afb48e68c34bdb1f4ba114b&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=c9206a4a0d49473f87a45d6c8abb2e51 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:42.556298 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b8f7f-dff9-45f4-a7e4-217f28201698 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:42.559040 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 343/676] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:42.560843 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77d4ee-512f-4caf-89dd-13e3c19034d4 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:42.561137 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77d4ee-512f-4caf-89dd-13e3c19034d4 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:42.561477 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77d4ee-512f-4caf-89dd-13e3c19034d4 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:42.563187 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 344/677] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:42.565866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77d4ee-512f-4caf-89dd-13e3c19034d4 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:42.620974 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:42.699456 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b8f7f-dff9-45f4-a7e4-217f28201698 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:42.708355 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77d4ee-512f-4caf-89dd-13e3c19034d4 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:42.741266 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2e9e6-29b5-4685-9c8b-aae872b4eb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:42.741952 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2e9e6-29b5-4685-9c8b-aae872b4eb71 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:42.742232 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2e9e6-29b5-4685-9c8b-aae872b4eb71 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:42.742547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2e9e6-29b5-4685-9c8b-aae872b4eb71 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:42.743288 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf2e9e6-29b5-4685-9c8b-aae872b4eb71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:42.743590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf2e9e6-29b5-4685-9c8b-aae872b4eb71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:42.744986 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf2e9e6-29b5-4685-9c8b-aae872b4eb71 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:42.759699 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 334/678] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:42.759699 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:42.765941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03452f3-99a8-4d4a-aa96-8b23bfcbc027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:42.765941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03452f3-99a8-4d4a-aa96-8b23bfcbc027 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:42.765941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03452f3-99a8-4d4a-aa96-8b23bfcbc027 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:42.765941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03452f3-99a8-4d4a-aa96-8b23bfcbc027 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:42.766401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707f85e-6920-48a3-be63-26f847b27b68 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:42.766401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707f85e-6920-48a3-be63-26f847b27b68 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:42.766401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707f85e-6920-48a3-be63-26f847b27b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:42.780858 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03452f3-99a8-4d4a-aa96-8b23bfcbc027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:42.782502 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03452f3-99a8-4d4a-aa96-8b23bfcbc027 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:42.788114 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-c707f85e-6920-48a3-be63-26f847b27b68 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:42.788855 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-c707f85e-6920-48a3-be63-26f847b27b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:42.805754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-f03452f3-99a8-4d4a-aa96-8b23bfcbc027 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:42.805754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:42.805754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:42.805754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:42.805754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:42.805754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:42.805754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:42.805754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:42.805754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:42.805754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:42.805754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:42.805754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:42.805754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:42.805754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:42.805754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:42.806941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:42.807970 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:42.807970 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:42.807970 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:42.807970 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:42.808728 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 345/679] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:42.809617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03452f3-99a8-4d4a-aa96-8b23bfcbc027 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:42.884480 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:42.887820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 335/680] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:42.888794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707f85e-6920-48a3-be63-26f847b27b68 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:42.947513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03452f3-99a8-4d4a-aa96-8b23bfcbc027 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:43.000681 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7345ba1-d41b-4c37-ac80-4ff6f1a76cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:43.001503 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7345ba1-d41b-4c37-ac80-4ff6f1a76cad None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:43.001785 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7345ba1-d41b-4c37-ac80-4ff6f1a76cad None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:43.002061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7345ba1-d41b-4c37-ac80-4ff6f1a76cad None admin] PATH_INFO: `/v3/users/swift` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:43.010882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7345ba1-d41b-4c37-ac80-4ff6f1a76cad None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:43.012105 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707f85e-6920-48a3-be63-26f847b27b68 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:43.012377 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7345ba1-d41b-4c37-ac80-4ff6f1a76cad None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-a7345ba1-d41b-4c37-ac80-4ff6f1a76cad None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:43.029039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:43.030224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:43.031376 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:43.031376 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:43.031376 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:43.031376 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:43.031376 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:43.031376 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:43.031376 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:43.031376 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:43.031376 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:18:43.031376 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:43.031376 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 346/681] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:18:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:43.032069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7345ba1-d41b-4c37-ac80-4ff6f1a76cad None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:43.042830 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:43.049590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da724638-4d3a-4d36-bc62-e18afb9ba9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:43.050288 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da724638-4d3a-4d36-bc62-e18afb9ba9b3 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:43.050581 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da724638-4d3a-4d36-bc62-e18afb9ba9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:43.050857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da724638-4d3a-4d36-bc62-e18afb9ba9b3 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:43.051581 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da724638-4d3a-4d36-bc62-e18afb9ba9b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:43.052140 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da724638-4d3a-4d36-bc62-e18afb9ba9b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:43.053282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da724638-4d3a-4d36-bc62-e18afb9ba9b3 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:43.070579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 336/682] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:43.071938 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da724638-4d3a-4d36-bc62-e18afb9ba9b3 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:43.153734 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7345ba1-d41b-4c37-ac80-4ff6f1a76cad None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:43.181448 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:43.184074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da724638-4d3a-4d36-bc62-e18afb9ba9b3 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:43.193942 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79787762-4a52-4c5d-a82d-d629d796cc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:43.195192 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79787762-4a52-4c5d-a82d-d629d796cc15 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:43.195741 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79787762-4a52-4c5d-a82d-d629d796cc15 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:43.196416 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79787762-4a52-4c5d-a82d-d629d796cc15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:43.208845 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79787762-4a52-4c5d-a82d-d629d796cc15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-79787762-4a52-4c5d-a82d-d629d796cc15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:43.218615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:43.219928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:43.221144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:43.221144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:43.221144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:43.221144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:43.221144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:43.221144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:43.221144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:43.221144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:43.221144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:43.221144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:43.221144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:43.221144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:43.222803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fde5a3-fd2e-4ee9-8775-bd2d3436d3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:43.223576 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fde5a3-fd2e-4ee9-8775-bd2d3436d3bc None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:43.223908 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fde5a3-fd2e-4ee9-8775-bd2d3436d3bc None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:43.224267 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 347/683] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:18:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:43.225487 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79787762-4a52-4c5d-a82d-d629d796cc15 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:43.225640 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fde5a3-fd2e-4ee9-8775-bd2d3436d3bc None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:43.226354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fde5a3-fd2e-4ee9-8775-bd2d3436d3bc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:43.230393 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fde5a3-fd2e-4ee9-8775-bd2d3436d3bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:43.231970 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fde5a3-fd2e-4ee9-8775-bd2d3436d3bc None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:43.251243 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 337/684] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 20:18:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:43.252283 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fde5a3-fd2e-4ee9-8775-bd2d3436d3bc None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:43.305122 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:43.341551 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79787762-4a52-4c5d-a82d-d629d796cc15 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:43.358121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fde5a3-fd2e-4ee9-8775-bd2d3436d3bc None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:43.378181 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44dc56a-9dbb-4c24-9a57-4b0a93bbc103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:43.379278 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dc56a-9dbb-4c24-9a57-4b0a93bbc103 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:43.379748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dc56a-9dbb-4c24-9a57-4b0a93bbc103 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:43.380189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dc56a-9dbb-4c24-9a57-4b0a93bbc103 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:43.388207 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44dc56a-9dbb-4c24-9a57-4b0a93bbc103 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:43.390078 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44dc56a-9dbb-4c24-9a57-4b0a93bbc103 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:43.392540 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55cb19-3aef-4678-a035-b4b4bfce55ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:43.393269 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55cb19-3aef-4678-a035-b4b4bfce55ce None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:43.393489 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55cb19-3aef-4678-a035-b4b4bfce55ce None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:43.393761 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55cb19-3aef-4678-a035-b4b4bfce55ce None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:43.394603 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55cb19-3aef-4678-a035-b4b4bfce55ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:43.394907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55cb19-3aef-4678-a035-b4b4bfce55ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:43.396256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55cb19-3aef-4678-a035-b4b4bfce55ce None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:43.409573 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 338/685] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:43.410560 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55cb19-3aef-4678-a035-b4b4bfce55ce None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:43.415764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-b44dc56a-9dbb-4c24-9a57-4b0a93bbc103 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:43.415764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:43.415764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:43.415764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:43.415764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:43.415764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:43.415764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:43.415764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:43.415764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:43.415764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:43.415764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:43.415764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:43.415764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:43.415764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:43.415764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:43.416915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:43.417931 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:43.417931 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:43.417931 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:43.417931 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:43.417931 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:43.417931 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:43.417931 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:43.417931 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:43.420442 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 348/686] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:43.421444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44dc56a-9dbb-4c24-9a57-4b0a93bbc103 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:43.469049 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:43.527311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55cb19-3aef-4678-a035-b4b4bfce55ce None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:43.538176 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:43.540741 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44dc56a-9dbb-4c24-9a57-4b0a93bbc103 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:43.564247 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805a091-91fc-4c24-a9f1-b09d093fadec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:43.564936 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805a091-91fc-4c24-a9f1-b09d093fadec None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:43.565216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805a091-91fc-4c24-a9f1-b09d093fadec None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:43.565499 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805a091-91fc-4c24-a9f1-b09d093fadec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:43.582142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19ff9d-67b7-4d58-b08b-9ac8d44668bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:43.583439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19ff9d-67b7-4d58-b08b-9ac8d44668bc None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:43.583901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19ff9d-67b7-4d58-b08b-9ac8d44668bc None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:43.584356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19ff9d-67b7-4d58-b08b-9ac8d44668bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:43.598392 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19ff9d-67b7-4d58-b08b-9ac8d44668bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:43.600234 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19ff9d-67b7-4d58-b08b-9ac8d44668bc None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:43.620908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-6e19ff9d-67b7-4d58-b08b-9ac8d44668bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:43.620908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:43.620908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:43.620908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:43.620908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:43.620908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:43.620908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:43.620908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:43.620908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:43.620908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:43.620908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:43.620908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:43.620908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:43.620908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:43.620908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:43.622188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:43.623442 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:43.623442 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:43.623442 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:43.623442 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:43.625456 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 349/687] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:43.626431 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19ff9d-67b7-4d58-b08b-9ac8d44668bc None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:43.644489 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805a091-91fc-4c24-a9f1-b09d093fadec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:43.645986 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805a091-91fc-4c24-a9f1-b09d093fadec None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:43.728703 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:43.739873 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19ff9d-67b7-4d58-b08b-9ac8d44668bc None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:43.761535 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 339/688] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:43.762766 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805a091-91fc-4c24-a9f1-b09d093fadec None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:43.775494 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:43.781684 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217fc05-4072-448c-8919-b01042853a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:43.782395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217fc05-4072-448c-8919-b01042853a6f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:43.782679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217fc05-4072-448c-8919-b01042853a6f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:43.782954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217fc05-4072-448c-8919-b01042853a6f None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:43.783750 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217fc05-4072-448c-8919-b01042853a6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:43.784062 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217fc05-4072-448c-8919-b01042853a6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:43.785529 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217fc05-4072-448c-8919-b01042853a6f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:43.798420 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 350/689] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:43.799482 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217fc05-4072-448c-8919-b01042853a6f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:43.900419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805a091-91fc-4c24-a9f1-b09d093fadec None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:43.943050 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f004377-b414-4af7-947e-c69bf0fa296e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:43.944179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f004377-b414-4af7-947e-c69bf0fa296e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:43.944675 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f004377-b414-4af7-947e-c69bf0fa296e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:43.945215 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f004377-b414-4af7-947e-c69bf0fa296e None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:43.946859 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f004377-b414-4af7-947e-c69bf0fa296e None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:43.947412 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f004377-b414-4af7-947e-c69bf0fa296e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:43.949084 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f004377-b414-4af7-947e-c69bf0fa296e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:43.964287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 340/690] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:43.965536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f004377-b414-4af7-947e-c69bf0fa296e None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:44.032369 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217fc05-4072-448c-8919-b01042853a6f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:44.055036 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:44.068855 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861bac5-24cb-4714-9369-4cf002b3f711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:44.069674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861bac5-24cb-4714-9369-4cf002b3f711 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:44.069946 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861bac5-24cb-4714-9369-4cf002b3f711 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:44.070293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861bac5-24cb-4714-9369-4cf002b3f711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:44.081747 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861bac5-24cb-4714-9369-4cf002b3f711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:44.083480 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861bac5-24cb-4714-9369-4cf002b3f711 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:44.099668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f004377-b414-4af7-947e-c69bf0fa296e None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:44.102592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-f861bac5-24cb-4714-9369-4cf002b3f711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:44.102592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:44.102592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:44.102592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:44.102592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:44.102592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:44.102592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:44.102592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:44.102592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:44.102592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:44.102592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:44.102592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:44.102592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:44.102592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:44.102592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:44.103574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:44.104575 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:44.104575 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:44.104575 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:44.104575 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:44.108482 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 351/691] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:44.109364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861bac5-24cb-4714-9369-4cf002b3f711 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:44.112229 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:44.144146 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6fac8-1a1d-4ed4-ba7b-bdd780552ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:44.144820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6fac8-1a1d-4ed4-ba7b-bdd780552ad9 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:44.145101 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6fac8-1a1d-4ed4-ba7b-bdd780552ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:44.145427 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6fac8-1a1d-4ed4-ba7b-bdd780552ad9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:44.146406 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e6fac8-1a1d-4ed4-ba7b-bdd780552ad9 None admin] RBAC: Adding query filter params (role.id=c9206a4a0d49473f87a45d6c8abb2e51, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=3d63f8a58ef24de897ab7fd0828ec467) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:44.146719 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e6fac8-1a1d-4ed4-ba7b-bdd780552ad9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:44.148236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e6fac8-1a1d-4ed4-ba7b-bdd780552ad9 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:44.155426 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 341/692] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:18:43 2022] GET /identity/v3/role_assignments?user.id=3d63f8a58ef24de897ab7fd0828ec467&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=c9206a4a0d49473f87a45d6c8abb2e51 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:44.156322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6fac8-1a1d-4ed4-ba7b-bdd780552ad9 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:44.233655 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861bac5-24cb-4714-9369-4cf002b3f711 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:44.269380 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6fac8-1a1d-4ed4-ba7b-bdd780552ad9 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:44.275555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192583b4-edcf-4153-ab46-4f337d32b1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:44.276755 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192583b4-edcf-4153-ab46-4f337d32b1fe None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:44.277334 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192583b4-edcf-4153-ab46-4f337d32b1fe None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:44.277871 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192583b4-edcf-4153-ab46-4f337d32b1fe None admin] PATH_INFO: `/v3/users/nova` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:44.286655 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192583b4-edcf-4153-ab46-4f337d32b1fe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:44.288761 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192583b4-edcf-4153-ab46-4f337d32b1fe None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-192583b4-edcf-4153-ab46-4f337d32b1fe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:44.307837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:44.308991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:44.310056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:44.310056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:44.310056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:44.310056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:44.310056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:44.310056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:44.310056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:44.310056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:44.310056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:18:44.310056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:44.310690 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd550c-b783-488c-9101-a2406db578fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:44.310690 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd550c-b783-488c-9101-a2406db578fa None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:44.310690 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd550c-b783-488c-9101-a2406db578fa None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:44.310690 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd550c-b783-488c-9101-a2406db578fa None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:44.310978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd550c-b783-488c-9101-a2406db578fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:44.310978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd550c-b783-488c-9101-a2406db578fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:44.311808 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd550c-b783-488c-9101-a2406db578fa None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:44.313357 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:44.314067 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 352/693] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:18:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:44.322875 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192583b4-edcf-4153-ab46-4f337d32b1fe None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:44.330314 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 342/694] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:44.332345 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd550c-b783-488c-9101-a2406db578fa None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:44.441105 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:44.443303 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd550c-b783-488c-9101-a2406db578fa None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:44.484106 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5a361-5a7a-4e08-8b55-320fb4425e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:44.485174 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5a361-5a7a-4e08-8b55-320fb4425e76 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:44.485610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5a361-5a7a-4e08-8b55-320fb4425e76 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:44.486134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5a361-5a7a-4e08-8b55-320fb4425e76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:44.496051 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5a361-5a7a-4e08-8b55-320fb4425e76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:44.497921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5a361-5a7a-4e08-8b55-320fb4425e76 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:44.511265 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192583b4-edcf-4153-ab46-4f337d32b1fe None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:44.526679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-55e5a361-5a7a-4e08-8b55-320fb4425e76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:44.526679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:44.526679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:44.526679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:44.526679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:44.526679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:44.526679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:44.526679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:44.526679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:44.526679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:44.526679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:44.526679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:44.526679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:44.526679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:44.526679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:44.528033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:44.529247 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:44.529247 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:44.529247 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:44.529247 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:44.529247 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:44.529247 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:44.529247 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:44.529247 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:44.529247 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 343/695] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:44.535735 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5a361-5a7a-4e08-8b55-320fb4425e76 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:44.571253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0625b06-dfee-4526-8d5d-db6f7139c8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:44.572108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0625b06-dfee-4526-8d5d-db6f7139c8c1 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:44.572485 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0625b06-dfee-4526-8d5d-db6f7139c8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:44.572853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0625b06-dfee-4526-8d5d-db6f7139c8c1 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:44.573796 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0625b06-dfee-4526-8d5d-db6f7139c8c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:44.574198 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0625b06-dfee-4526-8d5d-db6f7139c8c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:44.575782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0625b06-dfee-4526-8d5d-db6f7139c8c1 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:44.599041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 353/696] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:18:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:44.601205 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0625b06-dfee-4526-8d5d-db6f7139c8c1 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:44.642245 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:44.704227 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5a361-5a7a-4e08-8b55-320fb4425e76 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:44.742462 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cdb41-6767-4dc4-9fed-0ca6a4e256d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:44.743152 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cdb41-6767-4dc4-9fed-0ca6a4e256d8 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:44.743433 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cdb41-6767-4dc4-9fed-0ca6a4e256d8 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:44.743709 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cdb41-6767-4dc4-9fed-0ca6a4e256d8 None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:44.744501 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cdb41-6767-4dc4-9fed-0ca6a4e256d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:44.744814 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cdb41-6767-4dc4-9fed-0ca6a4e256d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:44.746273 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cdb41-6767-4dc4-9fed-0ca6a4e256d8 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:44.752606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0625b06-dfee-4526-8d5d-db6f7139c8c1 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:44.760451 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 344/697] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:44.766394 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cdb41-6767-4dc4-9fed-0ca6a4e256d8 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:44.778077 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:44.800670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad4787-0747-483b-9900-aec796095761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:44.801333 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad4787-0747-483b-9900-aec796095761 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:44.801617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad4787-0747-483b-9900-aec796095761 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:44.801892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad4787-0747-483b-9900-aec796095761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:44.885991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad4787-0747-483b-9900-aec796095761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:44.888041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad4787-0747-483b-9900-aec796095761 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:44.924192 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cdb41-6767-4dc4-9fed-0ca6a4e256d8 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:44.940459 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:44.964930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53baa617-02ce-48ec-b8c1-5569f228e473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:44.965521 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53baa617-02ce-48ec-b8c1-5569f228e473 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:44.965805 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53baa617-02ce-48ec-b8c1-5569f228e473 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:44.966093 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53baa617-02ce-48ec-b8c1-5569f228e473 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:44.966989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53baa617-02ce-48ec-b8c1-5569f228e473 None admin] RBAC: Adding query filter params (role.id=c9206a4a0d49473f87a45d6c8abb2e51, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=54395ae480884de38fc811a9e82c1f9e) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:44.967362 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53baa617-02ce-48ec-b8c1-5569f228e473 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:44.968877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53baa617-02ce-48ec-b8c1-5569f228e473 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:44.975481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 345/698] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:18:44 2022] GET /identity/v3/role_assignments?user.id=54395ae480884de38fc811a9e82c1f9e&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=c9206a4a0d49473f87a45d6c8abb2e51 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:44.976937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70887b81-9b7b-49e1-a1b3-442f5a3b635e None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:44.977394 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70887b81-9b7b-49e1-a1b3-442f5a3b635e None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:44.977823 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70887b81-9b7b-49e1-a1b3-442f5a3b635e None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:44.984097 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 346/699] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:44.985679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de958455-2881-4839-bd49-11c2bfd28032 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:44.985974 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de958455-2881-4839-bd49-11c2bfd28032 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:44.986666 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de958455-2881-4839-bd49-11c2bfd28032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:45.005807 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-de958455-2881-4839-bd49-11c2bfd28032 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:45.007115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-de958455-2881-4839-bd49-11c2bfd28032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:45.020327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 354/700] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:18:45.023879 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad4787-0747-483b-9900-aec796095761 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:45.095649 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:45.101781 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 347/701] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:45.103399 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317172b4-c86c-43bd-800c-92b605d0f858 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:45.103688 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317172b4-c86c-43bd-800c-92b605d0f858 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:45.103967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317172b4-c86c-43bd-800c-92b605d0f858 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:45.114753 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 348/702] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:45 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:45.132331 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5fda7-b461-48ba-bb1e-46187054b9d6 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:45.132711 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5fda7-b461-48ba-bb1e-46187054b9d6 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:45.132907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5fda7-b461-48ba-bb1e-46187054b9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:45.161302 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad4787-0747-483b-9900-aec796095761 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:45.163451 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-7ba5fda7-b461-48ba-bb1e-46187054b9d6 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:45.164816 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-7ba5fda7-b461-48ba-bb1e-46187054b9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:45.210208 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9352cc-a056-49f9-a4ca-00c8071aec00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:45.211499 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9352cc-a056-49f9-a4ca-00c8071aec00 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:45.212051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9352cc-a056-49f9-a4ca-00c8071aec00 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:45.212539 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9352cc-a056-49f9-a4ca-00c8071aec00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:45.225493 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9352cc-a056-49f9-a4ca-00c8071aec00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:45.226962 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9352cc-a056-49f9-a4ca-00c8071aec00 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:45.244947 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 349/703] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:45.246361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5fda7-b461-48ba-bb1e-46187054b9d6 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:45.247139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-8b9352cc-a056-49f9-a4ca-00c8071aec00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:45.247139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:45.247139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:45.247139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:45.247139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:45.247139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:45.247139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:45.247139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:45.247139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:45.247139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:45.247139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:45.247139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:45.247139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:45.247139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:45.247139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:45.248111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:45.249710 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:45.249710 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:45.249710 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:45.249710 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:45.249710 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 355/704] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:45.253966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9352cc-a056-49f9-a4ca-00c8071aec00 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:45.296167 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:45.337836 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:45.363992 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5fda7-b461-48ba-bb1e-46187054b9d6 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:45.371499 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9352cc-a056-49f9-a4ca-00c8071aec00 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:45.412014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04864928-f243-4930-a60a-a43c4ff67cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:45.412941 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04864928-f243-4930-a60a-a43c4ff67cf3 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:45.413226 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04864928-f243-4930-a60a-a43c4ff67cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:45.413591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04864928-f243-4930-a60a-a43c4ff67cf3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:45.414413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04864928-f243-4930-a60a-a43c4ff67cf3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:45.421091 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3072b51-17ac-4126-b078-e9eca65e910f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:45.421770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3072b51-17ac-4126-b078-e9eca65e910f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:45.421889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04864928-f243-4930-a60a-a43c4ff67cf3 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:45.422083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3072b51-17ac-4126-b078-e9eca65e910f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:45.422362 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3072b51-17ac-4126-b078-e9eca65e910f None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:45.423091 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3072b51-17ac-4126-b078-e9eca65e910f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:45.423399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3072b51-17ac-4126-b078-e9eca65e910f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:45.425112 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3072b51-17ac-4126-b078-e9eca65e910f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:45.438391 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 356/705] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:45.439309 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3072b51-17ac-4126-b078-e9eca65e910f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-04864928-f243-4930-a60a-a43c4ff67cf3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:45.440366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:45.441470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:45.442516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 350/706] 10.222.0.84 () {56 vars in 1129 bytes} [Thu May 19 20:18:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:45.450286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04864928-f243-4930-a60a-a43c4ff67cf3 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:45.555339 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04864928-f243-4930-a60a-a43c4ff67cf3 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:45.557065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:45.563126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3072b51-17ac-4126-b078-e9eca65e910f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:45.594768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf57b41-ac0a-482b-8fc7-8c15c5837952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:45.595626 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf57b41-ac0a-482b-8fc7-8c15c5837952 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:45.595957 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf57b41-ac0a-482b-8fc7-8c15c5837952 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:45.596239 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf57b41-ac0a-482b-8fc7-8c15c5837952 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:45.601233 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:45.603607 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf57b41-ac0a-482b-8fc7-8c15c5837952 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:45.605871 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf57b41-ac0a-482b-8fc7-8c15c5837952 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:45.607671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144097d-be76-4cf6-a81a-f559c4a298c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:45.608921 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144097d-be76-4cf6-a81a-f559c4a298c8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:45.609212 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144097d-be76-4cf6-a81a-f559c4a298c8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:45.609489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144097d-be76-4cf6-a81a-f559c4a298c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:45.620969 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144097d-be76-4cf6-a81a-f559c4a298c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:45.621319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-2bf57b41-ac0a-482b-8fc7-8c15c5837952 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:45.621319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:45.621319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:45.621319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:45.621319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:45.621319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:45.621319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:45.621319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:45.621319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:45.621319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:45.621319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:45.621319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:45.621319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:45.621319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:45.621319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:45.623429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:45.625593 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:45.625593 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:45.625593 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:45.625593 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:45.625593 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:45.625593 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:45.625593 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:45.625593 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:45.626639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144097d-be76-4cf6-a81a-f559c4a298c8 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:45.634288 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 351/707] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:45.634288 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf57b41-ac0a-482b-8fc7-8c15c5837952 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:45.648671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-f144097d-be76-4cf6-a81a-f559c4a298c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:45.648671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:45.648671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:45.648671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:45.648671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:45.648671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:45.648671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:45.648671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:45.648671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:45.648671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:45.648671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:45.648671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:45.648671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:45.648671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:45.648671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:45.650538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:45.652814 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:45.652814 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:45.652814 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:45.652814 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:45.658349 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 357/708] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:45.658349 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144097d-be76-4cf6-a81a-f559c4a298c8 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:45.752931 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf57b41-ac0a-482b-8fc7-8c15c5837952 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:45.789198 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b19d4-8a79-41e6-8c3c-581135cf01d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:45.789851 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b19d4-8a79-41e6-8c3c-581135cf01d8 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:45.790129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b19d4-8a79-41e6-8c3c-581135cf01d8 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:45.790490 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b19d4-8a79-41e6-8c3c-581135cf01d8 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:45.791217 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b19d4-8a79-41e6-8c3c-581135cf01d8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:45.791405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144097d-be76-4cf6-a81a-f559c4a298c8 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:45.791591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b19d4-8a79-41e6-8c3c-581135cf01d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:45.792957 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b19d4-8a79-41e6-8c3c-581135cf01d8 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:45.799087 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 352/709] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 20:18:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:45.800751 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b19d4-8a79-41e6-8c3c-581135cf01d8 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:45.833544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bd824-9b68-4cab-909f-87b554ed4be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:45.834123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bd824-9b68-4cab-909f-87b554ed4be2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:45.834455 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bd824-9b68-4cab-909f-87b554ed4be2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:45.834729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bd824-9b68-4cab-909f-87b554ed4be2 None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:45.836274 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bd824-9b68-4cab-909f-87b554ed4be2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:45.836784 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bd824-9b68-4cab-909f-87b554ed4be2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:45.839475 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:45.840181 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bd824-9b68-4cab-909f-87b554ed4be2 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:45.855711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 358/710] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:45.856559 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bd824-9b68-4cab-909f-87b554ed4be2 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:45.919496 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:45.926508 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b19d4-8a79-41e6-8c3c-581135cf01d8 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:45.966403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06298337-679c-49a7-b09c-c0da963188b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:45.967111 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06298337-679c-49a7-b09c-c0da963188b0 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:45.967396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06298337-679c-49a7-b09c-c0da963188b0 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:45.967675 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06298337-679c-49a7-b09c-c0da963188b0 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:45.968552 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06298337-679c-49a7-b09c-c0da963188b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:45.968924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06298337-679c-49a7-b09c-c0da963188b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:45.970466 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06298337-679c-49a7-b09c-c0da963188b0 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:45.985894 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bd824-9b68-4cab-909f-87b554ed4be2 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:45.987913 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 353/711] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:45.989059 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06298337-679c-49a7-b09c-c0da963188b0 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:46.031814 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee80089-c677-4cf8-bbf2-3b345719b3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:46.032507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee80089-c677-4cf8-bbf2-3b345719b3bf None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:46.032793 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee80089-c677-4cf8-bbf2-3b345719b3bf None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:46.033073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee80089-c677-4cf8-bbf2-3b345719b3bf None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:46.034156 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee80089-c677-4cf8-bbf2-3b345719b3bf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:46.034512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee80089-c677-4cf8-bbf2-3b345719b3bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:46.036557 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee80089-c677-4cf8-bbf2-3b345719b3bf None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:46.044296 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 359/712] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 20:18:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:46.045653 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee80089-c677-4cf8-bbf2-3b345719b3bf None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:46.084287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:46.104347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06298337-679c-49a7-b09c-c0da963188b0 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:46.142944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd507e-1e23-42ab-85ca-08ea5b8a11dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:46.143721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd507e-1e23-42ab-85ca-08ea5b8a11dc None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:46.144007 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd507e-1e23-42ab-85ca-08ea5b8a11dc None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:46.144286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd507e-1e23-42ab-85ca-08ea5b8a11dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:46.145211 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd507e-1e23-42ab-85ca-08ea5b8a11dc None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=edb1bcb781084f03a4ad2611a09bd77e) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:46.145523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd507e-1e23-42ab-85ca-08ea5b8a11dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:46.147428 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd507e-1e23-42ab-85ca-08ea5b8a11dc None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:46.154687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 354/713] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:18:45 2022] GET /identity/v3/role_assignments?user.id=edb1bcb781084f03a4ad2611a09bd77e&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=b239f941cb724331bf28dc489b70ad1d => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:46.164036 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee80089-c677-4cf8-bbf2-3b345719b3bf None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:46.174848 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:46.219287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2118c20-8c3f-4589-a87e-16d62a096716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:46.219993 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2118c20-8c3f-4589-a87e-16d62a096716 None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:46.220274 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2118c20-8c3f-4589-a87e-16d62a096716 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:46.220550 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2118c20-8c3f-4589-a87e-16d62a096716 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:46.221270 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2118c20-8c3f-4589-a87e-16d62a096716 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:46.222773 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2118c20-8c3f-4589-a87e-16d62a096716 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:46.254388 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 360/714] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:18:46 2022] POST /identity/v3/users => generated 254 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:18:47.115116 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a93b7-c563-4709-a000-8d7626b3bb62 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:47.115851 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a93b7-c563-4709-a000-8d7626b3bb62 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:47.115851 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a93b7-c563-4709-a000-8d7626b3bb62 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:47.117079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 355/715] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:47.132008 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ce04c-eb85-4c3d-a6ef-1596f9f2836d None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:47.132308 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ce04c-eb85-4c3d-a6ef-1596f9f2836d None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:47.132750 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ce04c-eb85-4c3d-a6ef-1596f9f2836d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:47.159339 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-f28ce04c-eb85-4c3d-a6ef-1596f9f2836d None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:47.160563 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-f28ce04c-eb85-4c3d-a6ef-1596f9f2836d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:47.259696 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 361/716] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:18:47.294146 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a93b7-c563-4709-a000-8d7626b3bb62 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:47.440101 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a93b7-c563-4709-a000-8d7626b3bb62 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:47.477196 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb854df-1a48-488c-a98b-f34da9b46449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:47.478005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb854df-1a48-488c-a98b-f34da9b46449 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:47.478307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb854df-1a48-488c-a98b-f34da9b46449 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:47.478591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb854df-1a48-488c-a98b-f34da9b46449 None admin] PATH_INFO: `/v3/services/network` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:47.479246 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb854df-1a48-488c-a98b-f34da9b46449 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:47.480627 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb854df-1a48-488c-a98b-f34da9b46449 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-8eb854df-1a48-488c-a98b-f34da9b46449 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:47.492731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:47.493872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:47.495415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:47.495415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:47.495415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:47.495415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:47.495415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:47.495415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:47.495415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:47.495415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:47.495415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:47.495415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:47.495415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:18:47.495415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:18:47.495415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:18:47.495415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:47.495415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 356/717] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:47 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:47.500016 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ce04c-eb85-4c3d-a6ef-1596f9f2836d None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:47.554006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64531599-f683-4784-b002-a228b6be31c5 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:47.554443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64531599-f683-4784-b002-a228b6be31c5 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:47.554683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64531599-f683-4784-b002-a228b6be31c5 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:47.556358 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 357/718] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:47.562288 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ccfde-3021-4be1-8004-026937fff57e None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:47.562592 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ccfde-3021-4be1-8004-026937fff57e None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:47.562878 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ccfde-3021-4be1-8004-026937fff57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:47.592351 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-332ccfde-3021-4be1-8004-026937fff57e None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:47.593598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-332ccfde-3021-4be1-8004-026937fff57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:47.614718 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:47.661368 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:47.670704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ce04c-eb85-4c3d-a6ef-1596f9f2836d None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:47.721823 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 358/719] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:18:47.739221 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d66d6-e84d-45f5-aefe-9c737e7f6571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:47.739893 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d66d6-e84d-45f5-aefe-9c737e7f6571 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:47.740175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d66d6-e84d-45f5-aefe-9c737e7f6571 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:47.740452 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d66d6-e84d-45f5-aefe-9c737e7f6571 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:47.741175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d66d6-e84d-45f5-aefe-9c737e7f6571 None admin] RBAC: Adding query filter params (name=network) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:47.741479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d66d6-e84d-45f5-aefe-9c737e7f6571 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:47.742983 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d66d6-e84d-45f5-aefe-9c737e7f6571 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:47.750073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 362/720] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:18:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:47.755331 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ccfde-3021-4be1-8004-026937fff57e None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:47.772415 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d66d6-e84d-45f5-aefe-9c737e7f6571 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:47.879344 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d66d6-e84d-45f5-aefe-9c737e7f6571 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:47.923208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ccfde-3021-4be1-8004-026937fff57e None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:47.965370 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b159e-2791-4e61-bcf6-5a995bdbb335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:47.966455 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb1146-5b64-4c84-85f8-9c0e287f4a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:47.966726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b159e-2791-4e61-bcf6-5a995bdbb335 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:47.967285 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb1146-5b64-4c84-85f8-9c0e287f4a35 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:47.967404 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b159e-2791-4e61-bcf6-5a995bdbb335 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:47.967580 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb1146-5b64-4c84-85f8-9c0e287f4a35 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:47.967836 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb1146-5b64-4c84-85f8-9c0e287f4a35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:47.968395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b159e-2791-4e61-bcf6-5a995bdbb335 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:47.969392 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b159e-2791-4e61-bcf6-5a995bdbb335 None admin] RBAC: Adding query filter params (type=network) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:47.969924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b159e-2791-4e61-bcf6-5a995bdbb335 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:47.971574 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b159e-2791-4e61-bcf6-5a995bdbb335 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:47.978827 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 359/721] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:18:47 2022] GET /identity/v3/services?type=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:48.007537 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb1146-5b64-4c84-85f8-9c0e287f4a35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-18fb1146-5b64-4c84-85f8-9c0e287f4a35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:48.015406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:48.016463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:48.017982 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:48.017982 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:48.017982 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:48.017982 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:48.017982 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:48.017982 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:48.017982 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:48.017982 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:48.017982 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:48.017982 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:48.017982 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:48.017982 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:48.017982 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 363/722] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:18:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:48.022777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b159e-2791-4e61-bcf6-5a995bdbb335 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:48.042629 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:48.178474 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b159e-2791-4e61-bcf6-5a995bdbb335 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:48.223421 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a55f1-6d78-4e7a-81fe-197c53fb11d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:48.224493 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a55f1-6d78-4e7a-81fe-197c53fb11d5 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:48.224976 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a55f1-6d78-4e7a-81fe-197c53fb11d5 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:48.225466 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a55f1-6d78-4e7a-81fe-197c53fb11d5 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:48.227073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a55f1-6d78-4e7a-81fe-197c53fb11d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:48.227618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a55f1-6d78-4e7a-81fe-197c53fb11d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:48.229320 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a55f1-6d78-4e7a-81fe-197c53fb11d5 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:48.244138 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 360/723] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:48.251404 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb1146-5b64-4c84-85f8-9c0e287f4a35 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:48.279139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:48.389284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb1146-5b64-4c84-85f8-9c0e287f4a35 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:48.444856 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9e837-835c-47d3-8a8b-4d8c2131da63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:48.445695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9e837-835c-47d3-8a8b-4d8c2131da63 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:48.445975 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9e837-835c-47d3-8a8b-4d8c2131da63 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:48.446248 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9e837-835c-47d3-8a8b-4d8c2131da63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:48.462949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9e837-835c-47d3-8a8b-4d8c2131da63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:48.464752 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9e837-835c-47d3-8a8b-4d8c2131da63 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:48.484712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-93c9e837-835c-47d3-8a8b-4d8c2131da63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:48.484712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:48.484712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:48.484712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:48.484712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:48.484712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:48.484712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:48.484712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:48.484712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:48.484712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:48.484712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:48.484712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:48.484712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:48.484712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:48.484712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:48.485759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:48.486780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:48.486780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:48.486780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:48.486780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:48.489103 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 364/724] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:48.494989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a55f1-6d78-4e7a-81fe-197c53fb11d5 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:48.619839 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:48.684483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a55f1-6d78-4e7a-81fe-197c53fb11d5 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:48.734189 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e768606-2fec-4890-bccb-5e3c0df64756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:48.734951 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e768606-2fec-4890-bccb-5e3c0df64756 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:48.735174 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e768606-2fec-4890-bccb-5e3c0df64756 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:48.735455 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e768606-2fec-4890-bccb-5e3c0df64756 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:48.736182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e768606-2fec-4890-bccb-5e3c0df64756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:48.736487 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e768606-2fec-4890-bccb-5e3c0df64756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:48.737877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e768606-2fec-4890-bccb-5e3c0df64756 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:48.750554 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 361/725] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:48.755942 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9e837-835c-47d3-8a8b-4d8c2131da63 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:48.872224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:48.918999 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9e837-835c-47d3-8a8b-4d8c2131da63 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:48.951884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac6ce1-1ff7-4c9a-bcd7-55b4277e80cd None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:48.952252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac6ce1-1ff7-4c9a-bcd7-55b4277e80cd None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:48.952447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac6ce1-1ff7-4c9a-bcd7-55b4277e80cd None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:48.954072 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 362/726] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:48.967969 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acbdf1-5b0d-4228-ab6b-3b4d9e2cfb97 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:48.968158 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f19e43-54ad-4fe8-a640-accde2db90b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:48.968291 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acbdf1-5b0d-4228-ab6b-3b4d9e2cfb97 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:48.968517 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acbdf1-5b0d-4228-ab6b-3b4d9e2cfb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:48.969519 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f19e43-54ad-4fe8-a640-accde2db90b7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:48.969999 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f19e43-54ad-4fe8-a640-accde2db90b7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:48.970518 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f19e43-54ad-4fe8-a640-accde2db90b7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:48.979402 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f19e43-54ad-4fe8-a640-accde2db90b7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:48.981279 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f19e43-54ad-4fe8-a640-accde2db90b7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-b3f19e43-54ad-4fe8-a640-accde2db90b7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:48.996617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:48.997695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:48.998763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:48.998763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:48.998763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:48.998763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:48.998763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:48.998763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:48.998763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:48.998763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:48.998763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:18:48.998763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:49.001434 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 365/727] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:18:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:49.007041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f19e43-54ad-4fe8-a640-accde2db90b7 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:49.007191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-16acbdf1-5b0d-4228-ab6b-3b4d9e2cfb97 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:49.008709 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-16acbdf1-5b0d-4228-ab6b-3b4d9e2cfb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:49.124432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 363/728] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:49.130394 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136d20b-f4c5-4859-afa4-6a2d9d4c4f91 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:49.130394 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136d20b-f4c5-4859-afa4-6a2d9d4c4f91 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:49.130394 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136d20b-f4c5-4859-afa4-6a2d9d4c4f91 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:49.130394 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 364/729] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:49.133406 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287954b4-b7fe-43a2-aed0-2b79c35da633 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:49.134008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287954b4-b7fe-43a2-aed0-2b79c35da633 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:49.134140 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287954b4-b7fe-43a2-aed0-2b79c35da633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:49.144851 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f19e43-54ad-4fe8-a640-accde2db90b7 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:49.155906 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-287954b4-b7fe-43a2-aed0-2b79c35da633 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:49.159430 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-287954b4-b7fe-43a2-aed0-2b79c35da633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:49.191682 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:49.203803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746ffa9-bc18-4873-b8d2-421b24c2b4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:49.204451 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746ffa9-bc18-4873-b8d2-421b24c2b4d5 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:49.204779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746ffa9-bc18-4873-b8d2-421b24c2b4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:49.205055 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746ffa9-bc18-4873-b8d2-421b24c2b4d5 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:49.206135 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746ffa9-bc18-4873-b8d2-421b24c2b4d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:49.206488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746ffa9-bc18-4873-b8d2-421b24c2b4d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:49.208856 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746ffa9-bc18-4873-b8d2-421b24c2b4d5 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:49.235643 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:49.236174 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 366/730] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 20:18:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:49.236174 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746ffa9-bc18-4873-b8d2-421b24c2b4d5 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:49.250678 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 365/731] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:18:49.251884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287954b4-b7fe-43a2-aed0-2b79c35da633 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:49.376951 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746ffa9-bc18-4873-b8d2-421b24c2b4d5 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:49.391594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287954b4-b7fe-43a2-aed0-2b79c35da633 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:49.422300 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72ff64-c0f1-4cad-854d-fb79ba79da30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:49.423077 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72ff64-c0f1-4cad-854d-fb79ba79da30 None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:49.423353 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72ff64-c0f1-4cad-854d-fb79ba79da30 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:49.423639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72ff64-c0f1-4cad-854d-fb79ba79da30 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:49.424844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72ff64-c0f1-4cad-854d-fb79ba79da30 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:49.426534 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72ff64-c0f1-4cad-854d-fb79ba79da30 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:49.443471 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf405f4c-1ca2-44a8-b387-b574fc875d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:49.444350 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf405f4c-1ca2-44a8-b387-b574fc875d38 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:49.444733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf405f4c-1ca2-44a8-b387-b574fc875d38 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:49.446882 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf405f4c-1ca2-44a8-b387-b574fc875d38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:49.447978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 367/732] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:18:49 2022] POST /identity/v3/services => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:18:49.449121 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72ff64-c0f1-4cad-854d-fb79ba79da30 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:49.529565 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:49.531690 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:49.547064 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf405f4c-1ca2-44a8-b387-b574fc875d38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:49.548559 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf405f4c-1ca2-44a8-b387-b574fc875d38 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:49.628382 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72ff64-c0f1-4cad-854d-fb79ba79da30 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:49.658883 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 366/733] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:18:49.660326 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16fcd9-5522-4157-8e38-a0410890fd24 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:49.660619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16fcd9-5522-4157-8e38-a0410890fd24 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:49.660892 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16fcd9-5522-4157-8e38-a0410890fd24 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:49.662327 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 367/734] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:49.667085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16fcd9-5522-4157-8e38-a0410890fd24 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:49.677116 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b7ba5-552d-4c80-a564-17faa9b89dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:49.677928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b7ba5-552d-4c80-a564-17faa9b89dbe None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:49.678211 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b7ba5-552d-4c80-a564-17faa9b89dbe None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:49.678515 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b7ba5-552d-4c80-a564-17faa9b89dbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:49.691319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b7ba5-552d-4c80-a564-17faa9b89dbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-133b7ba5-552d-4c80-a564-17faa9b89dbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:49.699164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:49.700260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:49.701239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:49.701239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:49.701239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:49.701239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:49.701239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:49.701239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:49.701239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:49.701239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:49.701239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:49.701239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:49.701239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:49.701239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:49.701239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 368/735] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:18:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:49.702049 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc6919-231b-47a6-8bfa-9d2a53e0325c None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:49.702721 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc6919-231b-47a6-8bfa-9d2a53e0325c None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:49.702721 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc6919-231b-47a6-8bfa-9d2a53e0325c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:49.722579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-6dfc6919-231b-47a6-8bfa-9d2a53e0325c None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:49.723775 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-6dfc6919-231b-47a6-8bfa-9d2a53e0325c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:49.793465 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:49.795985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 369/736] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:49.799121 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc6919-231b-47a6-8bfa-9d2a53e0325c None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:49.817413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16fcd9-5522-4157-8e38-a0410890fd24 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:49.849692 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:49.855698 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798c358-846d-4699-8725-16fcae9ec952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:49.856505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798c358-846d-4699-8725-16fcae9ec952 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:49.856779 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798c358-846d-4699-8725-16fcae9ec952 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:49.857047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798c358-846d-4699-8725-16fcae9ec952 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:49.867977 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798c358-846d-4699-8725-16fcae9ec952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:49.869551 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798c358-846d-4699-8725-16fcae9ec952 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:49.888848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-6798c358-846d-4699-8725-16fcae9ec952 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:49.888848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:49.888848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:49.888848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:49.888848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:49.888848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:49.888848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:49.888848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:49.888848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:49.888848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:49.888848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:49.888848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:49.888848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:49.888848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:49.888848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:49.889923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:49.890953 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:49.890953 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:49.890953 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:49.890953 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:49.890953 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 368/737] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:49.891865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798c358-846d-4699-8725-16fcae9ec952 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:49.928364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc6919-231b-47a6-8bfa-9d2a53e0325c None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:49.968840 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd41f6-c859-4a53-9ce2-6f09f53b98ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:49.970143 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd41f6-c859-4a53-9ce2-6f09f53b98ed None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:49.970668 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd41f6-c859-4a53-9ce2-6f09f53b98ed None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:49.971437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd41f6-c859-4a53-9ce2-6f09f53b98ed None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:49.972750 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd41f6-c859-4a53-9ce2-6f09f53b98ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:49.973274 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd41f6-c859-4a53-9ce2-6f09f53b98ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:49.974973 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd41f6-c859-4a53-9ce2-6f09f53b98ed None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:49.990523 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 370/738] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:49.992341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd41f6-c859-4a53-9ce2-6f09f53b98ed None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:50.028757 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798c358-846d-4699-8725-16fcae9ec952 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:50.077300 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209bf8f-5654-4fe2-b75c-e77881dc3740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:50.078030 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209bf8f-5654-4fe2-b75c-e77881dc3740 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:50.078361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209bf8f-5654-4fe2-b75c-e77881dc3740 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:50.078641 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209bf8f-5654-4fe2-b75c-e77881dc3740 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:50.094251 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209bf8f-5654-4fe2-b75c-e77881dc3740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-2209bf8f-5654-4fe2-b75c-e77881dc3740 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:50.105377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:50.106500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:50.111375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:50.111375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:50.111375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:50.111375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:50.111375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:50.111375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:50.111375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:50.111375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:50.111375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:50.111375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:50.111375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:50.111375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:50.111375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 369/739] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:50.111375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209bf8f-5654-4fe2-b75c-e77881dc3740 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:50.112416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:50.116313 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:50.124853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd41f6-c859-4a53-9ce2-6f09f53b98ed None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:50.165172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4006b0-7896-4d0f-9aaf-0fc20a7c74b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:50.166385 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4006b0-7896-4d0f-9aaf-0fc20a7c74b9 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:50.166385 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4006b0-7896-4d0f-9aaf-0fc20a7c74b9 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:50.166808 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4006b0-7896-4d0f-9aaf-0fc20a7c74b9 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:50.167680 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4006b0-7896-4d0f-9aaf-0fc20a7c74b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:50.168086 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4006b0-7896-4d0f-9aaf-0fc20a7c74b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:50.169647 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4006b0-7896-4d0f-9aaf-0fc20a7c74b9 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:50.186996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 371/740] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:50.186996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4006b0-7896-4d0f-9aaf-0fc20a7c74b9 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:50.258347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209bf8f-5654-4fe2-b75c-e77881dc3740 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:50.300820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5b884-cef9-430f-9852-a93b85885f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:50.301658 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5b884-cef9-430f-9852-a93b85885f60 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:50.301944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5b884-cef9-430f-9852-a93b85885f60 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:50.302227 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5b884-cef9-430f-9852-a93b85885f60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:50.314277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5b884-cef9-430f-9852-a93b85885f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:50.315828 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5b884-cef9-430f-9852-a93b85885f60 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:50.318134 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4006b0-7896-4d0f-9aaf-0fc20a7c74b9 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:50.334804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-88b5b884-cef9-430f-9852-a93b85885f60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:50.334804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:50.334804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:50.334804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:50.334804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:50.334804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:50.334804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:50.334804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:50.334804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:50.334804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:50.334804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:50.334804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:50.334804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:50.334804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:50.334804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:50.335884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:50.336953 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:50.336953 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:50.336953 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:50.336953 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:50.341099 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 370/741] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:50.342060 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5b884-cef9-430f-9852-a93b85885f60 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:50.364218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21060b6c-e0e9-446c-9aed-0f79f43984a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:50.365000 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21060b6c-e0e9-446c-9aed-0f79f43984a6 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:50.365286 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21060b6c-e0e9-446c-9aed-0f79f43984a6 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:50.365568 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21060b6c-e0e9-446c-9aed-0f79f43984a6 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:50.366574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21060b6c-e0e9-446c-9aed-0f79f43984a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:50.366963 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21060b6c-e0e9-446c-9aed-0f79f43984a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:50.368689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21060b6c-e0e9-446c-9aed-0f79f43984a6 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:50.382503 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 372/742] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:50.384166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21060b6c-e0e9-446c-9aed-0f79f43984a6 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:50.413736 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:50.434921 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:50.495474 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5b884-cef9-430f-9852-a93b85885f60 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:50.528664 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21060b6c-e0e9-446c-9aed-0f79f43984a6 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:50.540106 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ceb759-bfdf-478d-95a7-1aee8c59af03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:50.542484 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ceb759-bfdf-478d-95a7-1aee8c59af03 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:50.542794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ceb759-bfdf-478d-95a7-1aee8c59af03 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:50.543077 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ceb759-bfdf-478d-95a7-1aee8c59af03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:50.553429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ceb759-bfdf-478d-95a7-1aee8c59af03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:50.555476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ceb759-bfdf-478d-95a7-1aee8c59af03 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:50.572073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8173028-b789-47df-82b7-2cd90c1977e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:50.572866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8173028-b789-47df-82b7-2cd90c1977e3 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:50.573222 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8173028-b789-47df-82b7-2cd90c1977e3 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:50.573592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8173028-b789-47df-82b7-2cd90c1977e3 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:50.574484 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8173028-b789-47df-82b7-2cd90c1977e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:50.574865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8173028-b789-47df-82b7-2cd90c1977e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:50.576388 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8173028-b789-47df-82b7-2cd90c1977e3 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:50.576820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-33ceb759-bfdf-478d-95a7-1aee8c59af03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:50.576820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:50.576820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:50.576820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:50.576820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:50.576820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:50.576820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:50.576820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:50.576820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:50.576820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:50.576820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:50.576820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:50.576820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:50.576820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:50.576820 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:50.577769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:50.578818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:50.578818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:50.578818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:50.578818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:50.578818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:50.578818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:50.578818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:50.578818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:50.584573 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 371/743] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:50.584573 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ceb759-bfdf-478d-95a7-1aee8c59af03 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:50.597441 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 373/744] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:50.598218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8173028-b789-47df-82b7-2cd90c1977e3 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:50.682229 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:50.692245 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ceb759-bfdf-478d-95a7-1aee8c59af03 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:50.707178 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:50.710286 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8173028-b789-47df-82b7-2cd90c1977e3 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:50.731926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446794b-c1ae-4163-b89b-eed2d9e48743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:50.732737 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446794b-c1ae-4163-b89b-eed2d9e48743 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:50.733018 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446794b-c1ae-4163-b89b-eed2d9e48743 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:50.733290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446794b-c1ae-4163-b89b-eed2d9e48743 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:50.744239 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446794b-c1ae-4163-b89b-eed2d9e48743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:50.745712 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446794b-c1ae-4163-b89b-eed2d9e48743 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:50.752663 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff0370-99d4-4180-bf65-30c47bb69d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:50.753724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff0370-99d4-4180-bf65-30c47bb69d50 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:50.754227 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff0370-99d4-4180-bf65-30c47bb69d50 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:50.754775 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff0370-99d4-4180-bf65-30c47bb69d50 None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:50.755870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff0370-99d4-4180-bf65-30c47bb69d50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:50.756365 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff0370-99d4-4180-bf65-30c47bb69d50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:50.758300 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff0370-99d4-4180-bf65-30c47bb69d50 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:50.764254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-b446794b-c1ae-4163-b89b-eed2d9e48743 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:50.764254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:50.764254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:50.764254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:50.764254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:50.764254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:50.764254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:50.764254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:50.764254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:50.764254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:50.764254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:50.764254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:50.764254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:50.764254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:50.764254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:50.765257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:50.766295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:50.766295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:50.766295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:50.766295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:50.766295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 372/745] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:50.766940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebd0ce-0e45-4aad-b4fa-dbda04103b74 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:50.767041 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebd0ce-0e45-4aad-b4fa-dbda04103b74 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:50.767258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebd0ce-0e45-4aad-b4fa-dbda04103b74 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:50.768794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 373/746] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:50.769498 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebd0ce-0e45-4aad-b4fa-dbda04103b74 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:50.784980 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 374/747] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:50.784980 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504106d-72ae-485a-8df6-6e5f0f0ce471 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:50.784980 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504106d-72ae-485a-8df6-6e5f0f0ce471 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:50.785335 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504106d-72ae-485a-8df6-6e5f0f0ce471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:50.802526 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-f504106d-72ae-485a-8df6-6e5f0f0ce471 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:50.803725 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-f504106d-72ae-485a-8df6-6e5f0f0ce471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:50.874967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebd0ce-0e45-4aad-b4fa-dbda04103b74 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:50.886584 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 375/748] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:18:50.887950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504106d-72ae-485a-8df6-6e5f0f0ce471 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:50.912291 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f4cf5-f80f-42a2-b558-3c6f58b8f0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:50.913112 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f4cf5-f80f-42a2-b558-3c6f58b8f0dd None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:50.913399 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f4cf5-f80f-42a2-b558-3c6f58b8f0dd None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:50.913870 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f4cf5-f80f-42a2-b558-3c6f58b8f0dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:50.921721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f4cf5-f80f-42a2-b558-3c6f58b8f0dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:50.923225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f4cf5-f80f-42a2-b558-3c6f58b8f0dd None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-bc2f4cf5-f80f-42a2-b558-3c6f58b8f0dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:50.937940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:50.939344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:50.940413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:50.940413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:50.940413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:50.940413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:50.940413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:50.940413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:50.940413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:50.940413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:50.940413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:18:50.940413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:50.943872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 374/749] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:18:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:50.944619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f4cf5-f80f-42a2-b558-3c6f58b8f0dd None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:50.964857 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:50.973153 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:51.000333 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504106d-72ae-485a-8df6-6e5f0f0ce471 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:51.036158 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706aef8-8027-48cd-be02-bc2600e9ee9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:51.036829 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706aef8-8027-48cd-be02-bc2600e9ee9a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:51.037108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706aef8-8027-48cd-be02-bc2600e9ee9a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:51.037379 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706aef8-8027-48cd-be02-bc2600e9ee9a None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:51.038092 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706aef8-8027-48cd-be02-bc2600e9ee9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:51.038741 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706aef8-8027-48cd-be02-bc2600e9ee9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:51.040080 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706aef8-8027-48cd-be02-bc2600e9ee9a None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:51.056135 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 376/750] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:51.056923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706aef8-8027-48cd-be02-bc2600e9ee9a None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:51.065395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f4cf5-f80f-42a2-b558-3c6f58b8f0dd None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:51.099745 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a87a4-942f-402e-a2d6-707ce37baee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:51.100409 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a87a4-942f-402e-a2d6-707ce37baee4 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:51.100687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a87a4-942f-402e-a2d6-707ce37baee4 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:51.100959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a87a4-942f-402e-a2d6-707ce37baee4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:51.101841 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a87a4-942f-402e-a2d6-707ce37baee4 None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=54395ae480884de38fc811a9e82c1f9e) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:51.102156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a87a4-942f-402e-a2d6-707ce37baee4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:51.103626 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a87a4-942f-402e-a2d6-707ce37baee4 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:51.109174 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 375/751] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:18:50 2022] GET /identity/v3/role_assignments?user.id=54395ae480884de38fc811a9e82c1f9e&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=b239f941cb724331bf28dc489b70ad1d => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:51.109957 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a87a4-942f-402e-a2d6-707ce37baee4 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:51.191584 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706aef8-8027-48cd-be02-bc2600e9ee9a None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:51.221995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:51.231993 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfc4a3-5aa7-43cc-8a51-ec74017b0969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:51.232841 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfc4a3-5aa7-43cc-8a51-ec74017b0969 None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:51.233215 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfc4a3-5aa7-43cc-8a51-ec74017b0969 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:51.233591 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfc4a3-5aa7-43cc-8a51-ec74017b0969 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:51.234468 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfc4a3-5aa7-43cc-8a51-ec74017b0969 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:51.236010 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfc4a3-5aa7-43cc-8a51-ec74017b0969 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:51.245392 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:51.262311 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 377/752] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:18:51 2022] POST /identity/v3/services => generated 240 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:18:51.262311 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfc4a3-5aa7-43cc-8a51-ec74017b0969 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:51.263145 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a87a4-942f-402e-a2d6-707ce37baee4 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:51.361572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca014e-a6c4-4638-9ad9-13bdebda0299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:51.362727 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca014e-a6c4-4638-9ad9-13bdebda0299 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:51.363216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca014e-a6c4-4638-9ad9-13bdebda0299 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:51.363747 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca014e-a6c4-4638-9ad9-13bdebda0299 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:51.364907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca014e-a6c4-4638-9ad9-13bdebda0299 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:51.365494 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca014e-a6c4-4638-9ad9-13bdebda0299 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:51.367289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca014e-a6c4-4638-9ad9-13bdebda0299 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:51.390407 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 376/753] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:18:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:51.395399 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca014e-a6c4-4638-9ad9-13bdebda0299 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:51.399233 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfc4a3-5aa7-43cc-8a51-ec74017b0969 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:51.471180 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090c0dd-8c59-4457-a408-05ff9f6e6e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:51.472103 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090c0dd-8c59-4457-a408-05ff9f6e6e01 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:51.472402 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090c0dd-8c59-4457-a408-05ff9f6e6e01 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:51.472685 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090c0dd-8c59-4457-a408-05ff9f6e6e01 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:51.490646 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090c0dd-8c59-4457-a408-05ff9f6e6e01 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:51.492389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090c0dd-8c59-4457-a408-05ff9f6e6e01 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-f090c0dd-8c59-4457-a408-05ff9f6e6e01 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:51.519816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:51.521083 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:51.522148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:51.522148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:51.522148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:51.522148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:51.522148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:51.522148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:51.522148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:51.522148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:51.522148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:18:51.522148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:51.522148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 378/754] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:18:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:51.527332 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090c0dd-8c59-4457-a408-05ff9f6e6e01 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:51.582053 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca014e-a6c4-4638-9ad9-13bdebda0299 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:51.592365 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:51.626823 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dbe8f-64a7-472a-9bd3-e02c97db7535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:51.627768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dbe8f-64a7-472a-9bd3-e02c97db7535 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:51.628162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dbe8f-64a7-472a-9bd3-e02c97db7535 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:51.628507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dbe8f-64a7-472a-9bd3-e02c97db7535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:51.642699 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03dbe8f-64a7-472a-9bd3-e02c97db7535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:51.645198 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03dbe8f-64a7-472a-9bd3-e02c97db7535 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:51.648795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:51.658044 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090c0dd-8c59-4457-a408-05ff9f6e6e01 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:51.669135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-b03dbe8f-64a7-472a-9bd3-e02c97db7535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:51.669135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:51.669135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:51.669135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:51.669135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:51.669135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:51.669135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:51.669135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:51.669135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:51.669135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:51.669135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:51.669135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:51.669135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:51.669135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:51.669135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:51.670212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:51.671430 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:51.671430 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:51.671430 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:51.671430 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:51.673516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 377/755] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:51.679438 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dbe8f-64a7-472a-9bd3-e02c97db7535 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:51.711207 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23f8ca-a5ce-4830-8be8-69939d59198c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:51.711905 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23f8ca-a5ce-4830-8be8-69939d59198c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:51.712190 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23f8ca-a5ce-4830-8be8-69939d59198c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:51.712734 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23f8ca-a5ce-4830-8be8-69939d59198c None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:51.714395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23f8ca-a5ce-4830-8be8-69939d59198c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:51.714714 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23f8ca-a5ce-4830-8be8-69939d59198c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:51.716696 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23f8ca-a5ce-4830-8be8-69939d59198c None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:51.740503 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 379/756] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 20:18:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:51.745992 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23f8ca-a5ce-4830-8be8-69939d59198c None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:51.812979 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dbe8f-64a7-472a-9bd3-e02c97db7535 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:51.882064 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df994738-1990-4c10-ac22-18f35e93d523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:51.882868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df994738-1990-4c10-ac22-18f35e93d523 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:51.883175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df994738-1990-4c10-ac22-18f35e93d523 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:51.883612 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df994738-1990-4c10-ac22-18f35e93d523 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:51.884342 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df994738-1990-4c10-ac22-18f35e93d523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:51.884646 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df994738-1990-4c10-ac22-18f35e93d523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:51.886048 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df994738-1990-4c10-ac22-18f35e93d523 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:51.898813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 378/757] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:51.900309 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f0648-0a28-45b4-9b5e-196a1197ca8e None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:51.900592 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f0648-0a28-45b4-9b5e-196a1197ca8e None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:51.900868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f0648-0a28-45b4-9b5e-196a1197ca8e None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:51.902447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 379/758] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:51.907417 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f0648-0a28-45b4-9b5e-196a1197ca8e None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:51.920289 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23f8ca-a5ce-4830-8be8-69939d59198c None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:51.929475 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:51.969144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9751a-1754-4a21-a9ee-f81514134e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:51.970167 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9751a-1754-4a21-a9ee-f81514134e4d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:51.970634 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9751a-1754-4a21-a9ee-f81514134e4d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:51.971094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9751a-1754-4a21-a9ee-f81514134e4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:51.977393 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:52.036101 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f0648-0a28-45b4-9b5e-196a1197ca8e None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:52.047834 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a9751a-1754-4a21-a9ee-f81514134e4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:52.049733 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a9751a-1754-4a21-a9ee-f81514134e4d None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:52.074546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9184465-f405-428a-93b9-b06a884d4c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:52.075449 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9184465-f405-428a-93b9-b06a884d4c3f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:52.075731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9184465-f405-428a-93b9-b06a884d4c3f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:52.076006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9184465-f405-428a-93b9-b06a884d4c3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:52.083799 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9184465-f405-428a-93b9-b06a884d4c3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:52.085255 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9184465-f405-428a-93b9-b06a884d4c3f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:52.101242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-b9184465-f405-428a-93b9-b06a884d4c3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:52.101242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:52.101242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:52.101242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:52.101242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:52.101242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:52.101242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:52.101242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:52.101242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:52.101242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:52.101242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:52.101242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:52.101242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:52.101242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:52.101242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:52.102505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:52.103748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:52.103748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:52.103748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:52.103748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:52.103748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:52.103748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:52.103748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:52.103748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:52.103748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 380/759] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:52.104393 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d91fce-99d7-4d0a-843c-5eed0f7e7c32 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:52.104393 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d91fce-99d7-4d0a-843c-5eed0f7e7c32 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:52.104644 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d91fce-99d7-4d0a-843c-5eed0f7e7c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:52.124238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-44d91fce-99d7-4d0a-843c-5eed0f7e7c32 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:52.125710 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-44d91fce-99d7-4d0a-843c-5eed0f7e7c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:52.164351 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 380/760] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:51 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:18:52.165255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9751a-1754-4a21-a9ee-f81514134e4d None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:52.173668 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:52.207337 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 381/761] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:18:52.208451 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d91fce-99d7-4d0a-843c-5eed0f7e7c32 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:52.269323 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:52.306522 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9751a-1754-4a21-a9ee-f81514134e4d None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:52.327540 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d91fce-99d7-4d0a-843c-5eed0f7e7c32 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:52.352562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d04b0-0633-4f0c-9d85-4ad50ca0f7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:52.353360 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d04b0-0633-4f0c-9d85-4ad50ca0f7a1 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:52.353694 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d04b0-0633-4f0c-9d85-4ad50ca0f7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:52.353973 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d04b0-0633-4f0c-9d85-4ad50ca0f7a1 None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:52.354816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d04b0-0633-4f0c-9d85-4ad50ca0f7a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:52.355128 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d04b0-0633-4f0c-9d85-4ad50ca0f7a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:52.356592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d04b0-0633-4f0c-9d85-4ad50ca0f7a1 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:52.373174 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 381/762] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:52.374414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d04b0-0633-4f0c-9d85-4ad50ca0f7a1 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:52.374741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c75cf2-33c0-40a6-87be-de697fa47af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:52.375772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c75cf2-33c0-40a6-87be-de697fa47af6 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:52.376153 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c75cf2-33c0-40a6-87be-de697fa47af6 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:52.376592 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c75cf2-33c0-40a6-87be-de697fa47af6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:52.389713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c75cf2-33c0-40a6-87be-de697fa47af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:52.391833 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c75cf2-33c0-40a6-87be-de697fa47af6 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:52.411924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-c8c75cf2-33c0-40a6-87be-de697fa47af6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:52.411924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:52.411924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:52.411924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:52.411924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:52.411924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:52.411924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:52.411924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:52.411924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:52.411924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:52.411924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:52.411924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:52.411924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:52.411924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:52.411924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:52.413181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:52.414538 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:52.414538 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:52.414538 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:52.414538 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:52.415985 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 382/763] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:52.422386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c75cf2-33c0-40a6-87be-de697fa47af6 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:52.463399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:52.506368 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d04b0-0633-4f0c-9d85-4ad50ca0f7a1 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:52.536767 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:52.542789 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c75cf2-33c0-40a6-87be-de697fa47af6 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:52.552786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54fa66-ba0c-4745-9f03-6ea8061741c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:52.554394 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54fa66-ba0c-4745-9f03-6ea8061741c6 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:52.554869 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54fa66-ba0c-4745-9f03-6ea8061741c6 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:52.555701 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54fa66-ba0c-4745-9f03-6ea8061741c6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:52.556850 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54fa66-ba0c-4745-9f03-6ea8061741c6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:52.559293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54fa66-ba0c-4745-9f03-6ea8061741c6 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-3b54fa66-ba0c-4745-9f03-6ea8061741c6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:52.578189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:52.579453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:52.580602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:52.580602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:52.580602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:52.580602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:52.580602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:52.580602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:52.580602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:52.580602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:52.580602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:52.580602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:52.580602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:52.580602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:18:52.580602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:18:52.580602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:18:52.580602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:52.587950 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34734f49-eec9-4bde-b329-a487a5bfcd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:52.587950 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34734f49-eec9-4bde-b329-a487a5bfcd29 None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:52.587950 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34734f49-eec9-4bde-b329-a487a5bfcd29 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:52.587950 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34734f49-eec9-4bde-b329-a487a5bfcd29 None admin] PATH_INFO: `/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/edb1bcb781084f03a4ad2611a09bd77e/roles/b239f941cb724331bf28dc489b70ad1d` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:52.588362 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34734f49-eec9-4bde-b329-a487a5bfcd29 None admin] RBAC: Authorizing `identity:create_grant(project_id=59555e7adac54eddb47d1ef18ef870c4, user_id=edb1bcb781084f03a4ad2611a09bd77e, role_id=b239f941cb724331bf28dc489b70ad1d)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:52.589185 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 382/764] 10.222.0.84 () {56 vars in 1129 bytes} [Thu May 19 20:18:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:52.589479 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34734f49-eec9-4bde-b329-a487a5bfcd29 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:52.590516 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54fa66-ba0c-4745-9f03-6ea8061741c6 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:52.609639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 383/765] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:18:52 2022] PUT /identity/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/edb1bcb781084f03a4ad2611a09bd77e/roles/b239f941cb724331bf28dc489b70ad1d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:18:52.612064 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34734f49-eec9-4bde-b329-a487a5bfcd29 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:52.816389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54fa66-ba0c-4745-9f03-6ea8061741c6 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:52.820430 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34734f49-eec9-4bde-b329-a487a5bfcd29 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:52.863888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:52.872631 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd94109-639f-434b-86b3-1d4272c2162b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:52.873153 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b7fb2-fe13-4571-bc53-fd1106b3ccf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:52.873307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd94109-639f-434b-86b3-1d4272c2162b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:52.873573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd94109-639f-434b-86b3-1d4272c2162b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:52.873847 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd94109-639f-434b-86b3-1d4272c2162b None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:52.874468 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b7fb2-fe13-4571-bc53-fd1106b3ccf6 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:52.874603 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd94109-639f-434b-86b3-1d4272c2162b None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:52.874896 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd94109-639f-434b-86b3-1d4272c2162b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:52.875201 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b7fb2-fe13-4571-bc53-fd1106b3ccf6 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:52.875627 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b7fb2-fe13-4571-bc53-fd1106b3ccf6 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:52.876284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd94109-639f-434b-86b3-1d4272c2162b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:52.876658 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b7fb2-fe13-4571-bc53-fd1106b3ccf6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:52.877163 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b7fb2-fe13-4571-bc53-fd1106b3ccf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:52.879654 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b7fb2-fe13-4571-bc53-fd1106b3ccf6 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:52.885825 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 384/766] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 20:18:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:52.892177 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b7fb2-fe13-4571-bc53-fd1106b3ccf6 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:52.894805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 383/767] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:52.901018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd94109-639f-434b-86b3-1d4272c2162b None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:52.942757 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:53.073724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd94109-639f-434b-86b3-1d4272c2162b None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:53.083290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b7fb2-fe13-4571-bc53-fd1106b3ccf6 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:53.124210 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd92b6-be0f-4135-b6c1-d36f95307eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:53.125294 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd92b6-be0f-4135-b6c1-d36f95307eff None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:53.125757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd92b6-be0f-4135-b6c1-d36f95307eff None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:53.126191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd92b6-be0f-4135-b6c1-d36f95307eff None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:53.141110 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd92b6-be0f-4135-b6c1-d36f95307eff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:53.144677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd92b6-be0f-4135-b6c1-d36f95307eff None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:53.149625 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cb04e-5d13-4851-9ccf-dde8c3dc7b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:53.151290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cb04e-5d13-4851-9ccf-dde8c3dc7b57 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:53.151922 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cb04e-5d13-4851-9ccf-dde8c3dc7b57 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:53.155692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cb04e-5d13-4851-9ccf-dde8c3dc7b57 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:53.156849 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cb04e-5d13-4851-9ccf-dde8c3dc7b57 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:53.162629 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cb04e-5d13-4851-9ccf-dde8c3dc7b57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:53.164485 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cb04e-5d13-4851-9ccf-dde8c3dc7b57 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:53.169436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-c7fd92b6-be0f-4135-b6c1-d36f95307eff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:53.169436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:53.169436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:53.169436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:53.169436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:53.169436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:53.169436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:53.169436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:53.169436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:53.169436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:53.169436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:53.169436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:53.169436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:53.169436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:53.169436 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:53.170609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:53.171661 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:53.171661 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:53.171661 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:53.171661 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:53.171661 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:53.171661 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:53.171661 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:53.171661 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:53.173349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 385/768] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 20:18:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:53.178348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 384/769] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:53.179523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cb04e-5d13-4851-9ccf-dde8c3dc7b57 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:53.192186 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd92b6-be0f-4135-b6c1-d36f95307eff None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:53.286988 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:53.349638 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:53.372292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cb04e-5d13-4851-9ccf-dde8c3dc7b57 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:53.391144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd92b6-be0f-4135-b6c1-d36f95307eff None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:53.448703 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7662253-2324-44bd-ba63-93f0aa205e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:53.449386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7662253-2324-44bd-ba63-93f0aa205e8f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:53.449662 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7662253-2324-44bd-ba63-93f0aa205e8f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:53.449931 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7662253-2324-44bd-ba63-93f0aa205e8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:53.450758 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7662253-2324-44bd-ba63-93f0aa205e8f None admin] RBAC: Adding query filter params (interface=public, service_id=39044c73c61c4b818f2a4607b78cbff5, region_id=RegionOne) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:53.451067 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7662253-2324-44bd-ba63-93f0aa205e8f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:53.452445 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7662253-2324-44bd-ba63-93f0aa205e8f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:53.464743 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99ec42-78b8-410d-a16e-1b2abce92266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:53.465422 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99ec42-78b8-410d-a16e-1b2abce92266 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:53.465702 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99ec42-78b8-410d-a16e-1b2abce92266 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:53.465975 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99ec42-78b8-410d-a16e-1b2abce92266 None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:53.466801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99ec42-78b8-410d-a16e-1b2abce92266 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:53.467108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99ec42-78b8-410d-a16e-1b2abce92266 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:53.468562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99ec42-78b8-410d-a16e-1b2abce92266 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:53.472536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 386/770] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:18:53 2022] GET /identity/v3/endpoints?service_id=39044c73c61c4b818f2a4607b78cbff5&interface=public®ion_id=RegionOne => generated 194 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:53.477508 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7662253-2324-44bd-ba63-93f0aa205e8f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:53.502416 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 385/771] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:53.522879 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99ec42-78b8-410d-a16e-1b2abce92266 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:53.625101 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7662253-2324-44bd-ba63-93f0aa205e8f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:53.638569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99ec42-78b8-410d-a16e-1b2abce92266 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:53.659595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72b564-65a9-44dc-aa85-24fe520bb6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:53.660289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72b564-65a9-44dc-aa85-24fe520bb6a5 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:53.660566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72b564-65a9-44dc-aa85-24fe520bb6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:53.660838 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72b564-65a9-44dc-aa85-24fe520bb6a5 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:53.661554 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72b564-65a9-44dc-aa85-24fe520bb6a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:53.663040 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72b564-65a9-44dc-aa85-24fe520bb6a5 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:53.666686 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:53.669065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 387/772] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:18:53 2022] GET /identity/v3/services => generated 772 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:53.670484 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67666a01-ef08-4e4f-8841-25bae2dcd714 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:53.670773 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67666a01-ef08-4e4f-8841-25bae2dcd714 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:53.671050 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67666a01-ef08-4e4f-8841-25bae2dcd714 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:53.672700 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 388/773] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:53.678841 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f27c14-f000-4e95-93ed-dec1cdce86dc None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:53.679543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f27c14-f000-4e95-93ed-dec1cdce86dc None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:53.680074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f27c14-f000-4e95-93ed-dec1cdce86dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:53.691039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb460f-6845-48e7-93da-83f34825939e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:53.691729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb460f-6845-48e7-93da-83f34825939e None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:53.692011 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb460f-6845-48e7-93da-83f34825939e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:53.692286 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb460f-6845-48e7-93da-83f34825939e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:53.693132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb460f-6845-48e7-93da-83f34825939e None admin] RBAC: Adding query filter params (role.id=c9206a4a0d49473f87a45d6c8abb2e51, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=265a6cde297448adb102da0773e366fb) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:53.693441 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb460f-6845-48e7-93da-83f34825939e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:53.695102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb460f-6845-48e7-93da-83f34825939e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:53.700530 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-64f27c14-f000-4e95-93ed-dec1cdce86dc None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:53.702019 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-64f27c14-f000-4e95-93ed-dec1cdce86dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:53.704449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 386/774] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:18:53 2022] GET /identity/v3/role_assignments?user.id=265a6cde297448adb102da0773e366fb&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=c9206a4a0d49473f87a45d6c8abb2e51 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:53.726822 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:53.836619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 389/775] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:18:53.880453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb460f-6845-48e7-93da-83f34825939e None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:53.991434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f9364-7629-47f6-9e00-55625a762d17 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:53.991813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f9364-7629-47f6-9e00-55625a762d17 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:53.992027 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f9364-7629-47f6-9e00-55625a762d17 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:53.993944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 390/776] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:53.999649 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2deadf2-a44b-4132-afd1-6632685a3e7c None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:53.999936 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2deadf2-a44b-4132-afd1-6632685a3e7c None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:54.000218 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2deadf2-a44b-4132-afd1-6632685a3e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:54.004096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb460f-6845-48e7-93da-83f34825939e None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:54.019863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-d2deadf2-a44b-4132-afd1-6632685a3e7c None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:54.021080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-d2deadf2-a44b-4132-afd1-6632685a3e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:54.045222 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02a131-3f7d-4d4f-8694-e61b3c75c3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:54.046425 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02a131-3f7d-4d4f-8694-e61b3c75c3c5 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:54.046920 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02a131-3f7d-4d4f-8694-e61b3c75c3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:54.047537 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02a131-3f7d-4d4f-8694-e61b3c75c3c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:54.059164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02a131-3f7d-4d4f-8694-e61b3c75c3c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-da02a131-3f7d-4d4f-8694-e61b3c75c3c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:54.067490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:54.068589 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:54.069543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:54.069543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:54.069543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:54.069543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:54.069543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:54.069543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:54.069543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:54.069543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:54.069543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:54.069543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:54.069543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:54.069543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:54.073719 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 387/777] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:18:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:54.080154 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02a131-3f7d-4d4f-8694-e61b3c75c3c5 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:54.114306 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 391/778] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:18:54.127823 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:54.144484 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2deadf2-a44b-4132-afd1-6632685a3e7c None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:54.218204 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02a131-3f7d-4d4f-8694-e61b3c75c3c5 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:54.237011 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:54.285940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2deadf2-a44b-4132-afd1-6632685a3e7c None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:54.300574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2c86b-5e3d-426f-8fca-49959d7a638c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:54.301719 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2c86b-5e3d-426f-8fca-49959d7a638c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:54.302211 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2c86b-5e3d-426f-8fca-49959d7a638c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:54.302757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2c86b-5e3d-426f-8fca-49959d7a638c None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:54.303733 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2c86b-5e3d-426f-8fca-49959d7a638c None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:54.304420 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2c86b-5e3d-426f-8fca-49959d7a638c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:54.306205 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2c86b-5e3d-426f-8fca-49959d7a638c None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:54.316754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 388/779] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:54.327792 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2c86b-5e3d-426f-8fca-49959d7a638c None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:54.484775 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056cfc4-925e-4205-b4fd-f8e579b12959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:54.485590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056cfc4-925e-4205-b4fd-f8e579b12959 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:54.485870 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056cfc4-925e-4205-b4fd-f8e579b12959 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:54.486144 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056cfc4-925e-4205-b4fd-f8e579b12959 None admin] PATH_INFO: `/v3/services/network` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:54.486826 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056cfc4-925e-4205-b4fd-f8e579b12959 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:54.488465 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056cfc4-925e-4205-b4fd-f8e579b12959 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-c056cfc4-925e-4205-b4fd-f8e579b12959 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:54.505996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:54.507190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:54.508197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:54.508197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:54.508197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:54.508197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:54.508197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:54.508197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:54.508197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:54.508197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:54.508197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:54.508197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:54.508197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:18:54.508197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:18:54.508197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:18:54.508197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:54.509031 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 392/780] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:54 2022] GET /identity/v3/services/network => generated 88 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:54.520567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056cfc4-925e-4205-b4fd-f8e579b12959 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:54.528068 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:54.534347 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2c86b-5e3d-426f-8fca-49959d7a638c None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:54.606962 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5003f-6e90-4af0-bcb8-ef93bac94d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:54.607772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5003f-6e90-4af0-bcb8-ef93bac94d6b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:54.608048 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5003f-6e90-4af0-bcb8-ef93bac94d6b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:54.608323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5003f-6e90-4af0-bcb8-ef93bac94d6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:54.633599 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b5003f-6e90-4af0-bcb8-ef93bac94d6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:54.635113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b5003f-6e90-4af0-bcb8-ef93bac94d6b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:54.660544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-c7b5003f-6e90-4af0-bcb8-ef93bac94d6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:54.660544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:54.660544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:54.660544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:54.660544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:54.660544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:54.660544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:54.660544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:54.660544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:54.660544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:54.660544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:54.660544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:54.660544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:54.660544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:54.660544 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:54.661622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:54.662699 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:54.662699 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:54.662699 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:54.662699 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:54.662699 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 389/781] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:54.671951 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5003f-6e90-4af0-bcb8-ef93bac94d6b None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:54.723745 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056cfc4-925e-4205-b4fd-f8e579b12959 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:54.786993 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef19c8-dcf5-4e0d-a580-df4dfd9e824a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:54.787666 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef19c8-dcf5-4e0d-a580-df4dfd9e824a None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:54.787943 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef19c8-dcf5-4e0d-a580-df4dfd9e824a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:54.788214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef19c8-dcf5-4e0d-a580-df4dfd9e824a None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:54.788964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef19c8-dcf5-4e0d-a580-df4dfd9e824a None admin] RBAC: Adding query filter params (name=network) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:54.789266 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef19c8-dcf5-4e0d-a580-df4dfd9e824a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:54.792462 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef19c8-dcf5-4e0d-a580-df4dfd9e824a None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:54.800581 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 393/782] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:18:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:54.807265 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef19c8-dcf5-4e0d-a580-df4dfd9e824a None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:54.810557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:54.857742 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5003f-6e90-4af0-bcb8-ef93bac94d6b None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:54.922865 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85db9e4-53fc-40a1-b70c-66502409cea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:54.923541 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85db9e4-53fc-40a1-b70c-66502409cea8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:54.923817 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85db9e4-53fc-40a1-b70c-66502409cea8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:54.924092 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85db9e4-53fc-40a1-b70c-66502409cea8 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:54.924817 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85db9e4-53fc-40a1-b70c-66502409cea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:54.925121 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85db9e4-53fc-40a1-b70c-66502409cea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:54.926535 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85db9e4-53fc-40a1-b70c-66502409cea8 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:54.940609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 390/783] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:54.947297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85db9e4-53fc-40a1-b70c-66502409cea8 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:54.959415 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:55.014092 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef19c8-dcf5-4e0d-a580-df4dfd9e824a None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:55.090619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20b1a5-4f77-4067-b301-b4d68316cb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:55.091326 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20b1a5-4f77-4067-b301-b4d68316cb67 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:55.091603 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20b1a5-4f77-4067-b301-b4d68316cb67 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:55.091874 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20b1a5-4f77-4067-b301-b4d68316cb67 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:55.092635 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20b1a5-4f77-4067-b301-b4d68316cb67 None admin] RBAC: Adding query filter params (type=network) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:55.092938 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20b1a5-4f77-4067-b301-b4d68316cb67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:55.094400 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20b1a5-4f77-4067-b301-b4d68316cb67 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:55.103861 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85db9e4-53fc-40a1-b70c-66502409cea8 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:55.110553 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 394/784] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:18:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:55.116300 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20b1a5-4f77-4067-b301-b4d68316cb67 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:55.175172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354494ad-b3c7-4e0e-989a-3d025bd657fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:55.175872 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354494ad-b3c7-4e0e-989a-3d025bd657fc None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:55.176151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354494ad-b3c7-4e0e-989a-3d025bd657fc None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:55.176429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354494ad-b3c7-4e0e-989a-3d025bd657fc None admin] PATH_INFO: `/v3/users/swift` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:55.184685 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354494ad-b3c7-4e0e-989a-3d025bd657fc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:55.186164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354494ad-b3c7-4e0e-989a-3d025bd657fc None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:55.206619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-354494ad-b3c7-4e0e-989a-3d025bd657fc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:55.222142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:55.223293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:55.224348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:55.224348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:55.224348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:55.224348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:55.224348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:55.224348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:55.224348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:55.224348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:55.224348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:18:55.224348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:55.224348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 391/785] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:18:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:55.234819 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354494ad-b3c7-4e0e-989a-3d025bd657fc None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:55.240949 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20b1a5-4f77-4067-b301-b4d68316cb67 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:55.277745 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa9515-6cab-40ce-9bc8-c4a45894da0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:55.278764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa9515-6cab-40ce-9bc8-c4a45894da0d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:55.279274 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa9515-6cab-40ce-9bc8-c4a45894da0d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:55.279740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa9515-6cab-40ce-9bc8-c4a45894da0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:55.280741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa9515-6cab-40ce-9bc8-c4a45894da0d None admin] RBAC: Adding query filter params (interface=public, service_id=5d69fc7a42214dc68f0c9bdd9594013b, region_id=RegionOne) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:55.281262 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa9515-6cab-40ce-9bc8-c4a45894da0d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:55.283012 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa9515-6cab-40ce-9bc8-c4a45894da0d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:55.289969 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 395/786] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:18:55 2022] GET /identity/v3/endpoints?service_id=5d69fc7a42214dc68f0c9bdd9594013b&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:18:55.291916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc1751-ef0f-47a6-989a-c20e7109e38b None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:55.292424 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc1751-ef0f-47a6-989a-c20e7109e38b None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:55.292900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc1751-ef0f-47a6-989a-c20e7109e38b None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:55.294863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 396/787] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:55.300194 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc1751-ef0f-47a6-989a-c20e7109e38b None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:55.305915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:55.340138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354494ad-b3c7-4e0e-989a-3d025bd657fc None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:55.378732 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbfa6c-8a3d-4e16-bf36-da5372fb9ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:55.379480 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbfa6c-8a3d-4e16-bf36-da5372fb9ebe None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:55.379765 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbfa6c-8a3d-4e16-bf36-da5372fb9ebe None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:55.380042 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbfa6c-8a3d-4e16-bf36-da5372fb9ebe None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:55.381052 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbfa6c-8a3d-4e16-bf36-da5372fb9ebe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:55.381823 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbfa6c-8a3d-4e16-bf36-da5372fb9ebe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:55.383693 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbfa6c-8a3d-4e16-bf36-da5372fb9ebe None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:55.405670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 392/788] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 20:18:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:55.407549 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b12c5-5e6f-4d40-9121-0cf3d834f566 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:55.407841 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b12c5-5e6f-4d40-9121-0cf3d834f566 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:55.408397 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b12c5-5e6f-4d40-9121-0cf3d834f566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:55.424418 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc1751-ef0f-47a6-989a-c20e7109e38b None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:55.430833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-472b12c5-5e6f-4d40-9121-0cf3d834f566 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:55.432126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-472b12c5-5e6f-4d40-9121-0cf3d834f566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:55.458481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:55.467940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eba167-c56c-4778-aa1a-c3d5126cf69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:55.469128 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eba167-c56c-4778-aa1a-c3d5126cf69f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:55.469818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eba167-c56c-4778-aa1a-c3d5126cf69f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:55.470376 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eba167-c56c-4778-aa1a-c3d5126cf69f None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:55.471400 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eba167-c56c-4778-aa1a-c3d5126cf69f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:55.473018 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eba167-c56c-4778-aa1a-c3d5126cf69f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:55.480046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 397/789] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:18:55 2022] GET /identity/v3/services => generated 772 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:55.481498 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eba167-c56c-4778-aa1a-c3d5126cf69f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:55.531862 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 393/790] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:18:55.556628 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b12c5-5e6f-4d40-9121-0cf3d834f566 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:55.613698 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eba167-c56c-4778-aa1a-c3d5126cf69f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:55.658046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75428f-40d6-4d67-84b5-8212c298ac43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:55.658908 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75428f-40d6-4d67-84b5-8212c298ac43 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:55.659186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75428f-40d6-4d67-84b5-8212c298ac43 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:55.659461 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75428f-40d6-4d67-84b5-8212c298ac43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:55.677358 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75428f-40d6-4d67-84b5-8212c298ac43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:55.678987 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75428f-40d6-4d67-84b5-8212c298ac43 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:55.697595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-3e75428f-40d6-4d67-84b5-8212c298ac43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:55.697595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:55.697595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:55.697595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:55.697595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:55.697595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:55.697595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:55.697595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:55.697595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:55.697595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:55.697595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:55.697595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:55.697595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:55.697595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:55.697595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:55.698704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:55.699722 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:55.699722 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:55.699722 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:55.699722 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:55.699722 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 398/791] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:55.707329 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75428f-40d6-4d67-84b5-8212c298ac43 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:55.709949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:55.759981 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b12c5-5e6f-4d40-9121-0cf3d834f566 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:55.760837 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:55.804613 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e036e-04b3-42cf-a245-3ff3f7080a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:55.805427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e036e-04b3-42cf-a245-3ff3f7080a6d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:55.805712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e036e-04b3-42cf-a245-3ff3f7080a6d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:55.805984 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e036e-04b3-42cf-a245-3ff3f7080a6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:55.819473 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e036e-04b3-42cf-a245-3ff3f7080a6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-126e036e-04b3-42cf-a245-3ff3f7080a6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:55.827751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:55.828802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:55.829787 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:55.829787 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:55.829787 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:55.829787 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:55.829787 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:55.829787 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:55.829787 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:55.829787 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:55.829787 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:55.829787 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:55.829787 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:55.829787 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:55.830533 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75428f-40d6-4d67-84b5-8212c298ac43 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:55.830616 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 394/792] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:18:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:55.838120 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e036e-04b3-42cf-a245-3ff3f7080a6d None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:55.872759 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ab18b-64f9-4d63-a24b-23760c3299a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:55.873690 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ab18b-64f9-4d63-a24b-23760c3299a0 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:55.873976 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ab18b-64f9-4d63-a24b-23760c3299a0 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:55.874534 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ab18b-64f9-4d63-a24b-23760c3299a0 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:55.875661 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ab18b-64f9-4d63-a24b-23760c3299a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:55.875971 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ab18b-64f9-4d63-a24b-23760c3299a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:55.877865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ab18b-64f9-4d63-a24b-23760c3299a0 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:55.894580 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 399/793] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:55.900636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ab18b-64f9-4d63-a24b-23760c3299a0 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:56.020620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e036e-04b3-42cf-a245-3ff3f7080a6d None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:56.041806 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ab18b-64f9-4d63-a24b-23760c3299a0 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:56.052575 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:56.056451 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:56.091743 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb103a6-33db-4531-8663-a56424723888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:56.092428 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb103a6-33db-4531-8663-a56424723888 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:56.092710 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb103a6-33db-4531-8663-a56424723888 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:56.092986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb103a6-33db-4531-8663-a56424723888 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:56.093747 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb103a6-33db-4531-8663-a56424723888 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:56.094056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb103a6-33db-4531-8663-a56424723888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:56.095760 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb103a6-33db-4531-8663-a56424723888 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:56.105028 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 395/794] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:56.110561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb103a6-33db-4531-8663-a56424723888 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:56.110868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f24a54-b022-4d9a-b5c9-253814fcb2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:56.111923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f24a54-b022-4d9a-b5c9-253814fcb2ef None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:56.112525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f24a54-b022-4d9a-b5c9-253814fcb2ef None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:56.113010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f24a54-b022-4d9a-b5c9-253814fcb2ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:56.128173 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f24a54-b022-4d9a-b5c9-253814fcb2ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:56.129651 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f24a54-b022-4d9a-b5c9-253814fcb2ef None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:56.157668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-06f24a54-b022-4d9a-b5c9-253814fcb2ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:56.157668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:56.157668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:56.157668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:56.157668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:56.157668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:56.157668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:56.157668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:56.157668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:56.157668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:56.157668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:56.157668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:56.157668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:56.157668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:56.157668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:56.158794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:56.159795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:56.159795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:56.159795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:56.159795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:56.159795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:56.159795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:56.159795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:56.159795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:56.160824 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 400/795] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:56.166421 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f24a54-b022-4d9a-b5c9-253814fcb2ef None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:56.292308 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb103a6-33db-4531-8663-a56424723888 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:56.366297 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f24a54-b022-4d9a-b5c9-253814fcb2ef None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:56.379564 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e53655-9acd-4a38-8878-aed2e5207e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:56.380738 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e53655-9acd-4a38-8878-aed2e5207e91 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:56.381253 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e53655-9acd-4a38-8878-aed2e5207e91 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:56.381736 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e53655-9acd-4a38-8878-aed2e5207e91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:56.403663 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858942eb-be81-40ce-920f-4489acd40c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:56.404335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858942eb-be81-40ce-920f-4489acd40c27 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:56.404615 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858942eb-be81-40ce-920f-4489acd40c27 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:56.404891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858942eb-be81-40ce-920f-4489acd40c27 None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:56.405967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858942eb-be81-40ce-920f-4489acd40c27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:56.406320 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858942eb-be81-40ce-920f-4489acd40c27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:56.407146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e53655-9acd-4a38-8878-aed2e5207e91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:56.407765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858942eb-be81-40ce-920f-4489acd40c27 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:56.409109 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e53655-9acd-4a38-8878-aed2e5207e91 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:56.439452 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 401/796] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:56.445777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-23e53655-9acd-4a38-8878-aed2e5207e91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:56.445777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:56.445777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:56.445777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:56.445777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:56.445777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:56.445777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:56.445777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:56.445777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:56.445777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:56.445777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:56.445777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:56.445777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:56.445777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:56.445777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:56.446985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:56.448019 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:56.448019 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:56.448019 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:56.448019 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:56.454724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 396/797] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:18:56.457428 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e53655-9acd-4a38-8878-aed2e5207e91 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:56.457546 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858942eb-be81-40ce-920f-4489acd40c27 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:56.492172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:56.505194 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:56.567373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e53655-9acd-4a38-8878-aed2e5207e91 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:56.583623 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858942eb-be81-40ce-920f-4489acd40c27 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:56.603071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5cff4f-353c-4fbc-b3dd-333dc8bc09e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:56.603743 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cff4f-353c-4fbc-b3dd-333dc8bc09e8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:56.604025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cff4f-353c-4fbc-b3dd-333dc8bc09e8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:56.604301 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cff4f-353c-4fbc-b3dd-333dc8bc09e8 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:56.605037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5cff4f-353c-4fbc-b3dd-333dc8bc09e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:56.605340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5cff4f-353c-4fbc-b3dd-333dc8bc09e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:56.606805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5cff4f-353c-4fbc-b3dd-333dc8bc09e8 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:56.619650 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 397/798] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:56.619826 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77258e8c-d278-407e-89ce-5a83ee80a9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:56.620557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77258e8c-d278-407e-89ce-5a83ee80a9cf None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:56.620843 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77258e8c-d278-407e-89ce-5a83ee80a9cf None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:56.621121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77258e8c-d278-407e-89ce-5a83ee80a9cf None admin] PATH_INFO: `/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/54395ae480884de38fc811a9e82c1f9e/roles/b239f941cb724331bf28dc489b70ad1d` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:56.623141 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77258e8c-d278-407e-89ce-5a83ee80a9cf None admin] RBAC: Authorizing `identity:create_grant(project_id=59555e7adac54eddb47d1ef18ef870c4, user_id=54395ae480884de38fc811a9e82c1f9e, role_id=b239f941cb724331bf28dc489b70ad1d)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:56.624829 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77258e8c-d278-407e-89ce-5a83ee80a9cf None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:56.625044 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314f2be-f5d2-46a5-8611-ea67a4ff0d51 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:56.625334 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314f2be-f5d2-46a5-8611-ea67a4ff0d51 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:56.625613 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314f2be-f5d2-46a5-8611-ea67a4ff0d51 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:56.627076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 398/799] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:56.628054 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2000cd4-5481-4903-b839-7253cccff3f8 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:56.628332 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2000cd4-5481-4903-b839-7253cccff3f8 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:56.628609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2000cd4-5481-4903-b839-7253cccff3f8 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:56.629963 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 399/800] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:56.634821 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2000cd4-5481-4903-b839-7253cccff3f8 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:56.646196 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 402/801] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:18:56 2022] PUT /identity/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/54395ae480884de38fc811a9e82c1f9e/roles/b239f941cb724331bf28dc489b70ad1d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:18:56.647567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d82bd7-48b5-4336-ad1f-e3e8fcb40447 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:56.647919 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d82bd7-48b5-4336-ad1f-e3e8fcb40447 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:56.648210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d82bd7-48b5-4336-ad1f-e3e8fcb40447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:56.676351 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-66d82bd7-48b5-4336-ad1f-e3e8fcb40447 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:56.678069 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-66d82bd7-48b5-4336-ad1f-e3e8fcb40447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:56.755673 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:56.769549 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2000cd4-5481-4903-b839-7253cccff3f8 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:56.784597 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 403/802] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:18:56.786522 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1f7cb-bfe5-4ee2-ac35-0b8b6e2e2a10 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:56.788991 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1f7cb-bfe5-4ee2-ac35-0b8b6e2e2a10 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:56.790476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1f7cb-bfe5-4ee2-ac35-0b8b6e2e2a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:56.817230 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-22c1f7cb-bfe5-4ee2-ac35-0b8b6e2e2a10 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:56.818887 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-22c1f7cb-bfe5-4ee2-ac35-0b8b6e2e2a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:56.828348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae17f6-78dc-4d04-9fe8-83d241be6e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:56.829177 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae17f6-78dc-4d04-9fe8-83d241be6e92 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:56.829464 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae17f6-78dc-4d04-9fe8-83d241be6e92 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:56.829742 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae17f6-78dc-4d04-9fe8-83d241be6e92 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:56.840681 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:56.840916 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae17f6-78dc-4d04-9fe8-83d241be6e92 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:56.842211 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae17f6-78dc-4d04-9fe8-83d241be6e92 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-ebae17f6-78dc-4d04-9fe8-83d241be6e92 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:56.862122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:56.863291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:56.864486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:56.864486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:56.864486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:56.864486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:56.864486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:56.864486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:56.864486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:56.864486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:56.864486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:18:56.864486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:56.864486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 400/803] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:18:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:56.864486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae17f6-78dc-4d04-9fe8-83d241be6e92 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:56.915853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 404/804] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:18:56.915853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1f7cb-bfe5-4ee2-ac35-0b8b6e2e2a10 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:56.990246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae17f6-78dc-4d04-9fe8-83d241be6e92 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:57.033077 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6b6a5-142d-4ba3-b5b4-cf6c3db53c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:57.034185 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6b6a5-142d-4ba3-b5b4-cf6c3db53c7c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:57.034457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6b6a5-142d-4ba3-b5b4-cf6c3db53c7c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:57.034733 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6b6a5-142d-4ba3-b5b4-cf6c3db53c7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:57.048591 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6b6a5-142d-4ba3-b5b4-cf6c3db53c7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:57.052563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1f7cb-bfe5-4ee2-ac35-0b8b6e2e2a10 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-6fc6b6a5-142d-4ba3-b5b4-cf6c3db53c7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:57.058478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:57.059609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:57.061124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:57.061124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:57.061124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:57.061124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:57.061124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:57.061124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:57.061124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:57.061124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:57.061124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:57.061124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:57.061124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:18:57.061124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:57.061124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 401/805] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:18:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:57.062000 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6b6a5-142d-4ba3-b5b4-cf6c3db53c7c None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:57.077114 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:57.093206 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108a9ac-34e2-4bee-aec1-c33ada37c4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:57.094303 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108a9ac-34e2-4bee-aec1-c33ada37c4e9 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:57.094797 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108a9ac-34e2-4bee-aec1-c33ada37c4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:57.095396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108a9ac-34e2-4bee-aec1-c33ada37c4e9 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:57.096543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108a9ac-34e2-4bee-aec1-c33ada37c4e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:57.097058 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108a9ac-34e2-4bee-aec1-c33ada37c4e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:57.098987 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108a9ac-34e2-4bee-aec1-c33ada37c4e9 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:57.125904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 405/806] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:18:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:57.127176 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108a9ac-34e2-4bee-aec1-c33ada37c4e9 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:57.132020 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:57.199097 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6b6a5-142d-4ba3-b5b4-cf6c3db53c7c None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:57.235927 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108a9ac-34e2-4bee-aec1-c33ada37c4e9 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:57.243750 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0d173-921f-4d00-a6f6-161b153a3587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:57.244799 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0d173-921f-4d00-a6f6-161b153a3587 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:57.244920 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0d173-921f-4d00-a6f6-161b153a3587 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:57.245158 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0d173-921f-4d00-a6f6-161b153a3587 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:57.245819 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0d173-921f-4d00-a6f6-161b153a3587 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:57.247557 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0d173-921f-4d00-a6f6-161b153a3587 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-0ac0d173-921f-4d00-a6f6-161b153a3587 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:57.260638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:57.261716 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:57.262689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 402/807] 10.222.0.84 () {56 vars in 1129 bytes} [Thu May 19 20:18:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:57.263665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0d173-921f-4d00-a6f6-161b153a3587 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:57.279235 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416997e8-2ca7-4f7d-83cb-019d551416c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:57.280286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416997e8-2ca7-4f7d-83cb-019d551416c2 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:57.280777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416997e8-2ca7-4f7d-83cb-019d551416c2 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:57.281330 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416997e8-2ca7-4f7d-83cb-019d551416c2 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:57.282333 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416997e8-2ca7-4f7d-83cb-019d551416c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:57.282870 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416997e8-2ca7-4f7d-83cb-019d551416c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:57.284478 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416997e8-2ca7-4f7d-83cb-019d551416c2 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:57.293951 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 406/808] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:18:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:57.295151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416997e8-2ca7-4f7d-83cb-019d551416c2 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:57.362191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:57.362605 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:57.370846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0d173-921f-4d00-a6f6-161b153a3587 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:57.404374 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416997e8-2ca7-4f7d-83cb-019d551416c2 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:57.409955 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e0421-c06f-4275-ae82-81e3fd5d6b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:57.410785 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e0421-c06f-4275-ae82-81e3fd5d6b5b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:57.411151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e0421-c06f-4275-ae82-81e3fd5d6b5b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:57.411493 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e0421-c06f-4275-ae82-81e3fd5d6b5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:57.444795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bf617-688d-44dd-b4df-c387f10ddccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:57.445862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bf617-688d-44dd-b4df-c387f10ddccf None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:57.446364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bf617-688d-44dd-b4df-c387f10ddccf None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:57.446853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bf617-688d-44dd-b4df-c387f10ddccf None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:57.447747 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bf617-688d-44dd-b4df-c387f10ddccf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:57.448237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bf617-688d-44dd-b4df-c387f10ddccf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:57.449858 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bf617-688d-44dd-b4df-c387f10ddccf None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:57.455651 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 407/809] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 20:18:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:57.456809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bf617-688d-44dd-b4df-c387f10ddccf None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:57.509938 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e0421-c06f-4275-ae82-81e3fd5d6b5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:57.511746 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e0421-c06f-4275-ae82-81e3fd5d6b5b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:57.568022 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bf617-688d-44dd-b4df-c387f10ddccf None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:57.599076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:57.605543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb47e15-1f8b-4dd4-80ab-906dc2d212f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:57.606465 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb47e15-1f8b-4dd4-80ab-906dc2d212f7 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:57.606833 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb47e15-1f8b-4dd4-80ab-906dc2d212f7 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:57.607174 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb47e15-1f8b-4dd4-80ab-906dc2d212f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:57.609159 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:57.622245 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb47e15-1f8b-4dd4-80ab-906dc2d212f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:57.624407 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb47e15-1f8b-4dd4-80ab-906dc2d212f7 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:57.636414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 403/810] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:18:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:18:57.637283 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e0421-c06f-4275-ae82-81e3fd5d6b5b None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:57.644179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-ceb47e15-1f8b-4dd4-80ab-906dc2d212f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:57.644179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:57.644179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:57.644179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:57.644179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:57.644179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:57.644179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:57.644179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:57.644179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:57.644179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:57.644179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:57.644179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:57.644179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:57.644179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:57.644179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:57.645170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:57.646163 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:57.646163 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:57.646163 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:57.646163 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:57.649749 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 408/811] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:57.650903 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb47e15-1f8b-4dd4-80ab-906dc2d212f7 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:57.741057 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e0421-c06f-4275-ae82-81e3fd5d6b5b None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:57.758174 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb47e15-1f8b-4dd4-80ab-906dc2d212f7 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:57.777337 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f218616-46a1-4e7b-825e-d901c5a032f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:57.777995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f218616-46a1-4e7b-825e-d901c5a032f7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:57.778289 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f218616-46a1-4e7b-825e-d901c5a032f7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:57.778586 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f218616-46a1-4e7b-825e-d901c5a032f7 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:57.779290 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f218616-46a1-4e7b-825e-d901c5a032f7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:57.779594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f218616-46a1-4e7b-825e-d901c5a032f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:57.780979 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f218616-46a1-4e7b-825e-d901c5a032f7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:57.786705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 404/812] 10.222.0.84 () {56 vars in 1124 bytes} [Thu May 19 20:18:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:57.787785 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f218616-46a1-4e7b-825e-d901c5a032f7 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:57.802202 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe6905-de0a-4285-b259-37e38dbdeeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:57.803513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe6905-de0a-4285-b259-37e38dbdeeae None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:57.804088 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe6905-de0a-4285-b259-37e38dbdeeae None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:57.804625 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe6905-de0a-4285-b259-37e38dbdeeae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:57.816870 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe6905-de0a-4285-b259-37e38dbdeeae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:57.818627 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe6905-de0a-4285-b259-37e38dbdeeae None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:57.829847 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:57.840803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-18fe6905-de0a-4285-b259-37e38dbdeeae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:57.840803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:57.840803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:57.840803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:57.840803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:57.840803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:57.840803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:57.840803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:57.840803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:57.840803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:57.840803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:57.840803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:57.840803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:57.840803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:57.840803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:57.841879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:57.842956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:57.842956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:57.842956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:57.842956 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:57.845125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 409/813] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:57.846378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe6905-de0a-4285-b259-37e38dbdeeae None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:57.882406 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:57.901229 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f218616-46a1-4e7b-825e-d901c5a032f7 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:57.942101 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e7df2-77ab-4bc3-a51e-7f5da3b93a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:57.942869 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e7df2-77ab-4bc3-a51e-7f5da3b93a40 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:57.943153 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e7df2-77ab-4bc3-a51e-7f5da3b93a40 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:57.943477 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e7df2-77ab-4bc3-a51e-7f5da3b93a40 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:57.944295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e7df2-77ab-4bc3-a51e-7f5da3b93a40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:57.944746 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e7df2-77ab-4bc3-a51e-7f5da3b93a40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:57.946370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e7df2-77ab-4bc3-a51e-7f5da3b93a40 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:57.959779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 405/814] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:57.960662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e7df2-77ab-4bc3-a51e-7f5da3b93a40 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:57.962693 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe6905-de0a-4285-b259-37e38dbdeeae None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:58.000016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385e023-4bf1-4258-88c1-259b75783f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:58.001052 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385e023-4bf1-4258-88c1-259b75783f7c None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:58.001562 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385e023-4bf1-4258-88c1-259b75783f7c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:58.002039 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385e023-4bf1-4258-88c1-259b75783f7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:58.003014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385e023-4bf1-4258-88c1-259b75783f7c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:58.004641 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385e023-4bf1-4258-88c1-259b75783f7c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:58.038649 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 410/815] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:18:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:18:58.040043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385e023-4bf1-4258-88c1-259b75783f7c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:58.066932 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:58.074294 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e7df2-77ab-4bc3-a51e-7f5da3b93a40 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:58.121211 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f17b6-78b4-4ec2-b915-eecfe2aafb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:58.121890 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f17b6-78b4-4ec2-b915-eecfe2aafb4b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:58.122183 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f17b6-78b4-4ec2-b915-eecfe2aafb4b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:58.122474 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f17b6-78b4-4ec2-b915-eecfe2aafb4b None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:58.123200 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f17b6-78b4-4ec2-b915-eecfe2aafb4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:58.123507 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f17b6-78b4-4ec2-b915-eecfe2aafb4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:58.125032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f17b6-78b4-4ec2-b915-eecfe2aafb4b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:58.139276 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 406/816] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:58.141110 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1680bb-36d7-44c5-8929-49c8584d1364 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:58.141495 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1680bb-36d7-44c5-8929-49c8584d1364 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:58.141890 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1680bb-36d7-44c5-8929-49c8584d1364 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:58.143741 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 407/817] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:58.146044 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1680bb-36d7-44c5-8929-49c8584d1364 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:58.196241 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:58.210170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385e023-4bf1-4258-88c1-259b75783f7c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:58.266054 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23976ce-9260-4268-a54d-60587260e120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:58.267034 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23976ce-9260-4268-a54d-60587260e120 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:58.267322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23976ce-9260-4268-a54d-60587260e120 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:58.267602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23976ce-9260-4268-a54d-60587260e120 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:58.269456 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1680bb-36d7-44c5-8929-49c8584d1364 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:58.278716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23976ce-9260-4268-a54d-60587260e120 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:58.280145 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23976ce-9260-4268-a54d-60587260e120 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-e23976ce-9260-4268-a54d-60587260e120 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:18:58.299129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:58.300216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:58.301257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:58.301257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:58.301257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:18:58.301257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:18:58.301257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:18:58.301257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:18:58.301257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:18:58.301257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:18:58.301257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:18:58.301257 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:58.302569 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 411/818] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:18:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:58.302823 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726899c9-a19e-4000-9188-8b24cc4025ec None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:58.303109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726899c9-a19e-4000-9188-8b24cc4025ec None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:58.303385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726899c9-a19e-4000-9188-8b24cc4025ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:58.315329 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb920b-4955-49a6-9f63-ce31b464fbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:58.316500 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb920b-4955-49a6-9f63-ce31b464fbc1 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:58.316998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb920b-4955-49a6-9f63-ce31b464fbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:58.317470 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb920b-4955-49a6-9f63-ce31b464fbc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:58.322087 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-726899c9-a19e-4000-9188-8b24cc4025ec None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:58.323311 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-726899c9-a19e-4000-9188-8b24cc4025ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:58.325756 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb920b-4955-49a6-9f63-ce31b464fbc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:58.327514 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb920b-4955-49a6-9f63-ce31b464fbc1 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:58.343639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-94cb920b-4955-49a6-9f63-ce31b464fbc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:58.343639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:58.343639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:58.343639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:58.343639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:58.343639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:58.343639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:58.343639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:58.343639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:58.343639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:58.343639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:58.343639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:58.343639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:58.343639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:58.343639 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:58.344698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:58.346120 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:58.346120 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:58.346120 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:58.346120 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:58.346120 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:58.346120 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:58.346120 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:58.346120 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:58.346120 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 408/819] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:58.346120 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb920b-4955-49a6-9f63-ce31b464fbc1 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:58.368750 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:58.406687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 412/820] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:18:58.407564 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726899c9-a19e-4000-9188-8b24cc4025ec None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:58.468182 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb920b-4955-49a6-9f63-ce31b464fbc1 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:58.505167 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1ae09-63ab-49f4-93dd-4c50b486f145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:58.506152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1ae09-63ab-49f4-93dd-4c50b486f145 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:58.506625 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1ae09-63ab-49f4-93dd-4c50b486f145 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:58.507059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1ae09-63ab-49f4-93dd-4c50b486f145 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:58.508554 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1ae09-63ab-49f4-93dd-4c50b486f145 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:58.509029 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1ae09-63ab-49f4-93dd-4c50b486f145 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:58.510720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1ae09-63ab-49f4-93dd-4c50b486f145 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:58.526432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:58.537034 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 409/821] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 20:18:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:58.540898 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1ae09-63ab-49f4-93dd-4c50b486f145 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:58.551243 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726899c9-a19e-4000-9188-8b24cc4025ec None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:58.589515 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30cf2b-3b88-4467-b4fd-ea4d96e9bf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:58.589995 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30cf2b-3b88-4467-b4fd-ea4d96e9bf17 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:58.590290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30cf2b-3b88-4467-b4fd-ea4d96e9bf17 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:58.590582 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30cf2b-3b88-4467-b4fd-ea4d96e9bf17 None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:58.591365 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee30cf2b-3b88-4467-b4fd-ea4d96e9bf17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:58.591669 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee30cf2b-3b88-4467-b4fd-ea4d96e9bf17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:58.593275 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee30cf2b-3b88-4467-b4fd-ea4d96e9bf17 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:58.610937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 413/822] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:58.611855 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30cf2b-3b88-4467-b4fd-ea4d96e9bf17 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:58.623022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:58.660525 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1ae09-63ab-49f4-93dd-4c50b486f145 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:58.696770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17744ec-bc31-4b77-9ef0-61666e073c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:58.697712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17744ec-bc31-4b77-9ef0-61666e073c19 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:58.698111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17744ec-bc31-4b77-9ef0-61666e073c19 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:58.698501 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17744ec-bc31-4b77-9ef0-61666e073c19 None admin] PATH_INFO: `/v3/services/network` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:58.699307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17744ec-bc31-4b77-9ef0-61666e073c19 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:58.700816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17744ec-bc31-4b77-9ef0-61666e073c19 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-b17744ec-bc31-4b77-9ef0-61666e073c19 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:58.713405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:58.715256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:58.716656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:58.716656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:58.716656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:58.716656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:58.716656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:58.716656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:58.716656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:58.716656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:58.716656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:58.716656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:58.716656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:18:58.716656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:18:58.716656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:18:58.716656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:58.722948 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 410/823] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:58 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:58.722948 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17744ec-bc31-4b77-9ef0-61666e073c19 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:58.734685 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30cf2b-3b88-4467-b4fd-ea4d96e9bf17 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:58.777802 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d0a4b-6936-4565-80ed-cdb146963a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:58.779061 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d0a4b-6936-4565-80ed-cdb146963a26 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:58.779586 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d0a4b-6936-4565-80ed-cdb146963a26 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:58.780149 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d0a4b-6936-4565-80ed-cdb146963a26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:58.794656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d0a4b-6936-4565-80ed-cdb146963a26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:58.796448 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d0a4b-6936-4565-80ed-cdb146963a26 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:58.806697 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:58.821890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-9f6d0a4b-6936-4565-80ed-cdb146963a26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:58.821890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:58.821890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:58.821890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:58.821890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:58.821890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:58.821890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:58.821890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:58.821890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:58.821890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:58.821890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:58.821890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:58.821890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:18:58.821890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:18:58.821890 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:58.823081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:58.824107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:18:58.824107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:18:58.824107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:18:58.824107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:58.827150 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 414/824] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:58.828431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d0a4b-6936-4565-80ed-cdb146963a26 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:58.833033 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17744ec-bc31-4b77-9ef0-61666e073c19 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:58.874009 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d4fc8-56ec-49a5-aee3-fbee4f610716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:58.874822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d4fc8-56ec-49a5-aee3-fbee4f610716 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:58.875194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d4fc8-56ec-49a5-aee3-fbee4f610716 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:58.875594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d4fc8-56ec-49a5-aee3-fbee4f610716 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:58.876524 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d4fc8-56ec-49a5-aee3-fbee4f610716 None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=edb1bcb781084f03a4ad2611a09bd77e) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:58.876933 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d4fc8-56ec-49a5-aee3-fbee4f610716 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:58.878504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d4fc8-56ec-49a5-aee3-fbee4f610716 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:58.882050 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:58.885761 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 411/825] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:18:58 2022] GET /identity/v3/role_assignments?user.id=edb1bcb781084f03a4ad2611a09bd77e&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=b239f941cb724331bf28dc489b70ad1d => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:58.886958 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d4fc8-56ec-49a5-aee3-fbee4f610716 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:58.968135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d0a4b-6936-4565-80ed-cdb146963a26 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:59.001840 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d4fc8-56ec-49a5-aee3-fbee4f610716 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:59.017732 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a6ed8-0b5a-4b42-a766-aa80b2f332bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:59.019623 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a6ed8-0b5a-4b42-a766-aa80b2f332bc None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:59.020167 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a6ed8-0b5a-4b42-a766-aa80b2f332bc None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:59.020938 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a6ed8-0b5a-4b42-a766-aa80b2f332bc None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:59.021898 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a6ed8-0b5a-4b42-a766-aa80b2f332bc None admin] RBAC: Adding query filter params (name=network) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:59.022706 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a6ed8-0b5a-4b42-a766-aa80b2f332bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:59.025212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a6ed8-0b5a-4b42-a766-aa80b2f332bc None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:59.033030 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 415/826] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:18:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:59.039779 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea79543-3cd3-437c-b772-1073927e3f78 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:59.040936 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea79543-3cd3-437c-b772-1073927e3f78 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:59.041213 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea79543-3cd3-437c-b772-1073927e3f78 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:59.042930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 416/827] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:18:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:18:59.043849 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea79543-3cd3-437c-b772-1073927e3f78 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:59.050613 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6925d7d-43a8-4207-80fb-a50028d8ae4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:59.052826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6925d7d-43a8-4207-80fb-a50028d8ae4c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:59.053361 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6925d7d-43a8-4207-80fb-a50028d8ae4c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:59.053849 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6925d7d-43a8-4207-80fb-a50028d8ae4c None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:59.055131 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6925d7d-43a8-4207-80fb-a50028d8ae4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:59.055751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6925d7d-43a8-4207-80fb-a50028d8ae4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:59.057449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6925d7d-43a8-4207-80fb-a50028d8ae4c None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:59.076483 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 412/828] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:59.078421 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab927327-ae54-4b01-a3c4-ed0ced0bc951 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:59.079165 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab927327-ae54-4b01-a3c4-ed0ced0bc951 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:59.079962 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab927327-ae54-4b01-a3c4-ed0ced0bc951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:59.107615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-ab927327-ae54-4b01-a3c4-ed0ced0bc951 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:18:59.109714 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-ab927327-ae54-4b01-a3c4-ed0ced0bc951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:18:59.132290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:59.180073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:59.192624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea79543-3cd3-437c-b772-1073927e3f78 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:59.198292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 413/829] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:18:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:18:59.198991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab927327-ae54-4b01-a3c4-ed0ced0bc951 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:59.244985 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cfd70-2cf6-483e-a042-564a8fea1d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:59.245717 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cfd70-2cf6-483e-a042-564a8fea1d71 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:59.246008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cfd70-2cf6-483e-a042-564a8fea1d71 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:59.246319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cfd70-2cf6-483e-a042-564a8fea1d71 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:59.247158 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cfd70-2cf6-483e-a042-564a8fea1d71 None admin] RBAC: Adding query filter params (type=network) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:59.247472 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cfd70-2cf6-483e-a042-564a8fea1d71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:59.248965 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cfd70-2cf6-483e-a042-564a8fea1d71 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:59.255286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 417/830] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:18:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:59.256099 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cfd70-2cf6-483e-a042-564a8fea1d71 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:59.350108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab927327-ae54-4b01-a3c4-ed0ced0bc951 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:59.380971 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cfd70-2cf6-483e-a042-564a8fea1d71 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:59.396899 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6b17a-8b27-4333-bc1e-0594cd4ac312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:59.397728 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6b17a-8b27-4333-bc1e-0594cd4ac312 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:59.398086 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6b17a-8b27-4333-bc1e-0594cd4ac312 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:59.398578 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6b17a-8b27-4333-bc1e-0594cd4ac312 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:59.407051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6b17a-8b27-4333-bc1e-0594cd4ac312 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:59.407728 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:59.409297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6b17a-8b27-4333-bc1e-0594cd4ac312 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:59.434132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-a7b6b17a-8b27-4333-bc1e-0594cd4ac312 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:59.434132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:59.434132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:59.434132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:59.434132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:59.434132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:59.434132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:59.434132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:59.434132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:59.434132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:59.434132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:59.434132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:59.434132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:18:59.434132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:18:59.434132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:59.435191 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:59.436897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:18:59.436897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:18:59.436897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:18:59.436897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:18:59.436897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:18:59.436897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:18:59.436897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:18:59.436897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:18:59.436897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 414/831] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:18:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:59.436897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6b17a-8b27-4333-bc1e-0594cd4ac312 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:59.444774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3289c15-06d3-44bd-b6df-746ba9956e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:59.446932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3289c15-06d3-44bd-b6df-746ba9956e4c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:59.447453 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3289c15-06d3-44bd-b6df-746ba9956e4c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:59.447960 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3289c15-06d3-44bd-b6df-746ba9956e4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:59.463194 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3289c15-06d3-44bd-b6df-746ba9956e4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-a3289c15-06d3-44bd-b6df-746ba9956e4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:18:59.475236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:18:59.476511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:18:59.477619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:18:59.477619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:18:59.477619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:18:59.477619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:18:59.477619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:18:59.477619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:18:59.477619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:18:59.477619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:18:59.477619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:18:59.477619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:18:59.477619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:18:59.477619 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:18:59.479803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 418/832] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:18:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:18:59.481306 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3289c15-06d3-44bd-b6df-746ba9956e4c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:59.502308 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:59.558293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6b17a-8b27-4333-bc1e-0594cd4ac312 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:59.594959 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c55f8e-b410-4ac3-8aed-48ab224fad3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:59.595726 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c55f8e-b410-4ac3-8aed-48ab224fad3f None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:59.595940 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c55f8e-b410-4ac3-8aed-48ab224fad3f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:59.596162 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c55f8e-b410-4ac3-8aed-48ab224fad3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:59.596908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c55f8e-b410-4ac3-8aed-48ab224fad3f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:59.598316 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c55f8e-b410-4ac3-8aed-48ab224fad3f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:59.624742 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3289c15-06d3-44bd-b6df-746ba9956e4c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:59.625162 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 415/833] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:18:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:18:59.626073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c55f8e-b410-4ac3-8aed-48ab224fad3f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:59.676630 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308e5eb-7389-4605-803e-10daf3b723de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:59.677740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308e5eb-7389-4605-803e-10daf3b723de None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:59.678246 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308e5eb-7389-4605-803e-10daf3b723de None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:59.678906 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308e5eb-7389-4605-803e-10daf3b723de None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:59.680058 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308e5eb-7389-4605-803e-10daf3b723de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:59.680622 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308e5eb-7389-4605-803e-10daf3b723de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:59.682445 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308e5eb-7389-4605-803e-10daf3b723de None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:59.702240 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 419/834] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:18:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:59.704973 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308e5eb-7389-4605-803e-10daf3b723de None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:59.745878 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:59.774011 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c55f8e-b410-4ac3-8aed-48ab224fad3f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:59.785373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:18:59.819276 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57986a0-05d5-44a6-a723-9da74bfd4be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:59.820941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57986a0-05d5-44a6-a723-9da74bfd4be3 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:59.821411 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57986a0-05d5-44a6-a723-9da74bfd4be3 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:59.821845 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57986a0-05d5-44a6-a723-9da74bfd4be3 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:59.822819 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57986a0-05d5-44a6-a723-9da74bfd4be3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:18:59.823293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57986a0-05d5-44a6-a723-9da74bfd4be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:59.824891 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57986a0-05d5-44a6-a723-9da74bfd4be3 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:18:59.837282 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 416/835] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:18:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:18:59.843663 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57986a0-05d5-44a6-a723-9da74bfd4be3 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:18:59.919376 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308e5eb-7389-4605-803e-10daf3b723de None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:59.966090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcecf18-57f9-4092-8fdc-b62cd928b7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:18:59.966687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcecf18-57f9-4092-8fdc-b62cd928b7ce None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:18:59.966964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcecf18-57f9-4092-8fdc-b62cd928b7ce None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:18:59.967236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcecf18-57f9-4092-8fdc-b62cd928b7ce None admin] PATH_INFO: `/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/265a6cde297448adb102da0773e366fb/roles/c9206a4a0d49473f87a45d6c8abb2e51` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:18:59.970301 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57986a0-05d5-44a6-a723-9da74bfd4be3 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:18:59.989840 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcecf18-57f9-4092-8fdc-b62cd928b7ce None admin] RBAC: Authorizing `identity:create_grant(project_id=59555e7adac54eddb47d1ef18ef870c4, user_id=265a6cde297448adb102da0773e366fb, role_id=c9206a4a0d49473f87a45d6c8abb2e51)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:18:59.992466 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcecf18-57f9-4092-8fdc-b62cd928b7ce None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:00.014244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 420/836] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:18:59 2022] PUT /identity/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/265a6cde297448adb102da0773e366fb/roles/c9206a4a0d49473f87a45d6c8abb2e51 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:19:00.018494 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07a7a9-454c-436d-924e-053f677510c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:00.019591 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07a7a9-454c-436d-924e-053f677510c8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:00.020058 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07a7a9-454c-436d-924e-053f677510c8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:00.020498 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07a7a9-454c-436d-924e-053f677510c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:00.034217 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07a7a9-454c-436d-924e-053f677510c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:00.036130 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07a7a9-454c-436d-924e-053f677510c8 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:00.056762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-2b07a7a9-454c-436d-924e-053f677510c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:00.056762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:00.056762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:00.056762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:00.056762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:00.056762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:00.056762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:00.056762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:00.056762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:00.056762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:00.056762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:00.056762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:00.056762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:00.056762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:00.056762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:00.057882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:00.059803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:00.059803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:00.059803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:00.059803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:00.062095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 417/837] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:18:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:00.067515 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcecf18-57f9-4092-8fdc-b62cd928b7ce None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:00.180391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:00.204112 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcecf18-57f9-4092-8fdc-b62cd928b7ce None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:00.249326 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb4202-ba4c-47f7-ab5d-a319c96ee49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:00.250015 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb4202-ba4c-47f7-ab5d-a319c96ee49f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:00.250328 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb4202-ba4c-47f7-ab5d-a319c96ee49f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:00.250610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb4202-ba4c-47f7-ab5d-a319c96ee49f None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:00.251342 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb4202-ba4c-47f7-ab5d-a319c96ee49f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:00.251656 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb4202-ba4c-47f7-ab5d-a319c96ee49f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:00.253091 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb4202-ba4c-47f7-ab5d-a319c96ee49f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:00.266116 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 421/838] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:00.271699 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07a7a9-454c-436d-924e-053f677510c8 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:00.298140 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:00.381738 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07a7a9-454c-436d-924e-053f677510c8 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:00.432192 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf5bcd-b287-44e4-8c5b-0f3d4d0b6215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:00.432972 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf5bcd-b287-44e4-8c5b-0f3d4d0b6215 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:00.433251 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf5bcd-b287-44e4-8c5b-0f3d4d0b6215 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:00.433521 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf5bcd-b287-44e4-8c5b-0f3d4d0b6215 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:00.441740 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf5bcd-b287-44e4-8c5b-0f3d4d0b6215 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:00.443263 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf5bcd-b287-44e4-8c5b-0f3d4d0b6215 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-5abf5bcd-b287-44e4-8c5b-0f3d4d0b6215 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:19:00.458757 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:00.459949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:00.462051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:00.462051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:00.462051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:19:00.462051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:19:00.462051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:19:00.462051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:19:00.462051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:19:00.462051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:19:00.462051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:19:00.462051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:00.462051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 418/839] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:19:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:00.466085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb4202-ba4c-47f7-ab5d-a319c96ee49f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:00.572452 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb4202-ba4c-47f7-ab5d-a319c96ee49f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:00.589969 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468c319-866a-448e-882c-14fef4a06928 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:00.590332 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468c319-866a-448e-882c-14fef4a06928 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:00.590752 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468c319-866a-448e-882c-14fef4a06928 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:00.592391 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 419/840] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:19:00.597366 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e30b6-04ce-4526-abe2-b2ee5f6f6532 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:00.597695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e30b6-04ce-4526-abe2-b2ee5f6f6532 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:00.597921 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e30b6-04ce-4526-abe2-b2ee5f6f6532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:00.609605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8236fe-6895-48df-a8aa-35a233c20c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:00.610310 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8236fe-6895-48df-a8aa-35a233c20c6d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:00.610598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8236fe-6895-48df-a8aa-35a233c20c6d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:00.610874 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8236fe-6895-48df-a8aa-35a233c20c6d None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:00.611710 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8236fe-6895-48df-a8aa-35a233c20c6d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:00.612022 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8236fe-6895-48df-a8aa-35a233c20c6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:00.613457 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8236fe-6895-48df-a8aa-35a233c20c6d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:00.615922 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-af6e30b6-04ce-4526-abe2-b2ee5f6f6532 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:00.617136 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-af6e30b6-04ce-4526-abe2-b2ee5f6f6532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:00.633096 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 422/841] 10.222.0.84 () {56 vars in 1135 bytes} [Thu May 19 20:19:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:00.638717 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8236fe-6895-48df-a8aa-35a233c20c6d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:00.642654 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:00.694493 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 420/842] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:00.709510 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e30b6-04ce-4526-abe2-b2ee5f6f6532 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:00.721934 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:00.748450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8236fe-6895-48df-a8aa-35a233c20c6d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:00.783882 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39a055-f2a8-48f3-a38d-50b505363614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:00.784547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39a055-f2a8-48f3-a38d-50b505363614 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:00.784830 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39a055-f2a8-48f3-a38d-50b505363614 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:00.785105 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39a055-f2a8-48f3-a38d-50b505363614 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:00.812169 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e30b6-04ce-4526-abe2-b2ee5f6f6532 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:00.847493 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaed0d3-5327-40a3-8f5e-8e077b4be40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:00.848459 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaed0d3-5327-40a3-8f5e-8e077b4be40f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:00.848808 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaed0d3-5327-40a3-8f5e-8e077b4be40f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:00.849153 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaed0d3-5327-40a3-8f5e-8e077b4be40f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:00.849874 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaed0d3-5327-40a3-8f5e-8e077b4be40f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:00.852210 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaed0d3-5327-40a3-8f5e-8e077b4be40f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:00.857293 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b39a055-f2a8-48f3-a38d-50b505363614 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:00.858792 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b39a055-f2a8-48f3-a38d-50b505363614 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:00.861581 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-dcaed0d3-5327-40a3-8f5e-8e077b4be40f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:00.864870 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:00.866406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:00.867730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:00.867730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:00.867730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:00.867730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:00.867730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:00.867730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:00.867730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:00.867730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:00.867730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:00.867730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:00.867730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:00.867730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:00.867730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:19:00.867730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:00.872357 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 421/843] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:19:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:00.873369 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaed0d3-5327-40a3-8f5e-8e077b4be40f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:00.954118 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:00.974453 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 423/844] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:19:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:19:00.976046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39a055-f2a8-48f3-a38d-50b505363614 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:00.981776 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaed0d3-5327-40a3-8f5e-8e077b4be40f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:01.017497 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e86898-9ba1-46e2-a06e-7541cfe673f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:01.018373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e86898-9ba1-46e2-a06e-7541cfe673f7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:01.018755 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e86898-9ba1-46e2-a06e-7541cfe673f7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:01.019126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e86898-9ba1-46e2-a06e-7541cfe673f7 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:01.019928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e86898-9ba1-46e2-a06e-7541cfe673f7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:01.020330 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e86898-9ba1-46e2-a06e-7541cfe673f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:01.021802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e86898-9ba1-46e2-a06e-7541cfe673f7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:01.031502 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 422/845] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:19:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:01.032486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e86898-9ba1-46e2-a06e-7541cfe673f7 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:01.087647 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39a055-f2a8-48f3-a38d-50b505363614 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:01.102678 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:01.140434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68281af7-5168-48a0-96ce-aefb6ad6b59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:01.141644 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68281af7-5168-48a0-96ce-aefb6ad6b59c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:01.141925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68281af7-5168-48a0-96ce-aefb6ad6b59c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:01.142193 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68281af7-5168-48a0-96ce-aefb6ad6b59c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:01.145358 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e86898-9ba1-46e2-a06e-7541cfe673f7 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:01.163391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68281af7-5168-48a0-96ce-aefb6ad6b59c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:01.163391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68281af7-5168-48a0-96ce-aefb6ad6b59c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:01.188897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-68281af7-5168-48a0-96ce-aefb6ad6b59c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:01.188897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:01.188897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:01.188897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:01.188897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:01.188897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:01.188897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:01.188897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:01.188897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:01.188897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:01.188897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:01.188897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:01.188897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:01.188897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:01.188897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:01.190115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:01.196654 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:01.196654 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:01.196654 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:01.196654 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:01.196654 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 424/846] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:01.196654 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68281af7-5168-48a0-96ce-aefb6ad6b59c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:01.200601 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34741114-09d0-4caa-a14d-b19ed10539fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:01.202283 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34741114-09d0-4caa-a14d-b19ed10539fd None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:01.202584 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34741114-09d0-4caa-a14d-b19ed10539fd None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:01.202871 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34741114-09d0-4caa-a14d-b19ed10539fd None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:01.203600 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34741114-09d0-4caa-a14d-b19ed10539fd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:01.203911 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34741114-09d0-4caa-a14d-b19ed10539fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:01.205382 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34741114-09d0-4caa-a14d-b19ed10539fd None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:01.218363 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 423/847] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:19:01 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:01.326297 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68281af7-5168-48a0-96ce-aefb6ad6b59c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:01.371362 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5050f26-d379-418e-af59-f4c2a27a7df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:01.372068 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5050f26-d379-418e-af59-f4c2a27a7df6 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:01.372352 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5050f26-d379-418e-af59-f4c2a27a7df6 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:01.372633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5050f26-d379-418e-af59-f4c2a27a7df6 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:01.373443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5050f26-d379-418e-af59-f4c2a27a7df6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:01.373754 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5050f26-d379-418e-af59-f4c2a27a7df6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:01.375240 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5050f26-d379-418e-af59-f4c2a27a7df6 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:01.389073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 425/848] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:01.394868 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34741114-09d0-4caa-a14d-b19ed10539fd None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:01.398669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:01.505587 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7606e-9243-48a1-b3c2-94f25064c3b7 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:01.505798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7606e-9243-48a1-b3c2-94f25064c3b7 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:01.506080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7606e-9243-48a1-b3c2-94f25064c3b7 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:01.508420 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34741114-09d0-4caa-a14d-b19ed10539fd None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:01.508883 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 426/849] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:19:01.512897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f4e1b-31e1-481e-ba02-f4385b3ee085 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:01.513183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f4e1b-31e1-481e-ba02-f4385b3ee085 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:01.513464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f4e1b-31e1-481e-ba02-f4385b3ee085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:01.530346 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-3b6f4e1b-31e1-481e-ba02-f4385b3ee085 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:01.531518 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-3b6f4e1b-31e1-481e-ba02-f4385b3ee085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:01.545694 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:01.551543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435e984-36df-406c-8cf3-61ed5d4dfb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:01.552259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435e984-36df-406c-8cf3-61ed5d4dfb4d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:01.552539 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435e984-36df-406c-8cf3-61ed5d4dfb4d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:01.552813 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435e984-36df-406c-8cf3-61ed5d4dfb4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:01.560204 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435e984-36df-406c-8cf3-61ed5d4dfb4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:01.561673 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435e984-36df-406c-8cf3-61ed5d4dfb4d None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:01.577437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-9435e984-36df-406c-8cf3-61ed5d4dfb4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:01.577437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:01.577437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:01.577437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:01.577437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:01.577437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:01.577437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:01.577437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:01.577437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:01.577437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:01.577437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:01.577437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:01.577437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:19:01.577437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:19:01.577437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:01.578571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:01.579659 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:19:01.579659 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:19:01.579659 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:19:01.579659 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:19:01.579659 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:19:01.579659 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:19:01.579659 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:01.579659 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:01.583478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 424/850] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:19:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:01.584851 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435e984-36df-406c-8cf3-61ed5d4dfb4d None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:01.627380 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 427/851] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:01.650430 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f4e1b-31e1-481e-ba02-f4385b3ee085 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:01.655583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:01.706564 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435e984-36df-406c-8cf3-61ed5d4dfb4d None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:01.756237 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe8550-e6d0-4840-bc54-aefe7cd1843f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:01.756911 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe8550-e6d0-4840-bc54-aefe7cd1843f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:01.757334 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe8550-e6d0-4840-bc54-aefe7cd1843f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:01.757643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe8550-e6d0-4840-bc54-aefe7cd1843f None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:01.758546 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe8550-e6d0-4840-bc54-aefe7cd1843f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:01.758925 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe8550-e6d0-4840-bc54-aefe7cd1843f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:01.760676 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe8550-e6d0-4840-bc54-aefe7cd1843f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:01.770869 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f4e1b-31e1-481e-ba02-f4385b3ee085 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:01.778955 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 425/852] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:19:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:01.780305 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe8550-e6d0-4840-bc54-aefe7cd1843f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:01.816420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d9817-e2f7-492c-8817-ed87bf04c68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:01.818554 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d9817-e2f7-492c-8817-ed87bf04c68a None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:01.818919 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d9817-e2f7-492c-8817-ed87bf04c68a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:01.819268 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d9817-e2f7-492c-8817-ed87bf04c68a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:01.820300 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d9817-e2f7-492c-8817-ed87bf04c68a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:01.821817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d9817-e2f7-492c-8817-ed87bf04c68a None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-2a5d9817-e2f7-492c-8817-ed87bf04c68a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:01.835279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:01.836852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:01.837879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 428/853] 10.222.0.84 () {56 vars in 1132 bytes} [Thu May 19 20:19:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:01.843398 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d9817-e2f7-492c-8817-ed87bf04c68a None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:01.863428 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:01.901228 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe8550-e6d0-4840-bc54-aefe7cd1843f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:01.925764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:01.944402 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680c27f-415d-4f7a-808f-d7855ba62284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:01.945467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680c27f-415d-4f7a-808f-d7855ba62284 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:01.945759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680c27f-415d-4f7a-808f-d7855ba62284 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:01.946035 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680c27f-415d-4f7a-808f-d7855ba62284 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:01.947020 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680c27f-415d-4f7a-808f-d7855ba62284 None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=54395ae480884de38fc811a9e82c1f9e) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:01.947362 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680c27f-415d-4f7a-808f-d7855ba62284 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:01.949442 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680c27f-415d-4f7a-808f-d7855ba62284 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:01.962308 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 426/854] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:19:01 2022] GET /identity/v3/role_assignments?user.id=54395ae480884de38fc811a9e82c1f9e&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=b239f941cb724331bf28dc489b70ad1d => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:01.989627 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d9817-e2f7-492c-8817-ed87bf04c68a None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:02.029015 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7dbf2-802a-4014-ba46-600395f88291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:02.029613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7dbf2-802a-4014-ba46-600395f88291 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:02.029892 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7dbf2-802a-4014-ba46-600395f88291 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:02.030161 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7dbf2-802a-4014-ba46-600395f88291 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:02.030913 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7dbf2-802a-4014-ba46-600395f88291 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:02.031216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7dbf2-802a-4014-ba46-600395f88291 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:02.032624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7dbf2-802a-4014-ba46-600395f88291 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:02.042433 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 429/855] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:19:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:02.045220 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680c27f-415d-4f7a-808f-d7855ba62284 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:02.158707 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680c27f-415d-4f7a-808f-d7855ba62284 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:02.198433 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc9299-9c32-4784-86de-575aa41858da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:02.199531 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc9299-9c32-4784-86de-575aa41858da None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:02.200245 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc9299-9c32-4784-86de-575aa41858da None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:02.200837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc9299-9c32-4784-86de-575aa41858da None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:02.201765 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc9299-9c32-4784-86de-575aa41858da None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:02.202314 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc9299-9c32-4784-86de-575aa41858da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:02.203990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc9299-9c32-4784-86de-575aa41858da None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:02.209474 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 427/856] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:19:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:02.349548 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ed831-cd0b-4be5-b315-07e9c380e085 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:02.349817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ed831-cd0b-4be5-b315-07e9c380e085 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:02.350099 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ed831-cd0b-4be5-b315-07e9c380e085 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:02.351905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 430/857] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:02.357960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db63b2-9916-4540-b0bf-951f210bdfef None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:02.358311 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db63b2-9916-4540-b0bf-951f210bdfef None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:02.358559 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db63b2-9916-4540-b0bf-951f210bdfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:02.376756 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-90db63b2-9916-4540-b0bf-951f210bdfef None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:02.377985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-90db63b2-9916-4540-b0bf-951f210bdfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:02.438859 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:02.464419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 428/858] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:02.488112 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ed831-cd0b-4be5-b315-07e9c380e085 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:02.568590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:02.598440 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ed831-cd0b-4be5-b315-07e9c380e085 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:02.634123 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ae3dc-55b3-4237-98ae-819537d4e54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:02.634885 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ae3dc-55b3-4237-98ae-819537d4e54e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:02.635166 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ae3dc-55b3-4237-98ae-819537d4e54e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:02.635436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ae3dc-55b3-4237-98ae-819537d4e54e None admin] PATH_INFO: `/v3/roles/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:02.646926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ae3dc-55b3-4237-98ae-819537d4e54e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-5d3ae3dc-55b3-4237-98ae-819537d4e54e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:19:02.654962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:02.656258 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:02.657579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:02.657579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:02.657579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:02.657579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:02.657579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:19:02.657579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:19:02.657579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:19:02.657579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:19:02.657579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:19:02.657579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:19:02.657579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:19:02.657579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:02.657579 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 431/859] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:19:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:02.664674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db63b2-9916-4540-b0bf-951f210bdfef None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:02.772451 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db63b2-9916-4540-b0bf-951f210bdfef None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:02.813768 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decfaf7-66a9-4a4f-bc5b-ca2a752eace9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:02.814572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decfaf7-66a9-4a4f-bc5b-ca2a752eace9 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:02.814855 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decfaf7-66a9-4a4f-bc5b-ca2a752eace9 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:02.815129 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decfaf7-66a9-4a4f-bc5b-ca2a752eace9 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:02.815883 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decfaf7-66a9-4a4f-bc5b-ca2a752eace9 None admin] RBAC: Adding query filter params (name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:02.816189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decfaf7-66a9-4a4f-bc5b-ca2a752eace9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:02.817576 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decfaf7-66a9-4a4f-bc5b-ca2a752eace9 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:02.826823 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 429/860] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:19:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:02.832425 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ae3dc-55b3-4237-98ae-819537d4e54e None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:02.936485 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ae3dc-55b3-4237-98ae-819537d4e54e None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:02.972353 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88b18b-9014-4f50-9545-64197fe844af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:02.973160 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88b18b-9014-4f50-9545-64197fe844af None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:02.973438 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88b18b-9014-4f50-9545-64197fe844af None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:02.973710 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88b18b-9014-4f50-9545-64197fe844af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:02.975706 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:02.985625 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88b18b-9014-4f50-9545-64197fe844af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:02.987107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88b18b-9014-4f50-9545-64197fe844af None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:03.005386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-fc88b18b-9014-4f50-9545-64197fe844af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:03.005386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:03.005386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:03.005386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:03.005386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:03.005386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:03.005386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:03.005386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:03.005386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:03.005386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:03.005386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:03.005386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:03.005386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:03.005386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:03.005386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:03.006504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:03.009231 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:03.009231 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:03.009231 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:03.009231 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:03.009231 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 432/861] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:03.012688 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decfaf7-66a9-4a4f-bc5b-ca2a752eace9 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:03.058637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:03.119539 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decfaf7-66a9-4a4f-bc5b-ca2a752eace9 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:03.154504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae2ade-7efc-452a-8ea2-a521e253a582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:03.155176 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae2ade-7efc-452a-8ea2-a521e253a582 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:03.155456 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae2ade-7efc-452a-8ea2-a521e253a582 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:03.155730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae2ade-7efc-452a-8ea2-a521e253a582 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:03.156458 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae2ade-7efc-452a-8ea2-a521e253a582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:03.156763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae2ade-7efc-452a-8ea2-a521e253a582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:03.158149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae2ade-7efc-452a-8ea2-a521e253a582 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:03.174502 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 430/862] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:03.175977 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88b18b-9014-4f50-9545-64197fe844af None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:03.286926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88b18b-9014-4f50-9545-64197fe844af None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:03.320986 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376f01a-107a-4e81-b651-050318c24e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:03.322112 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376f01a-107a-4e81-b651-050318c24e8b None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:03.322600 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376f01a-107a-4e81-b651-050318c24e8b None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:03.323058 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376f01a-107a-4e81-b651-050318c24e8b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:03.331383 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376f01a-107a-4e81-b651-050318c24e8b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:03.333035 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376f01a-107a-4e81-b651-050318c24e8b None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-8376f01a-107a-4e81-b651-050318c24e8b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:19:03.348577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:03.349925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:03.351367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:03.351367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:03.351367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:19:03.351367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:19:03.351367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:19:03.351367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:19:03.351367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:19:03.351367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:19:03.351367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:19:03.351367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:03.353724 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 433/863] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:19:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:03.359158 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae2ade-7efc-452a-8ea2-a521e253a582 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:03.474679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae2ade-7efc-452a-8ea2-a521e253a582 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:03.476296 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:03.510555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9be2af-1d48-447d-847f-7e6f64b4a689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:03.511240 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9be2af-1d48-447d-847f-7e6f64b4a689 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:03.511517 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9be2af-1d48-447d-847f-7e6f64b4a689 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:03.511786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9be2af-1d48-447d-847f-7e6f64b4a689 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:03.512622 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9be2af-1d48-447d-847f-7e6f64b4a689 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:03.512928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9be2af-1d48-447d-847f-7e6f64b4a689 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:03.514406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9be2af-1d48-447d-847f-7e6f64b4a689 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:03.532642 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 431/864] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 20:19:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:03.538375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376f01a-107a-4e81-b651-050318c24e8b None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:03.587017 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:03.601430 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9e891-4d94-4ff0-a82c-44d9d4acb3f4 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:03.602054 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9e891-4d94-4ff0-a82c-44d9d4acb3f4 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:03.602581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9e891-4d94-4ff0-a82c-44d9d4acb3f4 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:03.604413 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 432/865] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:03.609606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e0b50-6fd9-4801-b9d9-c68749b66f57 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:03.610166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e0b50-6fd9-4801-b9d9-c68749b66f57 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:03.610688 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e0b50-6fd9-4801-b9d9-c68749b66f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:03.627585 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-fb0e0b50-6fd9-4801-b9d9-c68749b66f57 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:03.629108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-fb0e0b50-6fd9-4801-b9d9-c68749b66f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:03.646013 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376f01a-107a-4e81-b651-050318c24e8b None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:03.683878 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb6a2a-1657-47cf-aedb-f58ce0406f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:03.684545 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6a2a-1657-47cf-aedb-f58ce0406f09 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:03.684833 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6a2a-1657-47cf-aedb-f58ce0406f09 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:03.685114 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6a2a-1657-47cf-aedb-f58ce0406f09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:03.714318 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 433/866] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:19:03.736451 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e0b50-6fd9-4801-b9d9-c68749b66f57 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:03.766784 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb6a2a-1657-47cf-aedb-f58ce0406f09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:03.768643 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb6a2a-1657-47cf-aedb-f58ce0406f09 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:03.828223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:03.859375 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:03.873513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e0b50-6fd9-4801-b9d9-c68749b66f57 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:03.890446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 434/867] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:19:03 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:19:03.899560 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb6a2a-1657-47cf-aedb-f58ce0406f09 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:03.921571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282b07f-b9e1-45e8-ae1a-1a2c353a4f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:03.923158 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282b07f-b9e1-45e8-ae1a-1a2c353a4f0e None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:03.923901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282b07f-b9e1-45e8-ae1a-1a2c353a4f0e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:03.924359 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282b07f-b9e1-45e8-ae1a-1a2c353a4f0e None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:03.925558 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282b07f-b9e1-45e8-ae1a-1a2c353a4f0e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:03.927826 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282b07f-b9e1-45e8-ae1a-1a2c353a4f0e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:03.948959 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 434/868] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:19:03 2022] POST /identity/v3/services => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:19:04.020328 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb6a2a-1657-47cf-aedb-f58ce0406f09 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:04.067254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb8c55-fda7-4370-8e53-c099adc08415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:04.068075 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb8c55-fda7-4370-8e53-c099adc08415 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:04.068356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb8c55-fda7-4370-8e53-c099adc08415 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:04.068630 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb8c55-fda7-4370-8e53-c099adc08415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:04.081132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb8c55-fda7-4370-8e53-c099adc08415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:04.082561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb8c55-fda7-4370-8e53-c099adc08415 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:04.084687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:04.103877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-65eb8c55-fda7-4370-8e53-c099adc08415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:04.103877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:04.103877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:04.103877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:04.103877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:04.103877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:04.103877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:04.103877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:04.103877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:04.103877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:04.103877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:04.103877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:04.103877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:04.103877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:04.103877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:04.105005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:04.106070 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:04.106070 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:04.106070 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:04.106070 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:04.106070 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 435/869] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:04.111214 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282b07f-b9e1-45e8-ae1a-1a2c353a4f0e None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:04.222383 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282b07f-b9e1-45e8-ae1a-1a2c353a4f0e None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:04.256981 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560f591-4b63-4d6c-a1b8-2e2126474aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:04.257703 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560f591-4b63-4d6c-a1b8-2e2126474aca None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:04.257984 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560f591-4b63-4d6c-a1b8-2e2126474aca None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:04.258277 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560f591-4b63-4d6c-a1b8-2e2126474aca None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:04.259019 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560f591-4b63-4d6c-a1b8-2e2126474aca None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:04.259325 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560f591-4b63-4d6c-a1b8-2e2126474aca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:04.260715 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560f591-4b63-4d6c-a1b8-2e2126474aca None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:04.273076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 435/870] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:04.278608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb8c55-fda7-4370-8e53-c099adc08415 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:04.394995 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb8c55-fda7-4370-8e53-c099adc08415 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:04.431593 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe17715-de6f-4e6d-bd3f-826931f5927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:04.432455 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe17715-de6f-4e6d-bd3f-826931f5927c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:04.432738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe17715-de6f-4e6d-bd3f-826931f5927c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:04.433010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe17715-de6f-4e6d-bd3f-826931f5927c None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:04.440481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe17715-de6f-4e6d-bd3f-826931f5927c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:04.441930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe17715-de6f-4e6d-bd3f-826931f5927c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:04.456372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-efe17715-de6f-4e6d-bd3f-826931f5927c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:04.456372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:04.456372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:04.456372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:04.456372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:04.456372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:04.456372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:04.456372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:04.456372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:04.456372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:04.456372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:04.456372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:04.456372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:19:04.456372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:19:04.456372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:04.457476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:04.458594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:19:04.458594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:19:04.458594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:19:04.458594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:19:04.458594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:19:04.458594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:19:04.458594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:04.458594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:04.458594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 436/871] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:19:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:04.464132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560f591-4b63-4d6c-a1b8-2e2126474aca None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:04.476796 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:04.533603 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a0d5c-8b8c-4d4d-9d50-40ac9f1bece5 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:04.533963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a0d5c-8b8c-4d4d-9d50-40ac9f1bece5 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:04.534145 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a0d5c-8b8c-4d4d-9d50-40ac9f1bece5 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:04.535829 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 437/872] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:19:04.541936 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b812f0-cbe1-4073-b050-8465990f2f56 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:04.542217 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b812f0-cbe1-4073-b050-8465990f2f56 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:04.542536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b812f0-cbe1-4073-b050-8465990f2f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:04.564313 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-d4b812f0-cbe1-4073-b050-8465990f2f56 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:04.565780 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-d4b812f0-cbe1-4073-b050-8465990f2f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:04.585404 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560f591-4b63-4d6c-a1b8-2e2126474aca None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:04.624069 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:04.624983 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80c42c-d553-4075-97c5-d5fd73a1c463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:04.625743 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80c42c-d553-4075-97c5-d5fd73a1c463 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:04.626023 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80c42c-d553-4075-97c5-d5fd73a1c463 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:04.626366 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80c42c-d553-4075-97c5-d5fd73a1c463 None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:04.627257 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80c42c-d553-4075-97c5-d5fd73a1c463 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:04.627566 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80c42c-d553-4075-97c5-d5fd73a1c463 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:04.629068 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80c42c-d553-4075-97c5-d5fd73a1c463 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:04.640695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 436/873] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:19:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:04.642032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318bbc0-d452-4026-8f34-f18feba05508 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:04.642343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318bbc0-d452-4026-8f34-f18feba05508 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:04.642620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318bbc0-d452-4026-8f34-f18feba05508 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:04.646359 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 437/874] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:04.646359 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318bbc0-d452-4026-8f34-f18feba05508 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:04.651791 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 438/875] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:04 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:04.653284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99288f2e-9771-4bb1-8453-c93f20a6238f None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:04.653718 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99288f2e-9771-4bb1-8453-c93f20a6238f None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:04.654141 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99288f2e-9771-4bb1-8453-c93f20a6238f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:04.675528 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-99288f2e-9771-4bb1-8453-c93f20a6238f None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:04.676729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-99288f2e-9771-4bb1-8453-c93f20a6238f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:04.725208 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:04.750030 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 439/876] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:04 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:04.751267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99288f2e-9771-4bb1-8453-c93f20a6238f None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:04.767672 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318bbc0-d452-4026-8f34-f18feba05508 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:04.804365 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3464b-4c08-4292-8779-7f70c0dd99d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:04.805033 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3464b-4c08-4292-8779-7f70c0dd99d7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:04.805311 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3464b-4c08-4292-8779-7f70c0dd99d7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:04.805584 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3464b-4c08-4292-8779-7f70c0dd99d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:04.806435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3464b-4c08-4292-8779-7f70c0dd99d7 None admin] RBAC: Adding query filter params (role.id=c9206a4a0d49473f87a45d6c8abb2e51, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=265a6cde297448adb102da0773e366fb) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:04.806745 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3464b-4c08-4292-8779-7f70c0dd99d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:04.808194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3464b-4c08-4292-8779-7f70c0dd99d7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:04.814635 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 438/877] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:19:04 2022] GET /identity/v3/role_assignments?user.id=265a6cde297448adb102da0773e366fb&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=c9206a4a0d49473f87a45d6c8abb2e51 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:04.815496 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3464b-4c08-4292-8779-7f70c0dd99d7 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:04.882203 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99288f2e-9771-4bb1-8453-c93f20a6238f None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:04.904287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:04.932310 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9896f-c19a-4401-b542-018a90a2ac92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:04.933660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9896f-c19a-4401-b542-018a90a2ac92 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:04.934288 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9896f-c19a-4401-b542-018a90a2ac92 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:04.934797 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9896f-c19a-4401-b542-018a90a2ac92 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:04.935758 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9896f-c19a-4401-b542-018a90a2ac92 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:04.938485 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9896f-c19a-4401-b542-018a90a2ac92 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:04.951602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3464b-4c08-4292-8779-7f70c0dd99d7 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:04.959803 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 440/878] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:19:04 2022] POST /identity/v3/services => generated 269 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:19:05.000858 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:05.008972 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7513b26-4c49-404f-a1e6-e3f6530434a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:05.009811 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7513b26-4c49-404f-a1e6-e3f6530434a1 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:05.010180 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7513b26-4c49-404f-a1e6-e3f6530434a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:05.010520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7513b26-4c49-404f-a1e6-e3f6530434a1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:05.011187 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7513b26-4c49-404f-a1e6-e3f6530434a1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:05.012714 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7513b26-4c49-404f-a1e6-e3f6530434a1 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-b7513b26-4c49-404f-a1e6-e3f6530434a1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:05.027216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:05.028512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:05.029718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:05.029718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:05.029718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:05.029718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:05.029718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:05.029718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:05.029718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:05.029718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:05.029718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:05.029718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:05.029718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:05.029718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:05.029718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:19:05.029718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:05.029718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 439/879] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:19:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:05.037636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9896f-c19a-4401-b542-018a90a2ac92 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:05.162702 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9896f-c19a-4401-b542-018a90a2ac92 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:05.204863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083a1bc-4f62-4f17-aa1d-dcc28643ab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:05.205548 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083a1bc-4f62-4f17-aa1d-dcc28643ab87 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:05.205824 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083a1bc-4f62-4f17-aa1d-dcc28643ab87 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:05.206095 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083a1bc-4f62-4f17-aa1d-dcc28643ab87 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:05.206828 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083a1bc-4f62-4f17-aa1d-dcc28643ab87 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:05.207133 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083a1bc-4f62-4f17-aa1d-dcc28643ab87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:05.208504 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083a1bc-4f62-4f17-aa1d-dcc28643ab87 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:05.215192 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 441/880] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:19:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:05.220402 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7513b26-4c49-404f-a1e6-e3f6530434a1 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:05.326331 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7513b26-4c49-404f-a1e6-e3f6530434a1 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:05.362052 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc020146-b3a8-4e8b-b362-31a2921ac3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:05.362758 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc020146-b3a8-4e8b-b362-31a2921ac3d9 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:05.363042 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc020146-b3a8-4e8b-b362-31a2921ac3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:05.363316 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc020146-b3a8-4e8b-b362-31a2921ac3d9 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:05.364026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc020146-b3a8-4e8b-b362-31a2921ac3d9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:05.364329 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc020146-b3a8-4e8b-b362-31a2921ac3d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:05.365833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc020146-b3a8-4e8b-b362-31a2921ac3d9 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:05.370992 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 440/881] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:19:05 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:06.296605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895f76b-6b6d-43c2-abfc-8549653256b5 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:06.297192 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895f76b-6b6d-43c2-abfc-8549653256b5 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:06.297192 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895f76b-6b6d-43c2-abfc-8549653256b5 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:06.298911 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 442/882] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:06.305014 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273cd3f-54ca-48ac-b6ed-5c8b96912931 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:06.305465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273cd3f-54ca-48ac-b6ed-5c8b96912931 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:06.305653 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273cd3f-54ca-48ac-b6ed-5c8b96912931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:06.323329 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-3273cd3f-54ca-48ac-b6ed-5c8b96912931 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:06.324531 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-3273cd3f-54ca-48ac-b6ed-5c8b96912931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:06.379844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:06.402369 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 441/883] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:06.419452 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895f76b-6b6d-43c2-abfc-8549653256b5 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:06.478624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:06.534203 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895f76b-6b6d-43c2-abfc-8549653256b5 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:06.569250 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff1706-ce9f-47cd-8a72-cb11e247b92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:06.570098 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff1706-ce9f-47cd-8a72-cb11e247b92e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:06.570408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff1706-ce9f-47cd-8a72-cb11e247b92e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:06.570690 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff1706-ce9f-47cd-8a72-cb11e247b92e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:06.571348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff1706-ce9f-47cd-8a72-cb11e247b92e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:06.572735 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff1706-ce9f-47cd-8a72-cb11e247b92e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-50ff1706-ce9f-47cd-8a72-cb11e247b92e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:06.584937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:06.586191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:06.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:06.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:06.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:06.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:06.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:06.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:06.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:06.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:06.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:06.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:06.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:06.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:06.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:19:06.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:06.587225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 443/884] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:19:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:06.595598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273cd3f-54ca-48ac-b6ed-5c8b96912931 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:06.697278 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273cd3f-54ca-48ac-b6ed-5c8b96912931 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:06.733435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c3893-2efe-4f96-b7f0-76b532eb5526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:06.734530 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c3893-2efe-4f96-b7f0-76b532eb5526 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:06.735028 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c3893-2efe-4f96-b7f0-76b532eb5526 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:06.735547 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c3893-2efe-4f96-b7f0-76b532eb5526 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:06.736478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c3893-2efe-4f96-b7f0-76b532eb5526 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:06.737030 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c3893-2efe-4f96-b7f0-76b532eb5526 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:06.738678 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c3893-2efe-4f96-b7f0-76b532eb5526 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:06.744214 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 442/885] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:19:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:06.750292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff1706-ce9f-47cd-8a72-cb11e247b92e None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:06.856096 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff1706-ce9f-47cd-8a72-cb11e247b92e None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:06.857623 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:06.891662 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3ec9c-2e16-4f95-8af3-8f11246ce603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:06.892344 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3ec9c-2e16-4f95-8af3-8f11246ce603 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:06.892631 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3ec9c-2e16-4f95-8af3-8f11246ce603 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:06.892936 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3ec9c-2e16-4f95-8af3-8f11246ce603 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:06.893617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3ec9c-2e16-4f95-8af3-8f11246ce603 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:06.893926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3ec9c-2e16-4f95-8af3-8f11246ce603 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:06.895331 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3ec9c-2e16-4f95-8af3-8f11246ce603 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:06.901164 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 444/886] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:19:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:06.906795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c3893-2efe-4f96-b7f0-76b532eb5526 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:06.994688 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:07.011276 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c3893-2efe-4f96-b7f0-76b532eb5526 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:07.055142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4000f-dfbf-4b70-aa50-57bfca36814e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:07.056276 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4000f-dfbf-4b70-aa50-57bfca36814e None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:07.056821 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4000f-dfbf-4b70-aa50-57bfca36814e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:07.057321 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4000f-dfbf-4b70-aa50-57bfca36814e None admin] PATH_INFO: `/v3/endpoints` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:07.058389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4000f-dfbf-4b70-aa50-57bfca36814e None admin] RBAC: Adding query filter params (interface=public, service_id=e6b4152ef3de4b5e84f20af826bbd81a, region_id=RegionOne) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:07.058939 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4000f-dfbf-4b70-aa50-57bfca36814e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:07.060649 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4000f-dfbf-4b70-aa50-57bfca36814e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:07.069721 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 443/887] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:19:06 2022] GET /identity/v3/endpoints?service_id=e6b4152ef3de4b5e84f20af826bbd81a&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:07.077943 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3ec9c-2e16-4f95-8af3-8f11246ce603 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:07.187875 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3ec9c-2e16-4f95-8af3-8f11246ce603 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:07.219211 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786a8c2-afc9-4ea2-87ac-8931f2d6506b None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:07.219530 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786a8c2-afc9-4ea2-87ac-8931f2d6506b None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:07.219766 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786a8c2-afc9-4ea2-87ac-8931f2d6506b None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:07.221451 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 444/888] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:07.226072 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de33e7-dc5d-4e5f-9d2c-56db9136a3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:07.226800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de33e7-dc5d-4e5f-9d2c-56db9136a3d3 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:07.227089 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de33e7-dc5d-4e5f-9d2c-56db9136a3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:07.227366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de33e7-dc5d-4e5f-9d2c-56db9136a3d3 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:07.227488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf3357-708c-4f57-8f63-5f628a27ecdb None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:07.227702 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf3357-708c-4f57-8f63-5f628a27ecdb None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:07.227984 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf3357-708c-4f57-8f63-5f628a27ecdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:07.228104 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de33e7-dc5d-4e5f-9d2c-56db9136a3d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:07.229555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de33e7-dc5d-4e5f-9d2c-56db9136a3d3 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:07.236360 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 445/889] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:19:07 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:19:07.247102 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-ffcf3357-708c-4f57-8f63-5f628a27ecdb None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:07.248424 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-ffcf3357-708c-4f57-8f63-5f628a27ecdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:07.338955 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 445/890] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:07.359425 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de33e7-dc5d-4e5f-9d2c-56db9136a3d3 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:07.371887 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8480389-7f74-4e8a-ac27-6f6c02edbb24 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:07.372482 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8480389-7f74-4e8a-ac27-6f6c02edbb24 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:07.373001 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8480389-7f74-4e8a-ac27-6f6c02edbb24 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:07.375039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 446/891] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:07.383441 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67483a2-0c57-4bac-af21-eda0e938a735 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:07.384079 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67483a2-0c57-4bac-af21-eda0e938a735 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:07.384549 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67483a2-0c57-4bac-af21-eda0e938a735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:07.400256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:07.403669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-d67483a2-0c57-4bac-af21-eda0e938a735 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:07.405145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-d67483a2-0c57-4bac-af21-eda0e938a735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:07.458427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:07.496976 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 447/892] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:19:07.500168 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de33e7-dc5d-4e5f-9d2c-56db9136a3d3 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:07.521177 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67483a2-0c57-4bac-af21-eda0e938a735 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:07.540868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef04f9-7448-48c9-bb45-bb0f07ddb8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:07.542098 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef04f9-7448-48c9-bb45-bb0f07ddb8e2 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:07.542586 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef04f9-7448-48c9-bb45-bb0f07ddb8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:07.543042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef04f9-7448-48c9-bb45-bb0f07ddb8e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:07.556839 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef04f9-7448-48c9-bb45-bb0f07ddb8e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-d8ef04f9-7448-48c9-bb45-bb0f07ddb8e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:19:07.565924 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:07.567403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:07.568660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:07.568660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:07.568660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:07.568660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:07.568660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:19:07.568660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:19:07.568660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:19:07.568660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:19:07.568660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:19:07.568660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:19:07.568660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:19:07.568660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:07.572139 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 446/893] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 20:19:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:07.578860 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef04f9-7448-48c9-bb45-bb0f07ddb8e2 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:07.649311 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67483a2-0c57-4bac-af21-eda0e938a735 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:07.694484 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452461d-edf8-4627-a8b7-507770c1069f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:07.696050 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452461d-edf8-4627-a8b7-507770c1069f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:07.696336 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452461d-edf8-4627-a8b7-507770c1069f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:07.696618 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452461d-edf8-4627-a8b7-507770c1069f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:07.697294 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452461d-edf8-4627-a8b7-507770c1069f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:07.699248 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452461d-edf8-4627-a8b7-507770c1069f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:07.701528 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:07.715215 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef04f9-7448-48c9-bb45-bb0f07ddb8e2 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-0452461d-edf8-4627-a8b7-507770c1069f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:07.717457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:07.718577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:07.719590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:07.719590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:07.719590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:07.719590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:07.719590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:07.719590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:07.719590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:07.719590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:07.719590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:07.719590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:07.719590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:07.719590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:07.719590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:07.719590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:19:07.719590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:07.722342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 448/894] 10.222.0.84 () {56 vars in 1132 bytes} [Thu May 19 20:19:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:07.727447 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452461d-edf8-4627-a8b7-507770c1069f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:07.760684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a778e1-af9d-4270-b89a-a5a740f0fda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:07.761359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a778e1-af9d-4270-b89a-a5a740f0fda3 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:07.761634 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a778e1-af9d-4270-b89a-a5a740f0fda3 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:07.761909 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a778e1-af9d-4270-b89a-a5a740f0fda3 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:07.762688 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a778e1-af9d-4270-b89a-a5a740f0fda3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:07.762971 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a778e1-af9d-4270-b89a-a5a740f0fda3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:07.764343 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a778e1-af9d-4270-b89a-a5a740f0fda3 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:07.778412 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 447/895] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 20:19:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:07.778412 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93268c-23c2-45c6-8509-cb0fae4815e4 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:07.778412 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93268c-23c2-45c6-8509-cb0fae4815e4 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:07.778412 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93268c-23c2-45c6-8509-cb0fae4815e4 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:07.778412 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 448/896] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:07.781517 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93268c-23c2-45c6-8509-cb0fae4815e4 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:07.782337 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:07.847460 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452461d-edf8-4627-a8b7-507770c1069f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:07.887328 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5adb2d-8a7a-4dc0-bc91-204cf8ad81d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:07.888370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5adb2d-8a7a-4dc0-bc91-204cf8ad81d6 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:07.888889 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5adb2d-8a7a-4dc0-bc91-204cf8ad81d6 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:07.889394 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5adb2d-8a7a-4dc0-bc91-204cf8ad81d6 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:07.890426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5adb2d-8a7a-4dc0-bc91-204cf8ad81d6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:07.890955 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5adb2d-8a7a-4dc0-bc91-204cf8ad81d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:07.891503 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93268c-23c2-45c6-8509-cb0fae4815e4 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:07.892861 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5adb2d-8a7a-4dc0-bc91-204cf8ad81d6 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:07.898960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 449/897] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:19:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:07.900201 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc3f7b-fa36-492a-bfe0-27930269ab88 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:07.900483 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc3f7b-fa36-492a-bfe0-27930269ab88 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:07.900756 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc3f7b-fa36-492a-bfe0-27930269ab88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:07.917150 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-14fc3f7b-fa36-492a-bfe0-27930269ab88 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:07.918364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-14fc3f7b-fa36-492a-bfe0-27930269ab88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:07.932330 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34dd56-021f-458c-a363-e777c8a72d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:07.933516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34dd56-021f-458c-a363-e777c8a72d41 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:07.933762 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34dd56-021f-458c-a363-e777c8a72d41 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:07.934023 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34dd56-021f-458c-a363-e777c8a72d41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:07.945732 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34dd56-021f-458c-a363-e777c8a72d41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:07.947719 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34dd56-021f-458c-a363-e777c8a72d41 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:07.974729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-8b34dd56-021f-458c-a363-e777c8a72d41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:07.974729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:07.974729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:07.974729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:07.974729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:07.974729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:07.974729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:07.974729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:07.974729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:07.974729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:07.974729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:07.974729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:07.974729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:07.974729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:07.974729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:07.976103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:07.977469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:07.977469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:07.977469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:07.977469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:07.982604 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 449/898] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:07.983970 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34dd56-021f-458c-a363-e777c8a72d41 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:07.988000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:08.002450 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 450/899] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:08.003774 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc3f7b-fa36-492a-bfe0-27930269ab88 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:08.045218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:08.113368 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34dd56-021f-458c-a363-e777c8a72d41 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:08.114782 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc3f7b-fa36-492a-bfe0-27930269ab88 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:08.149636 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e4d4e-b83b-4175-bbda-37fedfb44118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:08.149897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71bccc9-a0e9-494e-98d2-3508e8bf08a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:08.150348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e4d4e-b83b-4175-bbda-37fedfb44118 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:08.150487 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bccc9-a0e9-494e-98d2-3508e8bf08a8 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:08.150657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bccc9-a0e9-494e-98d2-3508e8bf08a8 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:08.150770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e4d4e-b83b-4175-bbda-37fedfb44118 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:08.150933 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e4d4e-b83b-4175-bbda-37fedfb44118 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:08.151050 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bccc9-a0e9-494e-98d2-3508e8bf08a8 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:08.151589 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71bccc9-a0e9-494e-98d2-3508e8bf08a8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:08.151695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e4d4e-b83b-4175-bbda-37fedfb44118 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:08.151892 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71bccc9-a0e9-494e-98d2-3508e8bf08a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:08.152010 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e4d4e-b83b-4175-bbda-37fedfb44118 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:08.153261 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71bccc9-a0e9-494e-98d2-3508e8bf08a8 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:08.153386 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e4d4e-b83b-4175-bbda-37fedfb44118 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:08.163039 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 450/900] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:19:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:08.163039 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71bccc9-a0e9-494e-98d2-3508e8bf08a8 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:08.169054 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 451/901] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:08.173720 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e4d4e-b83b-4175-bbda-37fedfb44118 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:08.235891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:08.272234 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71bccc9-a0e9-494e-98d2-3508e8bf08a8 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:08.277840 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:08.285476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e4d4e-b83b-4175-bbda-37fedfb44118 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:08.309796 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d39a0-78ec-48f3-a8c9-8e115716fe4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:08.310526 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d39a0-78ec-48f3-a8c9-8e115716fe4f None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:08.310808 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d39a0-78ec-48f3-a8c9-8e115716fe4f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:08.311072 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d39a0-78ec-48f3-a8c9-8e115716fe4f None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:08.311804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d39a0-78ec-48f3-a8c9-8e115716fe4f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:08.313198 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d39a0-78ec-48f3-a8c9-8e115716fe4f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:08.324232 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36b715-0960-495c-af35-4f4f6e43b73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:08.324931 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36b715-0960-495c-af35-4f4f6e43b73f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:08.325195 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36b715-0960-495c-af35-4f4f6e43b73f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:08.325476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36b715-0960-495c-af35-4f4f6e43b73f None admin] PATH_INFO: `/v3/endpoints` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:08.326316 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36b715-0960-495c-af35-4f4f6e43b73f None admin] RBAC: Adding query filter params (interface=public, service_id=5c2eea7631644e11809f7ce4e12d8bc1, region_id=RegionOne) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:08.326642 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36b715-0960-495c-af35-4f4f6e43b73f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:08.328039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36b715-0960-495c-af35-4f4f6e43b73f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:08.334679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 451/902] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:19:08 2022] POST /identity/v3/services => generated 241 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:19:08.334679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d39a0-78ec-48f3-a8c9-8e115716fe4f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:08.334993 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 452/903] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:19:08 2022] GET /identity/v3/endpoints?service_id=5c2eea7631644e11809f7ce4e12d8bc1&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:08.340511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36b715-0960-495c-af35-4f4f6e43b73f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:08.460714 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d39a0-78ec-48f3-a8c9-8e115716fe4f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:08.461322 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36b715-0960-495c-af35-4f4f6e43b73f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:08.488882 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:08.501264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d19fb-b88b-4ee0-aea8-5ea5f0c146e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:08.501989 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d19fb-b88b-4ee0-aea8-5ea5f0c146e2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:08.502294 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d19fb-b88b-4ee0-aea8-5ea5f0c146e2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:08.502601 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d19fb-b88b-4ee0-aea8-5ea5f0c146e2 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:08.503103 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7de6a1-00e8-4c59-b9bb-bf5fa4ddf782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:08.503769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d19fb-b88b-4ee0-aea8-5ea5f0c146e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:08.504434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7de6a1-00e8-4c59-b9bb-bf5fa4ddf782 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:08.504923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7de6a1-00e8-4c59-b9bb-bf5fa4ddf782 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:08.505341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d19fb-b88b-4ee0-aea8-5ea5f0c146e2 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:08.505651 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7de6a1-00e8-4c59-b9bb-bf5fa4ddf782 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:08.518313 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 453/904] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:19:08 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:19:08.523780 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7de6a1-00e8-4c59-b9bb-bf5fa4ddf782 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:08.525724 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7de6a1-00e8-4c59-b9bb-bf5fa4ddf782 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-3a7de6a1-00e8-4c59-b9bb-bf5fa4ddf782 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:19:08.543628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:08.544815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:08.545944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:08.545944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:08.545944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:19:08.545944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:19:08.545944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:19:08.545944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:19:08.545944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:19:08.545944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:19:08.545944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:19:08.545944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:08.545944 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 452/905] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:19:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:08.550945 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d19fb-b88b-4ee0-aea8-5ea5f0c146e2 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:08.582568 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:08.671785 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d19fb-b88b-4ee0-aea8-5ea5f0c146e2 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:08.710695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7091d48-5ec6-44f7-bb73-c47112bd2d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:08.711401 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7091d48-5ec6-44f7-bb73-c47112bd2d05 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:08.711683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7091d48-5ec6-44f7-bb73-c47112bd2d05 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:08.711968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7091d48-5ec6-44f7-bb73-c47112bd2d05 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:08.712825 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7091d48-5ec6-44f7-bb73-c47112bd2d05 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:08.713142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7091d48-5ec6-44f7-bb73-c47112bd2d05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:08.714967 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7091d48-5ec6-44f7-bb73-c47112bd2d05 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:08.734472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 454/906] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 20:19:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:08.740138 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7de6a1-00e8-4c59-b9bb-bf5fa4ddf782 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:08.844156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7de6a1-00e8-4c59-b9bb-bf5fa4ddf782 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:08.878538 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac401c6-0577-4155-a2e4-aaf345b5430d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:08.879202 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac401c6-0577-4155-a2e4-aaf345b5430d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:08.879480 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac401c6-0577-4155-a2e4-aaf345b5430d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:08.879754 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac401c6-0577-4155-a2e4-aaf345b5430d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:08.944300 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:08.962808 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac401c6-0577-4155-a2e4-aaf345b5430d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:08.964701 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac401c6-0577-4155-a2e4-aaf345b5430d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:09.091755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 453/907] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:19:08 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:19:09.097148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7091d48-5ec6-44f7-bb73-c47112bd2d05 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:09.183751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:09.204736 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7091d48-5ec6-44f7-bb73-c47112bd2d05 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:09.242112 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e38421-5249-43f8-8d88-9627251e69b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:09.242952 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e38421-5249-43f8-8d88-9627251e69b6 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:09.243234 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e38421-5249-43f8-8d88-9627251e69b6 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:09.243513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e38421-5249-43f8-8d88-9627251e69b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:09.254454 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e38421-5249-43f8-8d88-9627251e69b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:09.255928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e38421-5249-43f8-8d88-9627251e69b6 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:09.274292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-53e38421-5249-43f8-8d88-9627251e69b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:09.274292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:09.274292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:09.274292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:09.274292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:09.274292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:09.274292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:09.274292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:09.274292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:09.274292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:09.274292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:09.274292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:09.274292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:09.274292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:09.274292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:09.275444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:09.276491 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:09.276491 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:09.276491 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:09.276491 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:09.276491 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 455/908] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:09.281856 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac401c6-0577-4155-a2e4-aaf345b5430d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:09.370286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:09.392146 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac401c6-0577-4155-a2e4-aaf345b5430d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:09.431294 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26b40c-c220-4809-aa15-d4434e681d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:09.434300 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26b40c-c220-4809-aa15-d4434e681d65 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:09.434300 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26b40c-c220-4809-aa15-d4434e681d65 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:09.434300 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26b40c-c220-4809-aa15-d4434e681d65 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:09.434300 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26b40c-c220-4809-aa15-d4434e681d65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:09.434300 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26b40c-c220-4809-aa15-d4434e681d65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:09.434816 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26b40c-c220-4809-aa15-d4434e681d65 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:09.446951 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 454/909] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:09.452565 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e38421-5249-43f8-8d88-9627251e69b6 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:09.559387 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e38421-5249-43f8-8d88-9627251e69b6 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:09.586762 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58296c46-b549-4642-99d1-8debd3a847cd None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:09.587781 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58296c46-b549-4642-99d1-8debd3a847cd None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:09.588270 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58296c46-b549-4642-99d1-8debd3a847cd None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:09.590617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 455/910] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:19:09.593266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:09.596773 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85100b3-fd99-496a-9b32-8fba5265a681 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:09.597058 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85100b3-fd99-496a-9b32-8fba5265a681 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:09.597334 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85100b3-fd99-496a-9b32-8fba5265a681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:09.599024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05478b6-087e-48d4-bf91-0d356d1c031f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:09.600092 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05478b6-087e-48d4-bf91-0d356d1c031f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:09.600765 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05478b6-087e-48d4-bf91-0d356d1c031f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:09.601463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05478b6-087e-48d4-bf91-0d356d1c031f None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:09.609735 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05478b6-087e-48d4-bf91-0d356d1c031f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:09.611553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05478b6-087e-48d4-bf91-0d356d1c031f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:09.614455 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-a85100b3-fd99-496a-9b32-8fba5265a681 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:09.615645 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-a85100b3-fd99-496a-9b32-8fba5265a681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:09.626412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-e05478b6-087e-48d4-bf91-0d356d1c031f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:09.626412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:09.626412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:09.626412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:09.626412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:09.626412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:09.626412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:09.626412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:09.626412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:09.626412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:09.626412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:09.626412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:09.626412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:19:09.626412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:19:09.626412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:09.627449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:09.628535 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:19:09.628535 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:19:09.628535 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:19:09.628535 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:19:09.628535 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:19:09.628535 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:19:09.628535 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:09.628535 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:09.635850 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 456/911] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:19:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:09.637019 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05478b6-087e-48d4-bf91-0d356d1c031f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:09.698283 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 456/912] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:09.714364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85100b3-fd99-496a-9b32-8fba5265a681 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:09.751501 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05478b6-087e-48d4-bf91-0d356d1c031f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:09.789081 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec123c8-f465-4c48-b64a-cbd9436b4df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:09.790791 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec123c8-f465-4c48-b64a-cbd9436b4df8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:09.791563 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec123c8-f465-4c48-b64a-cbd9436b4df8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:09.792077 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec123c8-f465-4c48-b64a-cbd9436b4df8 None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:09.793531 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec123c8-f465-4c48-b64a-cbd9436b4df8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:09.794324 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec123c8-f465-4c48-b64a-cbd9436b4df8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:09.796000 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec123c8-f465-4c48-b64a-cbd9436b4df8 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:09.815711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 457/913] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:19:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:09.815919 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:09.819232 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec123c8-f465-4c48-b64a-cbd9436b4df8 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:09.846764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85100b3-fd99-496a-9b32-8fba5265a681 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:09.892377 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:09.900013 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a6a97-aaeb-4129-90ac-86734c94b12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:09.900850 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a6a97-aaeb-4129-90ac-86734c94b12c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:09.901149 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a6a97-aaeb-4129-90ac-86734c94b12c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:09.901805 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a6a97-aaeb-4129-90ac-86734c94b12c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:09.902543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a6a97-aaeb-4129-90ac-86734c94b12c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:09.904492 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a6a97-aaeb-4129-90ac-86734c94b12c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-1e6a6a97-aaeb-4129-90ac-86734c94b12c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:09.920165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:09.921371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:09.922431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:09.922431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:09.922431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:09.922431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:09.922431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:09.922431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:09.922431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:09.922431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:09.922431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:09.922431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:09.922431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:09.922431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:09.922431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:19:09.922431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:09.926911 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 457/914] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:19:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:09.928003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a6a97-aaeb-4129-90ac-86734c94b12c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:09.957222 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec123c8-f465-4c48-b64a-cbd9436b4df8 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:09.994176 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc018d4c-701f-4823-9be0-df527dc15cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:09.994789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc018d4c-701f-4823-9be0-df527dc15cf8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:09.995067 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc018d4c-701f-4823-9be0-df527dc15cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:09.995340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc018d4c-701f-4823-9be0-df527dc15cf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:09.996270 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc018d4c-701f-4823-9be0-df527dc15cf8 None admin] RBAC: Adding query filter params (role.id=943b7648db36455b8073c12e3b45374d, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=265a6cde297448adb102da0773e366fb) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:09.996496 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc018d4c-701f-4823-9be0-df527dc15cf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:09.997919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc018d4c-701f-4823-9be0-df527dc15cf8 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:10.010704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 458/915] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:19:09 2022] GET /identity/v3/role_assignments?user.id=265a6cde297448adb102da0773e366fb&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=943b7648db36455b8073c12e3b45374d => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:10.048292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a6a97-aaeb-4129-90ac-86734c94b12c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:10.089351 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85c0f9-c086-4a41-874c-b150220d86e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:10.090511 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85c0f9-c086-4a41-874c-b150220d86e1 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:10.091014 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85c0f9-c086-4a41-874c-b150220d86e1 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:10.091524 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85c0f9-c086-4a41-874c-b150220d86e1 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:10.092485 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85c0f9-c086-4a41-874c-b150220d86e1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:10.093015 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85c0f9-c086-4a41-874c-b150220d86e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:10.094758 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85c0f9-c086-4a41-874c-b150220d86e1 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:10.101310 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 458/916] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:19:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:10.107226 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc018d4c-701f-4823-9be0-df527dc15cf8 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:10.220751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc018d4c-701f-4823-9be0-df527dc15cf8 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:10.238841 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:10.258368 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca30d3-7d05-49d8-b294-bf0cc7467299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:10.259042 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca30d3-7d05-49d8-b294-bf0cc7467299 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:10.259322 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca30d3-7d05-49d8-b294-bf0cc7467299 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:10.259595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca30d3-7d05-49d8-b294-bf0cc7467299 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:10.260305 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca30d3-7d05-49d8-b294-bf0cc7467299 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:10.260665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca30d3-7d05-49d8-b294-bf0cc7467299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:10.262115 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca30d3-7d05-49d8-b294-bf0cc7467299 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:10.267759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 459/917] 10.222.0.84 () {56 vars in 1114 bytes} [Thu May 19 20:19:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:10.273582 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85c0f9-c086-4a41-874c-b150220d86e1 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:10.279872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:10.387770 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85c0f9-c086-4a41-874c-b150220d86e1 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:10.432019 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf2886-849c-41f7-877e-bc6112805c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:10.432706 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf2886-849c-41f7-877e-bc6112805c89 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:10.432990 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf2886-849c-41f7-877e-bc6112805c89 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:10.433262 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf2886-849c-41f7-877e-bc6112805c89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:10.434016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf2886-849c-41f7-877e-bc6112805c89 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:10.436443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf2886-849c-41f7-877e-bc6112805c89 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:10.459279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 459/918] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:19:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:19:10.733151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048c319-bd10-49a4-af90-522bbfdb9192 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:10.733732 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048c319-bd10-49a4-af90-522bbfdb9192 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:10.733732 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048c319-bd10-49a4-af90-522bbfdb9192 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:10.735357 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 460/919] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:10.740748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91979202-4988-41d5-a4f6-0cee459133cc None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:10.741236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91979202-4988-41d5-a4f6-0cee459133cc None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:10.741427 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91979202-4988-41d5-a4f6-0cee459133cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:10.759851 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-91979202-4988-41d5-a4f6-0cee459133cc None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:10.761032 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-91979202-4988-41d5-a4f6-0cee459133cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:10.822858 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:10.847374 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 460/920] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:10.870052 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048c319-bd10-49a4-af90-522bbfdb9192 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:10.972093 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14cd68-775a-4b67-a2d4-639115b5a056 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:10.972359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14cd68-775a-4b67-a2d4-639115b5a056 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:10.972637 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14cd68-775a-4b67-a2d4-639115b5a056 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:10.975623 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 461/921] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:10.979902 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48882c61-012e-4e78-938f-03f1b64cd39a None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:10.980182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48882c61-012e-4e78-938f-03f1b64cd39a None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:10.980472 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48882c61-012e-4e78-938f-03f1b64cd39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:10.985600 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048c319-bd10-49a4-af90-522bbfdb9192 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:10.998519 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-48882c61-012e-4e78-938f-03f1b64cd39a None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:10.999902 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-48882c61-012e-4e78-938f-03f1b64cd39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:11.031229 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2783db8-7b52-43d8-8ffd-b3c11a35e5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:11.032098 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2783db8-7b52-43d8-8ffd-b3c11a35e5a1 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:11.032515 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2783db8-7b52-43d8-8ffd-b3c11a35e5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:11.032805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2783db8-7b52-43d8-8ffd-b3c11a35e5a1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:11.033583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2783db8-7b52-43d8-8ffd-b3c11a35e5a1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:11.035683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2783db8-7b52-43d8-8ffd-b3c11a35e5a1 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-e2783db8-7b52-43d8-8ffd-b3c11a35e5a1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:11.051435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:11.052662 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:11.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:11.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:11.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:11.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:11.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:11.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:11.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:11.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:11.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:11.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:11.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:11.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:11.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:19:11.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:11.053708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 461/922] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:19:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:11.059846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2783db8-7b52-43d8-8ffd-b3c11a35e5a1 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:11.091667 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 462/923] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:19:11.114305 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48882c61-012e-4e78-938f-03f1b64cd39a None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:11.130477 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:11.200919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2783db8-7b52-43d8-8ffd-b3c11a35e5a1 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:11.231520 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48882c61-012e-4e78-938f-03f1b64cd39a None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:11.239809 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343214f-c89b-4036-8854-22915c184d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:11.240451 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343214f-c89b-4036-8854-22915c184d57 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:11.240734 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343214f-c89b-4036-8854-22915c184d57 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:11.241011 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343214f-c89b-4036-8854-22915c184d57 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:11.241768 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343214f-c89b-4036-8854-22915c184d57 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:11.242154 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343214f-c89b-4036-8854-22915c184d57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:11.243714 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343214f-c89b-4036-8854-22915c184d57 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:11.249230 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 462/924] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:19:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:11.253818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:11.255543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343214f-c89b-4036-8854-22915c184d57 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:11.277066 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8b4ab-28f4-4266-9371-e8e400677a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:11.278413 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8b4ab-28f4-4266-9371-e8e400677a7c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:11.278825 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8b4ab-28f4-4266-9371-e8e400677a7c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:11.279105 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8b4ab-28f4-4266-9371-e8e400677a7c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:11.280068 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8b4ab-28f4-4266-9371-e8e400677a7c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:11.281466 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8b4ab-28f4-4266-9371-e8e400677a7c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-b6a8b4ab-28f4-4266-9371-e8e400677a7c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:11.297513 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:11.298716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:11.299723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 463/925] 10.222.0.84 () {56 vars in 1132 bytes} [Thu May 19 20:19:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:11.305622 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8b4ab-28f4-4266-9371-e8e400677a7c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:11.380088 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343214f-c89b-4036-8854-22915c184d57 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:11.410689 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:11.422316 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5f03b-fcc3-416f-bd4f-ec9d81d597cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:11.422316 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5f03b-fcc3-416f-bd4f-ec9d81d597cb None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:11.422316 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5f03b-fcc3-416f-bd4f-ec9d81d597cb None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:11.422316 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5f03b-fcc3-416f-bd4f-ec9d81d597cb None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:11.423190 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5f03b-fcc3-416f-bd4f-ec9d81d597cb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:11.423747 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5f03b-fcc3-416f-bd4f-ec9d81d597cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:11.425320 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5f03b-fcc3-416f-bd4f-ec9d81d597cb None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:11.437127 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 463/926] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:19:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:11.446367 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5f03b-fcc3-416f-bd4f-ec9d81d597cb None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:11.446542 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8b4ab-28f4-4266-9371-e8e400677a7c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:11.490336 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad3d4c-c6d3-4a2b-9c8e-5842ef610d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:11.490934 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad3d4c-c6d3-4a2b-9c8e-5842ef610d16 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:11.491310 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad3d4c-c6d3-4a2b-9c8e-5842ef610d16 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:11.491594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad3d4c-c6d3-4a2b-9c8e-5842ef610d16 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:11.492401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad3d4c-c6d3-4a2b-9c8e-5842ef610d16 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:11.492712 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad3d4c-c6d3-4a2b-9c8e-5842ef610d16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:11.494132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad3d4c-c6d3-4a2b-9c8e-5842ef610d16 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:11.502594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 464/927] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:19:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:11.508510 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad3d4c-c6d3-4a2b-9c8e-5842ef610d16 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:11.567241 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:11.582343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5f03b-fcc3-416f-bd4f-ec9d81d597cb None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:11.629882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b81a39-233c-4591-bcb0-285f7b9f6c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:11.630936 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b81a39-233c-4591-bcb0-285f7b9f6c8c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:11.631402 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b81a39-233c-4591-bcb0-285f7b9f6c8c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:11.631986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b81a39-233c-4591-bcb0-285f7b9f6c8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:11.632998 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b81a39-233c-4591-bcb0-285f7b9f6c8c None admin] RBAC: Adding query filter params (interface=public, service_id=c21c70bdc50745e58e26d3bb808b8644, region_id=RegionOne) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:11.633569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b81a39-233c-4591-bcb0-285f7b9f6c8c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:11.635196 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b81a39-233c-4591-bcb0-285f7b9f6c8c None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:11.641466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 464/928] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:19:11 2022] GET /identity/v3/endpoints?service_id=c21c70bdc50745e58e26d3bb808b8644&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:11.642567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad3d4c-c6d3-4a2b-9c8e-5842ef610d16 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:11.646547 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b81a39-233c-4591-bcb0-285f7b9f6c8c None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:11.683869 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:11.691942 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9431e6-bd3f-4719-abdb-c844122c5ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:11.692951 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9431e6-bd3f-4719-abdb-c844122c5ebc None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:11.693405 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9431e6-bd3f-4719-abdb-c844122c5ebc None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:11.694437 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9431e6-bd3f-4719-abdb-c844122c5ebc None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:11.695966 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9431e6-bd3f-4719-abdb-c844122c5ebc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:11.696450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9431e6-bd3f-4719-abdb-c844122c5ebc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:11.698347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9431e6-bd3f-4719-abdb-c844122c5ebc None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:11.711436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 465/929] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:19:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:11.713746 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9431e6-bd3f-4719-abdb-c844122c5ebc None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:11.771593 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b81a39-233c-4591-bcb0-285f7b9f6c8c None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:11.809018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f79d45-9868-48df-b5f9-c01194f93876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:11.809747 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f79d45-9868-48df-b5f9-c01194f93876 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:11.810072 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f79d45-9868-48df-b5f9-c01194f93876 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:11.810561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f79d45-9868-48df-b5f9-c01194f93876 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:11.811418 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f79d45-9868-48df-b5f9-c01194f93876 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:11.813156 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f79d45-9868-48df-b5f9-c01194f93876 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:11.820353 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 465/930] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:19:11 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:19:11.839925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9431e6-bd3f-4719-abdb-c844122c5ebc None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:11.848829 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:11.883207 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e774514-2798-4cfd-8cac-d29c8d843413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:11.884096 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e774514-2798-4cfd-8cac-d29c8d843413 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:11.884439 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e774514-2798-4cfd-8cac-d29c8d843413 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:11.884727 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e774514-2798-4cfd-8cac-d29c8d843413 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:11.885507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e774514-2798-4cfd-8cac-d29c8d843413 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:11.886967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e774514-2798-4cfd-8cac-d29c8d843413 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:11.918517 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 466/931] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:19:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:19:12.352058 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeed680-c75f-4efb-a6f2-7893f52ce4d9 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:12.352790 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeed680-c75f-4efb-a6f2-7893f52ce4d9 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:12.353293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeed680-c75f-4efb-a6f2-7893f52ce4d9 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:12.355209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 466/932] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:12.360991 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91005f5f-91eb-483b-83e8-c62a7340b5ad None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:12.361285 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91005f5f-91eb-483b-83e8-c62a7340b5ad None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:12.361568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91005f5f-91eb-483b-83e8-c62a7340b5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:12.379550 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-91005f5f-91eb-483b-83e8-c62a7340b5ad None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:12.381128 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-91005f5f-91eb-483b-83e8-c62a7340b5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:12.459647 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 467/933] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:12.483146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeed680-c75f-4efb-a6f2-7893f52ce4d9 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:12.593307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:12.598117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeed680-c75f-4efb-a6f2-7893f52ce4d9 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:12.633904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e1605-ad39-4254-9201-6bf256d2271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:12.635078 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1605-ad39-4254-9201-6bf256d2271f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:12.635571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1605-ad39-4254-9201-6bf256d2271f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:12.636043 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1605-ad39-4254-9201-6bf256d2271f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:12.646956 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e1605-ad39-4254-9201-6bf256d2271f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-d35e1605-ad39-4254-9201-6bf256d2271f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:19:12.655759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:12.656961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:12.658076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:12.658076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:12.658076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:12.658076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:12.658076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:19:12.658076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:19:12.658076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:19:12.658076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:19:12.658076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:19:12.658076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:19:12.658076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:19:12.658076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:12.660867 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 467/934] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 20:19:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:12.667169 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91005f5f-91eb-483b-83e8-c62a7340b5ad None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:12.762053 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:12.777175 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91005f5f-91eb-483b-83e8-c62a7340b5ad None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:12.814865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e66d6e-06dd-4048-adb0-14021e91d547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:12.815831 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e66d6e-06dd-4048-adb0-14021e91d547 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:12.816265 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e66d6e-06dd-4048-adb0-14021e91d547 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:12.816690 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e66d6e-06dd-4048-adb0-14021e91d547 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:12.817605 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e66d6e-06dd-4048-adb0-14021e91d547 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:12.818085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e66d6e-06dd-4048-adb0-14021e91d547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:12.819849 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e66d6e-06dd-4048-adb0-14021e91d547 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:12.829286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 468/935] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 20:19:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:12.834866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e1605-ad39-4254-9201-6bf256d2271f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:12.870460 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380e0e4-63dd-44ef-bc7e-7b31c79557b0 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:12.871377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380e0e4-63dd-44ef-bc7e-7b31c79557b0 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:12.871825 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380e0e4-63dd-44ef-bc7e-7b31c79557b0 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:12.874738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 469/936] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:19:12.878867 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb0917-cc17-4d24-b499-1054f5aca1b7 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:12.879345 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb0917-cc17-4d24-b499-1054f5aca1b7 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:12.879809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb0917-cc17-4d24-b499-1054f5aca1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:12.904385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-53eb0917-cc17-4d24-b499-1054f5aca1b7 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:12.906000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-53eb0917-cc17-4d24-b499-1054f5aca1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:12.941617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e1605-ad39-4254-9201-6bf256d2271f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:12.979222 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7c791-58bd-4907-8e09-aa9a08ebdc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:12.980328 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7c791-58bd-4907-8e09-aa9a08ebdc07 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:12.980796 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7c791-58bd-4907-8e09-aa9a08ebdc07 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:12.981292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7c791-58bd-4907-8e09-aa9a08ebdc07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:12.993939 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7c791-58bd-4907-8e09-aa9a08ebdc07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:12.995678 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7c791-58bd-4907-8e09-aa9a08ebdc07 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:12.998350 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 470/937] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:19:13.022699 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb0917-cc17-4d24-b499-1054f5aca1b7 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:13.023476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-26c7c791-58bd-4907-8e09-aa9a08ebdc07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:13.023476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:13.023476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:13.023476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:13.023476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:13.023476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:13.023476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:13.023476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:13.023476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:13.023476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:13.023476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:13.023476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:13.023476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:13.023476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:13.023476 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:13.024711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:13.025819 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:13.025819 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:13.025819 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:13.025819 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:13.025819 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 468/938] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:13.030353 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7c791-58bd-4907-8e09-aa9a08ebdc07 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:13.047358 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:13.137299 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:13.160616 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7c791-58bd-4907-8e09-aa9a08ebdc07 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:13.160815 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb0917-cc17-4d24-b499-1054f5aca1b7 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:13.196915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa9058-dd0b-4e35-bcc6-19ea7ad28202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:13.197962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa9058-dd0b-4e35-bcc6-19ea7ad28202 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:13.198495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa9058-dd0b-4e35-bcc6-19ea7ad28202 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:13.198975 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa9058-dd0b-4e35-bcc6-19ea7ad28202 None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:13.199901 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa9058-dd0b-4e35-bcc6-19ea7ad28202 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:13.201614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa9058-dd0b-4e35-bcc6-19ea7ad28202 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:13.203169 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: WARNING keystone.server.flask.common [None req-37fa9058-dd0b-4e35-bcc6-19ea7ad28202 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 20:19:13.205305 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a91c3-00ce-4e44-a7b8-9e8d6d0c9462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:13.206051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a91c3-00ce-4e44-a7b8-9e8d6d0c9462 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:13.206444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a91c3-00ce-4e44-a7b8-9e8d6d0c9462 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:13.206791 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a91c3-00ce-4e44-a7b8-9e8d6d0c9462 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:13.207760 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a91c3-00ce-4e44-a7b8-9e8d6d0c9462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:13.208143 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a91c3-00ce-4e44-a7b8-9e8d6d0c9462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:13.210065 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a91c3-00ce-4e44-a7b8-9e8d6d0c9462 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:13.224017 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.sql.core [None req-37fa9058-dd0b-4e35-bcc6-19ea7ad28202 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:19:13.224017 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 20:19:13.224017 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [parameters: {'id': 'c2a1399fb60b4884ad3f6246df9be5ad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 20:19:13.224017 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=277682) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:19:13.226538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 469/939] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:13.238051 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a91c3-00ce-4e44-a7b8-9e8d6d0c9462 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:13.242289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-37fa9058-dd0b-4e35-bcc6-19ea7ad28202 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c2a1399fb60b4884ad3f6246df9be5ad.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c2a1399fb60b4884ad3f6246df9be5ad. May 19 20:19:13.242289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:13.242289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:19:13.242289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:19:13.242289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:19:13.242289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:19:13.242289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:19:13.242289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:19:13.242289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:19:13.242289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application conn.query(q) May 19 20:19:13.242289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:19:13.242289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:19:13.242289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application result.read() May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 20:19:13.243403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application next(self.gen) May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application yield resource May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application next(self.gen) May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise self.value May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application session.commit() May 19 20:19:13.244793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self.session.flush() May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application compat.raise_( May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:19:13.246085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:19:13.248010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:19:13.248010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:19:13.248010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:19:13.248010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:19:13.248010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 20:19:13.248010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 20:19:13.248010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:19:13.248010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:19:13.248010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:19:13.248010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:19:13.248010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:19:13.248010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:19:13.248010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:19:13.248010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:19:13.248010 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application conn.query(q) May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application result.read() May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:19:13.249812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:19:13.251904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:19:13.251904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:19:13.251904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:19:13.251904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:19:13.251904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:19:13.251904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:19:13.251904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:19:13.251904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 20:19:13.251904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [parameters: {'id': 'c2a1399fb60b4884ad3f6246df9be5ad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 20:19:13.251904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:19:13.251904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:13.251904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:13.251904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:13.251904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:13.251904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:13.255401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:13.257701 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:13.257701 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 20:19:13.257701 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 20:19:13.257701 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:13.257701 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:13.257701 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 20:19:13.257701 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 20:19:13.257701 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c2a1399fb60b4884ad3f6246df9be5ad. May 19 20:19:13.257701 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:13.259238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 471/940] 10.222.0.84 () {60 vars in 1140 bytes} [Thu May 19 20:19:13 2022] POST /identity/v3/projects => generated 262 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:13.261362 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa9058-dd0b-4e35-bcc6-19ea7ad28202 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:13.316958 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:13.350039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a91c3-00ce-4e44-a7b8-9e8d6d0c9462 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:13.385612 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa9058-dd0b-4e35-bcc6-19ea7ad28202 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:13.388425 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333b5ae-b109-4d69-8b75-efca2cd6285f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:13.389437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333b5ae-b109-4d69-8b75-efca2cd6285f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:13.389620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333b5ae-b109-4d69-8b75-efca2cd6285f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:13.390001 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333b5ae-b109-4d69-8b75-efca2cd6285f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:13.397732 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333b5ae-b109-4d69-8b75-efca2cd6285f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:13.399380 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333b5ae-b109-4d69-8b75-efca2cd6285f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-7333b5ae-b109-4d69-8b75-efca2cd6285f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:19:13.415119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:13.416255 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:13.417398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:13.417398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:13.417398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:19:13.417398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:19:13.417398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:19:13.417398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:19:13.417398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:19:13.417398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:19:13.417398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:19:13.417398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:13.421707 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 470/941] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:19:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:13.422771 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333b5ae-b109-4d69-8b75-efca2cd6285f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:13.428299 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7fb1b1-23d2-49ea-b794-53f5ced1a067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:13.429730 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7fb1b1-23d2-49ea-b794-53f5ced1a067 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:13.430197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7fb1b1-23d2-49ea-b794-53f5ced1a067 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:13.430696 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7fb1b1-23d2-49ea-b794-53f5ced1a067 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:13.438465 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:13.445485 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7fb1b1-23d2-49ea-b794-53f5ced1a067 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:13.448333 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7fb1b1-23d2-49ea-b794-53f5ced1a067 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:13.468683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-ed7fb1b1-23d2-49ea-b794-53f5ced1a067 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:13.468683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:13.468683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:13.468683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:13.468683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:13.468683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:13.468683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:13.468683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:13.468683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:13.468683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:13.468683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:13.468683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:13.468683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:19:13.468683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:19:13.468683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:13.469811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:13.470960 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:19:13.470960 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:19:13.470960 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:19:13.470960 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:19:13.470960 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:19:13.470960 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:19:13.470960 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:13.470960 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:13.472082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 472/942] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:19:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:13.479381 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7fb1b1-23d2-49ea-b794-53f5ced1a067 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:13.557558 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333b5ae-b109-4d69-8b75-efca2cd6285f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:13.591861 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7fb1b1-23d2-49ea-b794-53f5ced1a067 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:13.592304 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:13.597958 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c35ddf-93c4-47bc-b364-bbafa9b74ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:13.598886 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c35ddf-93c4-47bc-b364-bbafa9b74ac7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:13.600109 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c35ddf-93c4-47bc-b364-bbafa9b74ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:13.600109 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c35ddf-93c4-47bc-b364-bbafa9b74ac7 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:13.600477 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c35ddf-93c4-47bc-b364-bbafa9b74ac7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:13.600800 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c35ddf-93c4-47bc-b364-bbafa9b74ac7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:13.603326 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c35ddf-93c4-47bc-b364-bbafa9b74ac7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:13.623265 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 471/943] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 20:19:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:13.631494 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c35ddf-93c4-47bc-b364-bbafa9b74ac7 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:13.634563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327312c-1b3b-436f-83be-103d10312c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:13.635283 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327312c-1b3b-436f-83be-103d10312c41 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:13.635565 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327312c-1b3b-436f-83be-103d10312c41 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:13.635816 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327312c-1b3b-436f-83be-103d10312c41 None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:13.636583 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327312c-1b3b-436f-83be-103d10312c41 None admin] RBAC: Adding query filter params (name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:13.636895 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327312c-1b3b-436f-83be-103d10312c41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:13.638383 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327312c-1b3b-436f-83be-103d10312c41 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:13.666373 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 473/944] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:19:13 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:13.752050 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c35ddf-93c4-47bc-b364-bbafa9b74ac7 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:13.791652 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe1db1-bc7a-4a80-a2b9-6f9e6618c36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:13.792469 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe1db1-bc7a-4a80-a2b9-6f9e6618c36a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:13.792746 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe1db1-bc7a-4a80-a2b9-6f9e6618c36a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:13.793017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe1db1-bc7a-4a80-a2b9-6f9e6618c36a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:13.805345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe1db1-bc7a-4a80-a2b9-6f9e6618c36a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:13.806890 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe1db1-bc7a-4a80-a2b9-6f9e6618c36a None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:13.826795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-2ffe1db1-bc7a-4a80-a2b9-6f9e6618c36a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:13.826795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:13.826795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:13.826795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:13.826795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:13.826795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:13.826795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:13.826795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:13.826795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:13.826795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:13.826795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:13.826795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:13.826795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:13.826795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:13.826795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:13.827935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:13.829621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:13.829621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:13.829621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:13.829621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:13.829621 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 472/945] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:13.833845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327312c-1b3b-436f-83be-103d10312c41 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:13.899336 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:13.947174 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327312c-1b3b-436f-83be-103d10312c41 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:13.983453 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad3ffd-efd9-40db-ac63-8d250e9a7686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:13.984144 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad3ffd-efd9-40db-ac63-8d250e9a7686 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:13.984425 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad3ffd-efd9-40db-ac63-8d250e9a7686 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:13.984702 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad3ffd-efd9-40db-ac63-8d250e9a7686 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:13.985437 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad3ffd-efd9-40db-ac63-8d250e9a7686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:13.985744 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad3ffd-efd9-40db-ac63-8d250e9a7686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:13.987160 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad3ffd-efd9-40db-ac63-8d250e9a7686 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:14.000940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 474/946] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:14.006961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe1db1-bc7a-4a80-a2b9-6f9e6618c36a None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:14.056124 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:14.121467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe1db1-bc7a-4a80-a2b9-6f9e6618c36a None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:14.158201 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7e99e-9d05-4b16-92fb-37c937f2b580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:14.159047 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7e99e-9d05-4b16-92fb-37c937f2b580 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:14.159324 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7e99e-9d05-4b16-92fb-37c937f2b580 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:14.159595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7e99e-9d05-4b16-92fb-37c937f2b580 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:14.167898 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7e99e-9d05-4b16-92fb-37c937f2b580 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:14.169570 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7e99e-9d05-4b16-92fb-37c937f2b580 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:14.185341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-b0d7e99e-9d05-4b16-92fb-37c937f2b580 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:14.185341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:14.185341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:14.185341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:14.185341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:14.185341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:14.185341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:14.185341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:14.185341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:14.185341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:14.185341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:14.185341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:14.185341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:19:14.185341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:19:14.185341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:14.186562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:14.188273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:19:14.188273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:19:14.188273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:19:14.188273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:19:14.188273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:19:14.188273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:19:14.188273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:14.188273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:14.188273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 473/947] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:19:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:14.192738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad3ffd-efd9-40db-ac63-8d250e9a7686 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:14.222524 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d343a1-4a8f-4434-a6b2-2032795c188e None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:14.222718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d343a1-4a8f-4434-a6b2-2032795c188e None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:14.222988 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d343a1-4a8f-4434-a6b2-2032795c188e None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:14.224394 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 474/948] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:19:14.229652 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38314b6a-a40a-4cdb-8227-e64cb9303fa8 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:14.229937 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38314b6a-a40a-4cdb-8227-e64cb9303fa8 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:14.230214 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38314b6a-a40a-4cdb-8227-e64cb9303fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:14.247770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-38314b6a-a40a-4cdb-8227-e64cb9303fa8 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:14.248959 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-38314b6a-a40a-4cdb-8227-e64cb9303fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:14.301904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad3ffd-efd9-40db-ac63-8d250e9a7686 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:14.320393 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:14.325945 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 475/949] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:14.341173 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fd2df-6c7e-4c57-9c6a-0b9602087b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:14.342215 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd2df-6c7e-4c57-9c6a-0b9602087b39 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:14.342762 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd2df-6c7e-4c57-9c6a-0b9602087b39 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:14.343251 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd2df-6c7e-4c57-9c6a-0b9602087b39 None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:14.344333 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fd2df-6c7e-4c57-9c6a-0b9602087b39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:14.344897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fd2df-6c7e-4c57-9c6a-0b9602087b39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:14.345307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38314b6a-a40a-4cdb-8227-e64cb9303fa8 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:14.346868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fd2df-6c7e-4c57-9c6a-0b9602087b39 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:14.364585 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 475/950] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:19:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:14.366487 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff5b01-c572-4c67-b537-7aab46402468 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:14.366968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff5b01-c572-4c67-b537-7aab46402468 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:14.367327 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff5b01-c572-4c67-b537-7aab46402468 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:14.368910 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 476/951] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:14.371316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff5b01-c572-4c67-b537-7aab46402468 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:14.380954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:14.471640 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38314b6a-a40a-4cdb-8227-e64cb9303fa8 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:14.491012 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff5b01-c572-4c67-b537-7aab46402468 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:14.510819 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1142bf-351b-48bb-9b26-5cd3eb286803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:14.511648 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1142bf-351b-48bb-9b26-5cd3eb286803 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:14.511932 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1142bf-351b-48bb-9b26-5cd3eb286803 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:14.512212 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1142bf-351b-48bb-9b26-5cd3eb286803 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:14.512875 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1142bf-351b-48bb-9b26-5cd3eb286803 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:14.514291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1142bf-351b-48bb-9b26-5cd3eb286803 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-3d1142bf-351b-48bb-9b26-5cd3eb286803 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:14.527466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:14.528705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:14.529772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:14.529772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:14.529772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:14.529772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:14.529772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:14.529772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:14.529772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:14.529772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:14.529772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:14.529772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:14.529772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:14.529772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:14.529772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:19:14.529772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:14.529772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 476/952] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:19:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:14.531165 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44d39a-4f1b-41f8-ae33-bf9dcdb76898 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:14.531165 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44d39a-4f1b-41f8-ae33-bf9dcdb76898 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:14.531165 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44d39a-4f1b-41f8-ae33-bf9dcdb76898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:14.534451 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcde37-2c75-44fd-8064-bfa78c6a2aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:14.535848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcde37-2c75-44fd-8064-bfa78c6a2aa9 None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:14.537223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcde37-2c75-44fd-8064-bfa78c6a2aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:14.537758 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcde37-2c75-44fd-8064-bfa78c6a2aa9 None admin] PATH_INFO: `/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/265a6cde297448adb102da0773e366fb/roles/943b7648db36455b8073c12e3b45374d` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:14.540519 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcde37-2c75-44fd-8064-bfa78c6a2aa9 None admin] RBAC: Authorizing `identity:create_grant(project_id=59555e7adac54eddb47d1ef18ef870c4, user_id=265a6cde297448adb102da0773e366fb, role_id=943b7648db36455b8073c12e3b45374d)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:14.542714 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcde37-2c75-44fd-8064-bfa78c6a2aa9 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:14.547570 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-2b44d39a-4f1b-41f8-ae33-bf9dcdb76898 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:14.548896 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-2b44d39a-4f1b-41f8-ae33-bf9dcdb76898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:14.569775 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 477/953] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:19:14 2022] PUT /identity/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/265a6cde297448adb102da0773e366fb/roles/943b7648db36455b8073c12e3b45374d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:19:14.569775 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcde37-2c75-44fd-8064-bfa78c6a2aa9 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:14.612738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:14.642994 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 477/954] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:14.665402 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44d39a-4f1b-41f8-ae33-bf9dcdb76898 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:14.691518 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcde37-2c75-44fd-8064-bfa78c6a2aa9 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:14.696670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:14.731306 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b599b7-360c-491d-8bd6-8a072c605e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:14.732378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b599b7-360c-491d-8bd6-8a072c605e32 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:14.732843 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b599b7-360c-491d-8bd6-8a072c605e32 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:14.733296 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b599b7-360c-491d-8bd6-8a072c605e32 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:14.734202 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b599b7-360c-491d-8bd6-8a072c605e32 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:14.734778 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b599b7-360c-491d-8bd6-8a072c605e32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:14.736377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b599b7-360c-491d-8bd6-8a072c605e32 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:14.747714 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 478/955] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:19:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:14.749358 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b599b7-360c-491d-8bd6-8a072c605e32 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:14.787616 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44d39a-4f1b-41f8-ae33-bf9dcdb76898 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:14.828431 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987d8b4-594e-40c6-811e-bae1d8d9e84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:14.828752 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987d8b4-594e-40c6-811e-bae1d8d9e84b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:14.829040 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987d8b4-594e-40c6-811e-bae1d8d9e84b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:14.829316 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987d8b4-594e-40c6-811e-bae1d8d9e84b None admin] PATH_INFO: `/v3/services/compute` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:14.829975 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987d8b4-594e-40c6-811e-bae1d8d9e84b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:14.831407 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987d8b4-594e-40c6-811e-bae1d8d9e84b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-8987d8b4-594e-40c6-811e-bae1d8d9e84b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:14.843995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:14.845697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:14.847018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:14.847018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:14.847018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:14.847018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:14.847018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:14.847018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:14.847018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:14.847018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:14.847018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:14.847018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:14.847018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:14.847018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:14.847018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:19:14.847018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:14.847018 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 478/956] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:19:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:14.851540 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987d8b4-594e-40c6-811e-bae1d8d9e84b None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:14.863418 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:14.870235 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b599b7-360c-491d-8bd6-8a072c605e32 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:14.908770 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ab08b-66c1-414d-8629-127abd2c1f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:14.910383 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ab08b-66c1-414d-8629-127abd2c1f90 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:14.910842 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ab08b-66c1-414d-8629-127abd2c1f90 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:14.911339 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ab08b-66c1-414d-8629-127abd2c1f90 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:14.912203 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ab08b-66c1-414d-8629-127abd2c1f90 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:14.912681 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ab08b-66c1-414d-8629-127abd2c1f90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:14.914573 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ab08b-66c1-414d-8629-127abd2c1f90 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:14.925191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 479/957] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:19:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:14.926611 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ab08b-66c1-414d-8629-127abd2c1f90 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:14.962616 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:14.983147 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987d8b4-594e-40c6-811e-bae1d8d9e84b None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:15.027630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a7b40-9a1c-46b3-aeab-702a32fc448a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:15.028289 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a7b40-9a1c-46b3-aeab-702a32fc448a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:15.028601 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a7b40-9a1c-46b3-aeab-702a32fc448a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:15.028951 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a7b40-9a1c-46b3-aeab-702a32fc448a None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:15.029734 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a7b40-9a1c-46b3-aeab-702a32fc448a None admin] RBAC: Adding query filter params (name=compute) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:15.030109 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a7b40-9a1c-46b3-aeab-702a32fc448a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:15.031609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a7b40-9a1c-46b3-aeab-702a32fc448a None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:15.038400 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 479/958] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:19:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:15.043711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a7b40-9a1c-46b3-aeab-702a32fc448a None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:15.050023 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ab08b-66c1-414d-8629-127abd2c1f90 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:15.097259 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0dd274-5bd3-4a1d-8c20-4a510f9f2d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:15.098419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0dd274-5bd3-4a1d-8c20-4a510f9f2d7c None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:15.099365 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0dd274-5bd3-4a1d-8c20-4a510f9f2d7c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:15.099858 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0dd274-5bd3-4a1d-8c20-4a510f9f2d7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:15.100880 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0dd274-5bd3-4a1d-8c20-4a510f9f2d7c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:15.103173 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0dd274-5bd3-4a1d-8c20-4a510f9f2d7c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:15.132655 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 480/959] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:19:14 2022] POST /identity/v3/endpoints => generated 349 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:19:15.175784 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a7b40-9a1c-46b3-aeab-702a32fc448a None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:15.221376 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8b59d-43ec-4bb5-840e-4c6d7ab83a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:15.222110 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8b59d-43ec-4bb5-840e-4c6d7ab83a29 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:15.222432 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8b59d-43ec-4bb5-840e-4c6d7ab83a29 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:15.222713 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8b59d-43ec-4bb5-840e-4c6d7ab83a29 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:15.223426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8b59d-43ec-4bb5-840e-4c6d7ab83a29 None admin] RBAC: Adding query filter params (type=compute) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:15.223738 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8b59d-43ec-4bb5-840e-4c6d7ab83a29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:15.225212 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8b59d-43ec-4bb5-840e-4c6d7ab83a29 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:15.231819 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 480/960] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:19:15 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:16.089957 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca63fb-0348-49e4-9bc0-d27c170b88f9 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:16.090556 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca63fb-0348-49e4-9bc0-d27c170b88f9 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:16.090556 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca63fb-0348-49e4-9bc0-d27c170b88f9 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:16.092176 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 481/961] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:16.097754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857b55c-6ae1-44b6-9289-a5d360d1386d None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:16.098212 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857b55c-6ae1-44b6-9289-a5d360d1386d None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:16.098440 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857b55c-6ae1-44b6-9289-a5d360d1386d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:16.115570 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-4857b55c-6ae1-44b6-9289-a5d360d1386d None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:16.116853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-4857b55c-6ae1-44b6-9289-a5d360d1386d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:16.126220 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:16.184695 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 481/962] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:16.209904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca63fb-0348-49e4-9bc0-d27c170b88f9 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:16.232281 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:16.314084 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca63fb-0348-49e4-9bc0-d27c170b88f9 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:16.351397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decadab5-e4b6-4b3c-8652-36671ca8cd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:16.352056 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decadab5-e4b6-4b3c-8652-36671ca8cd7f None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:16.352342 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decadab5-e4b6-4b3c-8652-36671ca8cd7f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:16.352626 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decadab5-e4b6-4b3c-8652-36671ca8cd7f None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:16.353353 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decadab5-e4b6-4b3c-8652-36671ca8cd7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:16.354834 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decadab5-e4b6-4b3c-8652-36671ca8cd7f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:16.371651 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.sql.core [None req-decadab5-e4b6-4b3c-8652-36671ca8cd7f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:19:16.371651 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 20:19:16.371651 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [parameters: {'user_id': '2031df9f97d94006b5263a015c9ef611', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 20:19:16.371651 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=277682) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:19:16.379602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-decadab5-e4b6-4b3c-8652-36671ca8cd7f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 20:19:16.379602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:16.379602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:19:16.379602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:19:16.379602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:19:16.379602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:19:16.379602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:19:16.379602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:19:16.379602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:19:16.379602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application conn.query(q) May 19 20:19:16.379602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:19:16.379602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:19:16.379602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:19:16.379602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application result.read() May 19 20:19:16.379602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:19:16.380636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application next(self.gen) May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application yield resource May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application next(self.gen) May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise self.value May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application session.commit() May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:19:16.382617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self.session.flush() May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application compat.raise_( May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:19:16.384147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:19:16.386618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application conn.query(q) May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application result.read() May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:19:16.387684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:19:16.388674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:19:16.388674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:19:16.388674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:19:16.388674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:19:16.388674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:19:16.388674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 20:19:16.388674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [parameters: {'user_id': '2031df9f97d94006b5263a015c9ef611', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 20:19:16.388674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:19:16.388674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:16.388674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:16.388674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:16.388674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:16.388674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:16.388674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:16.388674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 20:19:16.389676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 20:19:16.390794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 20:19:16.390794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 20:19:16.390794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:19:16.390794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:19:16.390794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 20:19:16.390794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:16.390794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 482/963] 10.222.0.84 () {60 vars in 1131 bytes} [Thu May 19 20:19:16 2022] POST /identity/v3/users => generated 160 bytes in 172 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:16.392412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857b55c-6ae1-44b6-9289-a5d360d1386d None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:16.506975 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857b55c-6ae1-44b6-9289-a5d360d1386d None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:16.543272 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795c529-2b16-4d06-b258-529f5bb04080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:16.544089 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795c529-2b16-4d06-b258-529f5bb04080 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:16.544370 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795c529-2b16-4d06-b258-529f5bb04080 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:16.544648 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795c529-2b16-4d06-b258-529f5bb04080 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:16.552189 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795c529-2b16-4d06-b258-529f5bb04080 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:16.553775 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795c529-2b16-4d06-b258-529f5bb04080 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-f795c529-2b16-4d06-b258-529f5bb04080 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:19:16.568624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:16.569990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:16.571172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:16.571172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:16.571172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:19:16.571172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:19:16.571172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:19:16.571172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:19:16.571172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:19:16.571172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:19:16.571172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:19:16.571172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:16.571172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 482/964] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:19:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:16.576049 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decadab5-e4b6-4b3c-8652-36671ca8cd7f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:16.679115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:16.687456 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decadab5-e4b6-4b3c-8652-36671ca8cd7f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:16.725870 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8f40b-d952-4de1-a0e7-54c4fe024a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:16.726739 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8f40b-d952-4de1-a0e7-54c4fe024a6a None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:16.727028 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8f40b-d952-4de1-a0e7-54c4fe024a6a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:16.727313 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8f40b-d952-4de1-a0e7-54c4fe024a6a None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:16.728127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8f40b-d952-4de1-a0e7-54c4fe024a6a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:16.728439 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8f40b-d952-4de1-a0e7-54c4fe024a6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:16.730107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8f40b-d952-4de1-a0e7-54c4fe024a6a None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:16.770914 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 483/965] 10.222.0.84 () {56 vars in 1101 bytes} [Thu May 19 20:19:16 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:16.991774 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aea1a7-efa8-481f-b21d-5836e6316255 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:16.992168 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aea1a7-efa8-481f-b21d-5836e6316255 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:16.992360 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aea1a7-efa8-481f-b21d-5836e6316255 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:16.993921 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 483/966] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:16.999722 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4ed8c-70de-4ccf-99c9-c06215200327 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:17.000007 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4ed8c-70de-4ccf-99c9-c06215200327 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:17.000283 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4ed8c-70de-4ccf-99c9-c06215200327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:17.021405 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-cde4ed8c-70de-4ccf-99c9-c06215200327 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:17.023679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-cde4ed8c-70de-4ccf-99c9-c06215200327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:17.100356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 484/967] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:17.124367 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aea1a7-efa8-481f-b21d-5836e6316255 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:17.128607 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:17.231598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aea1a7-efa8-481f-b21d-5836e6316255 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:17.269656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a1fb6-801e-4d76-b287-2fbbcaeb687a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:17.270433 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a1fb6-801e-4d76-b287-2fbbcaeb687a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:17.270718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a1fb6-801e-4d76-b287-2fbbcaeb687a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:17.270992 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a1fb6-801e-4d76-b287-2fbbcaeb687a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:17.282997 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a1fb6-801e-4d76-b287-2fbbcaeb687a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-4c2a1fb6-801e-4d76-b287-2fbbcaeb687a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:19:17.291490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:17.292610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:17.293630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:17.293630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:17.293630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:17.293630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:17.293630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:19:17.293630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:19:17.293630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:19:17.293630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:19:17.293630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:19:17.293630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:19:17.293630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:19:17.293630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:17.297825 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 484/968] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 20:19:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:17.298876 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4ed8c-70de-4ccf-99c9-c06215200327 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:17.400541 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:17.439874 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4ed8c-70de-4ccf-99c9-c06215200327 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:17.486436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4d787-2a55-4ac3-a1c7-4e590368756a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:17.487127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4d787-2a55-4ac3-a1c7-4e590368756a None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:17.487392 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4d787-2a55-4ac3-a1c7-4e590368756a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:17.487687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4d787-2a55-4ac3-a1c7-4e590368756a None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:17.488443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4d787-2a55-4ac3-a1c7-4e590368756a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:17.488755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4d787-2a55-4ac3-a1c7-4e590368756a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:17.490120 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4d787-2a55-4ac3-a1c7-4e590368756a None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:17.505630 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 485/969] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 20:19:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:17.506925 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a1fb6-801e-4d76-b287-2fbbcaeb687a None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:17.533786 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fbe82-5430-40db-8c25-f65b2e68497a None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:17.534102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fbe82-5430-40db-8c25-f65b2e68497a None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:17.534823 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fbe82-5430-40db-8c25-f65b2e68497a None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:17.536831 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 486/970] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:17.544032 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925afb2-647d-49e4-92d7-d65ba40d6e76 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:17.544324 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925afb2-647d-49e4-92d7-d65ba40d6e76 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:17.544608 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925afb2-647d-49e4-92d7-d65ba40d6e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:17.564840 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-4925afb2-647d-49e4-92d7-d65ba40d6e76 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:17.566058 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-4925afb2-647d-49e4-92d7-d65ba40d6e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:17.574474 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:17.615198 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a1fb6-801e-4d76-b287-2fbbcaeb687a None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:17.636390 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 487/971] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:17.637734 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7762758-3f8e-4452-8138-842a8ca0d06b None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:17.638027 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7762758-3f8e-4452-8138-842a8ca0d06b None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:17.638333 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7762758-3f8e-4452-8138-842a8ca0d06b None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:17.639629 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 488/972] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:17.645022 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89168d1-e363-423e-8e36-e97954b2e4e8 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:17.645315 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89168d1-e363-423e-8e36-e97954b2e4e8 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:17.645599 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89168d1-e363-423e-8e36-e97954b2e4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:17.653520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32090d-7c58-4e31-97a6-85b239df923c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:17.654425 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32090d-7c58-4e31-97a6-85b239df923c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:17.654703 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32090d-7c58-4e31-97a6-85b239df923c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:17.654976 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32090d-7c58-4e31-97a6-85b239df923c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:17.666066 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32090d-7c58-4e31-97a6-85b239df923c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:17.667586 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32090d-7c58-4e31-97a6-85b239df923c None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:17.667830 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-d89168d1-e363-423e-8e36-e97954b2e4e8 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:17.669047 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-d89168d1-e363-423e-8e36-e97954b2e4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:17.685122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-1e32090d-7c58-4e31-97a6-85b239df923c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:17.685122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:17.685122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:17.685122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:17.685122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:17.685122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:17.685122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:17.685122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:17.685122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:17.685122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:17.685122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:17.685122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:17.685122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:17.685122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:17.685122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:17.686312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:17.692566 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:17.692566 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:17.692566 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:17.692566 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:17.692566 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 485/973] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:17.692566 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32090d-7c58-4e31-97a6-85b239df923c None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:17.727713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:17.739297 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 489/974] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:19:17.740366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89168d1-e363-423e-8e36-e97954b2e4e8 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:17.794850 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32090d-7c58-4e31-97a6-85b239df923c None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:17.811321 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:17.830386 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af4103-82b5-4ef3-9212-dc2945d3748b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:17.831200 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af4103-82b5-4ef3-9212-dc2945d3748b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:17.831509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af4103-82b5-4ef3-9212-dc2945d3748b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:17.831788 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af4103-82b5-4ef3-9212-dc2945d3748b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:17.832444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af4103-82b5-4ef3-9212-dc2945d3748b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:17.833844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af4103-82b5-4ef3-9212-dc2945d3748b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:17.843754 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89168d1-e363-423e-8e36-e97954b2e4e8 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-e4af4103-82b5-4ef3-9212-dc2945d3748b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:17.846919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:17.848243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:17.849266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:17.849266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:17.849266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:17.849266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:17.849266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:17.849266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:17.849266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:17.849266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:17.849266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:17.849266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:17.849266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:17.849266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:17.849266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:19:17.849266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:17.849266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 486/975] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:19:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:17.850367 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af4103-82b5-4ef3-9212-dc2945d3748b None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:17.878725 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9e22c-5927-4c03-8b4c-031fcfffacf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:17.879404 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9e22c-5927-4c03-8b4c-031fcfffacf9 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:17.879689 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9e22c-5927-4c03-8b4c-031fcfffacf9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:17.879975 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9e22c-5927-4c03-8b4c-031fcfffacf9 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:17.880704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9e22c-5927-4c03-8b4c-031fcfffacf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:17.881016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9e22c-5927-4c03-8b4c-031fcfffacf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:17.882972 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9e22c-5927-4c03-8b4c-031fcfffacf9 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:17.899012 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 490/976] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:17.899012 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9e22c-5927-4c03-8b4c-031fcfffacf9 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:17.950100 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:17.954779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af4103-82b5-4ef3-9212-dc2945d3748b None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:17.989879 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4806cc-d79a-4caa-a9a0-22d20d4bee95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:17.990613 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4806cc-d79a-4caa-a9a0-22d20d4bee95 None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:17.990926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4806cc-d79a-4caa-a9a0-22d20d4bee95 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:17.991697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4806cc-d79a-4caa-a9a0-22d20d4bee95 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:17.992428 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4806cc-d79a-4caa-a9a0-22d20d4bee95 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:17.993831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4806cc-d79a-4caa-a9a0-22d20d4bee95 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:18.002075 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9e22c-5927-4c03-8b4c-031fcfffacf9 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:18.011105 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 487/977] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:19:17 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:19:18.011940 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4806cc-d79a-4caa-a9a0-22d20d4bee95 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:18.051159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e473b-1e96-4fc4-bd3f-e17a9d3d4541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:18.051779 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e473b-1e96-4fc4-bd3f-e17a9d3d4541 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:18.052079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e473b-1e96-4fc4-bd3f-e17a9d3d4541 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:18.052345 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e473b-1e96-4fc4-bd3f-e17a9d3d4541 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:18.053208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e473b-1e96-4fc4-bd3f-e17a9d3d4541 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:18.053663 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e473b-1e96-4fc4-bd3f-e17a9d3d4541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:18.055947 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e473b-1e96-4fc4-bd3f-e17a9d3d4541 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:18.062073 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 491/978] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:19:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:18.062774 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:18.068463 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e473b-1e96-4fc4-bd3f-e17a9d3d4541 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:18.138283 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4806cc-d79a-4caa-a9a0-22d20d4bee95 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:18.178801 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab23d2d-fd22-4947-8593-4afc80d5c706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:18.180145 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab23d2d-fd22-4947-8593-4afc80d5c706 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:18.180427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab23d2d-fd22-4947-8593-4afc80d5c706 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:18.180701 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab23d2d-fd22-4947-8593-4afc80d5c706 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:18.183147 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e473b-1e96-4fc4-bd3f-e17a9d3d4541 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:18.189611 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab23d2d-fd22-4947-8593-4afc80d5c706 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:18.191289 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab23d2d-fd22-4947-8593-4afc80d5c706 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:18.197425 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-4ab23d2d-fd22-4947-8593-4afc80d5c706 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:19:18.207479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:18.208644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:18.209844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:18.209844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:18.209844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:19:18.209844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:19:18.209844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:19:18.209844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:19:18.209844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:19:18.209844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:19:18.209844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:19:18.209844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:18.209844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 488/979] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:19:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:18.215150 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab23d2d-fd22-4947-8593-4afc80d5c706 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:18.228500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc230c-635d-4f61-aba4-1ab792fc5cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:18.229177 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc230c-635d-4f61-aba4-1ab792fc5cbe None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:18.230106 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc230c-635d-4f61-aba4-1ab792fc5cbe None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:18.230106 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc230c-635d-4f61-aba4-1ab792fc5cbe None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:18.230486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc230c-635d-4f61-aba4-1ab792fc5cbe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:18.231072 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc230c-635d-4f61-aba4-1ab792fc5cbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:18.232291 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc230c-635d-4f61-aba4-1ab792fc5cbe None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:18.238200 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 492/980] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:19:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:18.244591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc230c-635d-4f61-aba4-1ab792fc5cbe None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:18.328134 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:18.331850 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab23d2d-fd22-4947-8593-4afc80d5c706 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:18.349514 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc230c-635d-4f61-aba4-1ab792fc5cbe None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:18.368661 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93b52c-d49e-479e-a6f8-51e7d654f6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:18.369345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93b52c-d49e-479e-a6f8-51e7d654f6d0 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:18.369625 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93b52c-d49e-479e-a6f8-51e7d654f6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:18.369906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93b52c-d49e-479e-a6f8-51e7d654f6d0 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:18.370780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93b52c-d49e-479e-a6f8-51e7d654f6d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:18.371096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93b52c-d49e-479e-a6f8-51e7d654f6d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:18.372549 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93b52c-d49e-479e-a6f8-51e7d654f6d0 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:18.389473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13981ce6-4b7c-469c-8b22-4ce68ef94e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:18.390137 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13981ce6-4b7c-469c-8b22-4ce68ef94e78 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:18.390445 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13981ce6-4b7c-469c-8b22-4ce68ef94e78 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:18.390724 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13981ce6-4b7c-469c-8b22-4ce68ef94e78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:18.391505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13981ce6-4b7c-469c-8b22-4ce68ef94e78 None admin] RBAC: Adding query filter params (interface=admin, service_id=c21c70bdc50745e58e26d3bb808b8644, region_id=RegionOne) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:18.391812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13981ce6-4b7c-469c-8b22-4ce68ef94e78 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:18.393169 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13981ce6-4b7c-469c-8b22-4ce68ef94e78 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:18.395959 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 489/981] 10.222.0.84 () {56 vars in 1149 bytes} [Thu May 19 20:19:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:18.397132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93b52c-d49e-479e-a6f8-51e7d654f6d0 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:18.399164 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 493/982] 10.222.0.84 () {56 vars in 1249 bytes} [Thu May 19 20:19:18 2022] GET /identity/v3/endpoints?service_id=c21c70bdc50745e58e26d3bb808b8644&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:18.404414 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13981ce6-4b7c-469c-8b22-4ce68ef94e78 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:18.454786 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:18.510969 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93b52c-d49e-479e-a6f8-51e7d654f6d0 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:18.521433 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13981ce6-4b7c-469c-8b22-4ce68ef94e78 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:18.546483 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212217f4-beac-48fb-9501-871b471c33f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:18.547148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212217f4-beac-48fb-9501-871b471c33f1 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:18.547426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212217f4-beac-48fb-9501-871b471c33f1 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:18.547699 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212217f4-beac-48fb-9501-871b471c33f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:18.559925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c5582-91f1-47d4-91a8-53dae0e33487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:18.560872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c5582-91f1-47d4-91a8-53dae0e33487 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:18.561334 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c5582-91f1-47d4-91a8-53dae0e33487 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:18.561762 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c5582-91f1-47d4-91a8-53dae0e33487 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:18.562625 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c5582-91f1-47d4-91a8-53dae0e33487 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:18.564225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c5582-91f1-47d4-91a8-53dae0e33487 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:18.582366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 494/983] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:19:18 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:19:18.582590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:18.624307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212217f4-beac-48fb-9501-871b471c33f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:18.625787 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212217f4-beac-48fb-9501-871b471c33f1 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:18.741945 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 490/984] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:19:18 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:19:18.747936 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c5582-91f1-47d4-91a8-53dae0e33487 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:18.856678 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c5582-91f1-47d4-91a8-53dae0e33487 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:18.860003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:18.894075 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e6a97-d2fe-4bc1-93bd-fd48595d21a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:18.894914 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e6a97-d2fe-4bc1-93bd-fd48595d21a4 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:18.895199 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e6a97-d2fe-4bc1-93bd-fd48595d21a4 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:18.895476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e6a97-d2fe-4bc1-93bd-fd48595d21a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:18.907198 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e6a97-d2fe-4bc1-93bd-fd48595d21a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:18.908724 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e6a97-d2fe-4bc1-93bd-fd48595d21a4 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:18.926282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-133e6a97-d2fe-4bc1-93bd-fd48595d21a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:18.926282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:18.926282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:18.926282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:18.926282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:18.926282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:18.926282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:18.926282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:18.926282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:18.926282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:18.926282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:18.926282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:18.926282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:18.926282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:18.926282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:18.927335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:18.928460 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:18.928460 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:18.928460 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:18.928460 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:18.928460 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 495/985] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:18.935090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212217f4-beac-48fb-9501-871b471c33f1 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:18.996347 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:19.039733 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212217f4-beac-48fb-9501-871b471c33f1 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:19.074061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916610d-ee85-4d82-b3e5-784f51efd1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:19.074766 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916610d-ee85-4d82-b3e5-784f51efd1f5 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:19.075048 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916610d-ee85-4d82-b3e5-784f51efd1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:19.075323 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916610d-ee85-4d82-b3e5-784f51efd1f5 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:19.076038 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916610d-ee85-4d82-b3e5-784f51efd1f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:19.076345 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916610d-ee85-4d82-b3e5-784f51efd1f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:19.077723 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916610d-ee85-4d82-b3e5-784f51efd1f5 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:19.089944 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 491/986] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:19.095442 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e6a97-d2fe-4bc1-93bd-fd48595d21a4 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:19.203933 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e6a97-d2fe-4bc1-93bd-fd48595d21a4 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:19.240667 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d4f24-84e0-4708-8a2f-28506cac317d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:19.241688 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d4f24-84e0-4708-8a2f-28506cac317d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:19.241971 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d4f24-84e0-4708-8a2f-28506cac317d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:19.242244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d4f24-84e0-4708-8a2f-28506cac317d None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:19.249255 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d4f24-84e0-4708-8a2f-28506cac317d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:19.250719 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d4f24-84e0-4708-8a2f-28506cac317d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:19.266110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-060d4f24-84e0-4708-8a2f-28506cac317d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:19.266110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:19.266110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:19.266110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:19.266110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:19.266110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:19.266110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:19.266110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:19.266110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:19.266110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:19.266110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:19.266110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:19.266110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:19:19.266110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:19:19.266110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:19.267354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:19.268491 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:19:19.268491 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:19:19.268491 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:19:19.268491 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:19:19.268491 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:19:19.268491 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:19:19.268491 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:19.268491 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:19.268491 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 496/987] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:19:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:19.274398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916610d-ee85-4d82-b3e5-784f51efd1f5 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:19.379797 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916610d-ee85-4d82-b3e5-784f51efd1f5 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:19.397258 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:19.415854 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbb14d-e584-4cee-8f1e-40ee0901cb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:19.416530 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbb14d-e584-4cee-8f1e-40ee0901cb52 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:19.416805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbb14d-e584-4cee-8f1e-40ee0901cb52 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:19.417081 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbb14d-e584-4cee-8f1e-40ee0901cb52 None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:19.418355 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dbb14d-e584-4cee-8f1e-40ee0901cb52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:19.418355 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dbb14d-e584-4cee-8f1e-40ee0901cb52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:19.420611 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dbb14d-e584-4cee-8f1e-40ee0901cb52 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:19.435372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 492/988] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:19:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:19.441223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d4f24-84e0-4708-8a2f-28506cac317d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:19.460396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:19.544181 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d4f24-84e0-4708-8a2f-28506cac317d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:19.579510 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9858e-4116-4285-b326-c1e762302aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:19.580648 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9858e-4116-4285-b326-c1e762302aaf None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:19.581116 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9858e-4116-4285-b326-c1e762302aaf None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:19.581571 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9858e-4116-4285-b326-c1e762302aaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:19.582604 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9858e-4116-4285-b326-c1e762302aaf None admin] RBAC: Adding query filter params (role.id=943b7648db36455b8073c12e3b45374d, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=265a6cde297448adb102da0773e366fb) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:19.582919 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9858e-4116-4285-b326-c1e762302aaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:19.584353 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9858e-4116-4285-b326-c1e762302aaf None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:19.591095 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 497/989] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:19:19 2022] GET /identity/v3/role_assignments?user.id=265a6cde297448adb102da0773e366fb&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=943b7648db36455b8073c12e3b45374d => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:20.292069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb63b0-5f8e-4f22-a700-40d524bd9ee4 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:20.292673 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb63b0-5f8e-4f22-a700-40d524bd9ee4 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:20.292673 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb63b0-5f8e-4f22-a700-40d524bd9ee4 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:20.294158 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 493/990] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:20.299467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ff284-b245-4d9c-89b2-2e53ab6737a3 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:20.299912 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ff284-b245-4d9c-89b2-2e53ab6737a3 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:20.300012 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ff284-b245-4d9c-89b2-2e53ab6737a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:20.317980 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-ef6ff284-b245-4d9c-89b2-2e53ab6737a3 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:20.319263 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-ef6ff284-b245-4d9c-89b2-2e53ab6737a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:20.390852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 498/991] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:20.414447 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb63b0-5f8e-4f22-a700-40d524bd9ee4 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:20.530113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb63b0-5f8e-4f22-a700-40d524bd9ee4 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:20.566637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b78b0b-9379-4ed3-b137-a93c1480460d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:20.567361 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b78b0b-9379-4ed3-b137-a93c1480460d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:20.567646 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b78b0b-9379-4ed3-b137-a93c1480460d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:20.567923 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b78b0b-9379-4ed3-b137-a93c1480460d None admin] PATH_INFO: `/v3/services/compute` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:20.568579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b78b0b-9379-4ed3-b137-a93c1480460d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:20.570216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b78b0b-9379-4ed3-b137-a93c1480460d None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-07b78b0b-9379-4ed3-b137-a93c1480460d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:20.582119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:20.583291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:20.584307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:20.584307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:20.584307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:20.584307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:20.584307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:20.584307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:20.584307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:20.584307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:20.584307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:20.584307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:20.584307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:20.584307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:20.584307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:19:20.584307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:20.586631 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 494/992] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:19:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:20.589955 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ff284-b245-4d9c-89b2-2e53ab6737a3 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:20.602622 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:20.696162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ff284-b245-4d9c-89b2-2e53ab6737a3 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:20.737613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80d7f4-3219-4073-b430-d0513ce4061e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:20.738336 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80d7f4-3219-4073-b430-d0513ce4061e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:20.738630 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80d7f4-3219-4073-b430-d0513ce4061e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:20.738907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80d7f4-3219-4073-b430-d0513ce4061e None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:20.739633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80d7f4-3219-4073-b430-d0513ce4061e None admin] RBAC: Adding query filter params (name=compute) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:20.739949 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80d7f4-3219-4073-b430-d0513ce4061e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:20.741628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80d7f4-3219-4073-b430-d0513ce4061e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:20.747528 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 499/993] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:19:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:20.753117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b78b0b-9379-4ed3-b137-a93c1480460d None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:20.798683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:20.857647 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b78b0b-9379-4ed3-b137-a93c1480460d None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:20.892691 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a3601-df90-45c5-945d-50b6b06e50bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:20.893367 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a3601-df90-45c5-945d-50b6b06e50bb None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:20.893656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a3601-df90-45c5-945d-50b6b06e50bb None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:20.893932 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a3601-df90-45c5-945d-50b6b06e50bb None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:20.894679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a3601-df90-45c5-945d-50b6b06e50bb None admin] RBAC: Adding query filter params (type=compute) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:20.894990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a3601-df90-45c5-945d-50b6b06e50bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:20.896383 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a3601-df90-45c5-945d-50b6b06e50bb None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:20.898527 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf509e-1c31-452a-a7fb-19ea9f2376d4 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:20.898745 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf509e-1c31-452a-a7fb-19ea9f2376d4 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:20.899033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf509e-1c31-452a-a7fb-19ea9f2376d4 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:20.900578 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 500/994] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:19:20.902770 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 495/995] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:19:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:20.905812 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5b4bf-5f21-4555-aa96-c696de5b319d None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:20.906100 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5b4bf-5f21-4555-aa96-c696de5b319d None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:20.906421 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5b4bf-5f21-4555-aa96-c696de5b319d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:20.908313 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a3601-df90-45c5-945d-50b6b06e50bb None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:20.924375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-9be5b4bf-5f21-4555-aa96-c696de5b319d None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:20.925567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-9be5b4bf-5f21-4555-aa96-c696de5b319d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:20.998358 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 501/996] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:21.016157 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a3601-df90-45c5-945d-50b6b06e50bb None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:21.021872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5b4bf-5f21-4555-aa96-c696de5b319d None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:21.023658 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:21.025942 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:21.056328 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471629e5-a724-4f5f-9ea0-62c66b6e80c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:21.057004 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471629e5-a724-4f5f-9ea0-62c66b6e80c8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:21.057287 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471629e5-a724-4f5f-9ea0-62c66b6e80c8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:21.057572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471629e5-a724-4f5f-9ea0-62c66b6e80c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:21.058399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471629e5-a724-4f5f-9ea0-62c66b6e80c8 None admin] RBAC: Adding query filter params (interface=public, service_id=c46adf2d2f2a44a78b6078869e366992, region_id=RegionOne) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:21.058715 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471629e5-a724-4f5f-9ea0-62c66b6e80c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:21.060137 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471629e5-a724-4f5f-9ea0-62c66b6e80c8 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:21.070170 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 496/997] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:19:20 2022] GET /identity/v3/endpoints?service_id=c46adf2d2f2a44a78b6078869e366992&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:21.071555 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471629e5-a724-4f5f-9ea0-62c66b6e80c8 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:21.131287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5b4bf-5f21-4555-aa96-c696de5b319d None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:21.166640 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1e8ae-d87f-419f-9968-b9b38142d66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:21.167472 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1e8ae-d87f-419f-9968-b9b38142d66d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:21.167764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1e8ae-d87f-419f-9968-b9b38142d66d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:21.168045 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1e8ae-d87f-419f-9968-b9b38142d66d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:21.168699 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1e8ae-d87f-419f-9968-b9b38142d66d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:21.170078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1e8ae-d87f-419f-9968-b9b38142d66d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:21.176820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471629e5-a724-4f5f-9ea0-62c66b6e80c8 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-6ae1e8ae-d87f-419f-9968-b9b38142d66d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:21.182772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:21.183945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:21.187614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:21.187614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:21.187614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:21.187614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:21.187614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:21.187614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:21.187614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:21.187614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:21.187614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:21.187614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:21.187614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:21.187614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:21.187614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:19:21.187614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:21.187614 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 502/998] 10.222.0.84 () {56 vars in 1126 bytes} [Thu May 19 20:19:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:21.190846 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1e8ae-d87f-419f-9968-b9b38142d66d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:21.219234 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcf276-a784-4471-a275-524971b40399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:21.220185 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcf276-a784-4471-a275-524971b40399 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:21.220468 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcf276-a784-4471-a275-524971b40399 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:21.220874 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcf276-a784-4471-a275-524971b40399 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:21.221874 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edcf276-a784-4471-a275-524971b40399 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:21.223564 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edcf276-a784-4471-a275-524971b40399 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:21.242294 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 497/999] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:19:21 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:19:21.266219 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:21.310143 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1e8ae-d87f-419f-9968-b9b38142d66d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:21.351123 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17a1de-91ea-491f-917e-4b499625f452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:21.351938 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17a1de-91ea-491f-917e-4b499625f452 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:21.352223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17a1de-91ea-491f-917e-4b499625f452 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:21.352503 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17a1de-91ea-491f-917e-4b499625f452 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:21.353233 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17a1de-91ea-491f-917e-4b499625f452 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:21.353544 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17a1de-91ea-491f-917e-4b499625f452 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:21.354990 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17a1de-91ea-491f-917e-4b499625f452 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:21.362629 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 503/1000] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:19:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:21.367201 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcf276-a784-4471-a275-524971b40399 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:21.410977 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:21.495001 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcf276-a784-4471-a275-524971b40399 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:21.528777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fca0f-8e02-46d5-bf1c-c23c239f27a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:21.529444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fca0f-8e02-46d5-bf1c-c23c239f27a3 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:21.529730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fca0f-8e02-46d5-bf1c-c23c239f27a3 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:21.529991 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fca0f-8e02-46d5-bf1c-c23c239f27a3 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:21.530717 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fca0f-8e02-46d5-bf1c-c23c239f27a3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:21.531022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fca0f-8e02-46d5-bf1c-c23c239f27a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:21.532407 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fca0f-8e02-46d5-bf1c-c23c239f27a3 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:21.538392 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 498/1001] 10.222.0.84 () {56 vars in 1122 bytes} [Thu May 19 20:19:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:21.544084 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17a1de-91ea-491f-917e-4b499625f452 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:21.651769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17a1de-91ea-491f-917e-4b499625f452 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:21.685537 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:21.692301 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d178c-9b4a-4879-ac07-98dff398ec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:21.692971 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d178c-9b4a-4879-ac07-98dff398ec40 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:21.693256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d178c-9b4a-4879-ac07-98dff398ec40 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:21.693527 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d178c-9b4a-4879-ac07-98dff398ec40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:21.694409 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d178c-9b4a-4879-ac07-98dff398ec40 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:21.695780 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d178c-9b4a-4879-ac07-98dff398ec40 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:21.712286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 504/1002] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:19:21 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:19:21.838582 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b34b81-1619-46a2-9256-b2d03a01b350 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:21.838901 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b34b81-1619-46a2-9256-b2d03a01b350 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:21.839176 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b34b81-1619-46a2-9256-b2d03a01b350 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:21.840748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 499/1003] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:21.846872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43bc51-ac75-4231-9f2d-c28e3e733cd7 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:21.847172 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43bc51-ac75-4231-9f2d-c28e3e733cd7 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:21.847442 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43bc51-ac75-4231-9f2d-c28e3e733cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:21.867061 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-bf43bc51-ac75-4231-9f2d-c28e3e733cd7 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:21.868317 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-bf43bc51-ac75-4231-9f2d-c28e3e733cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:21.956867 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 505/1004] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:21 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:21.980348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b34b81-1619-46a2-9256-b2d03a01b350 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:22.086060 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:22.093153 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b34b81-1619-46a2-9256-b2d03a01b350 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:22.128389 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f23ed-5a68-42f8-b68e-f22c73ad2a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:22.129194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f23ed-5a68-42f8-b68e-f22c73ad2a84 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:22.129478 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f23ed-5a68-42f8-b68e-f22c73ad2a84 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:22.129752 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f23ed-5a68-42f8-b68e-f22c73ad2a84 None admin] PATH_INFO: `/v3/services/image` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:22.130459 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f23ed-5a68-42f8-b68e-f22c73ad2a84 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:22.132256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f23ed-5a68-42f8-b68e-f22c73ad2a84 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-e79f23ed-5a68-42f8-b68e-f22c73ad2a84 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:22.144553 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:22.145704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:22.146795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:22.146795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:22.146795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:22.146795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:22.146795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:22.146795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:22.146795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:22.146795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:22.146795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:22.146795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:22.146795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:22.146795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:22.146795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:19:22.146795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:22.150352 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 500/1005] 10.222.0.84 () {56 vars in 1105 bytes} [Thu May 19 20:19:21 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:22.152695 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43bc51-ac75-4231-9f2d-c28e3e733cd7 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:22.258524 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43bc51-ac75-4231-9f2d-c28e3e733cd7 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:22.276471 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:22.296884 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd292cc7-2550-4c62-b1be-39bec318c44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:22.297565 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd292cc7-2550-4c62-b1be-39bec318c44c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:22.297842 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd292cc7-2550-4c62-b1be-39bec318c44c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:22.298122 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd292cc7-2550-4c62-b1be-39bec318c44c None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:22.298867 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd292cc7-2550-4c62-b1be-39bec318c44c None admin] RBAC: Adding query filter params (name=image) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:22.299177 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd292cc7-2550-4c62-b1be-39bec318c44c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:22.300559 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd292cc7-2550-4c62-b1be-39bec318c44c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:22.305970 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 506/1006] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:22.311548 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f23ed-5a68-42f8-b68e-f22c73ad2a84 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:22.423823 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f23ed-5a68-42f8-b68e-f22c73ad2a84 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:22.461671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9e785-2837-415d-9242-776cdaf5a03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:22.462374 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9e785-2837-415d-9242-776cdaf5a03e None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:22.462667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9e785-2837-415d-9242-776cdaf5a03e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:22.462943 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9e785-2837-415d-9242-776cdaf5a03e None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:22.463979 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9e785-2837-415d-9242-776cdaf5a03e None admin] RBAC: Adding query filter params (type=image) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:22.464282 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9e785-2837-415d-9242-776cdaf5a03e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:22.465676 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9e785-2837-415d-9242-776cdaf5a03e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:22.478538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 501/1007] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:22 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:23.485358 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0987a-32bc-4841-8b92-1105badb78d3 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:23.485920 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0987a-32bc-4841-8b92-1105badb78d3 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:23.485920 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0987a-32bc-4841-8b92-1105badb78d3 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:23.487574 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 507/1008] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:23.492970 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfccbfd-3982-41d0-8dcf-aa6973127f76 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:23.493400 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfccbfd-3982-41d0-8dcf-aa6973127f76 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:23.493597 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfccbfd-3982-41d0-8dcf-aa6973127f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:23.511469 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-3dfccbfd-3982-41d0-8dcf-aa6973127f76 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:23.512699 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-3dfccbfd-3982-41d0-8dcf-aa6973127f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:23.536022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:23.591837 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 502/1009] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:23 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:23.614700 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0987a-32bc-4841-8b92-1105badb78d3 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:23.727434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0987a-32bc-4841-8b92-1105badb78d3 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:23.766307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aff0b-769e-41e0-88e4-053c26372407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:23.767157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aff0b-769e-41e0-88e4-053c26372407 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:23.767443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aff0b-769e-41e0-88e4-053c26372407 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:23.767716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aff0b-769e-41e0-88e4-053c26372407 None admin] PATH_INFO: `/v3/services/compute` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:23.768379 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67aff0b-769e-41e0-88e4-053c26372407 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:23.769819 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67aff0b-769e-41e0-88e4-053c26372407 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-f67aff0b-769e-41e0-88e4-053c26372407 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:23.783109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:23.784348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:23.785446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:23.785446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:23.785446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:23.785446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:23.785446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:23.785446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:23.785446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:23.785446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:23.785446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:23.785446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:23.785446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:23.785446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:23.785446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:19:23.785446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:23.785446 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 508/1010] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:19:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:23.791119 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfccbfd-3982-41d0-8dcf-aa6973127f76 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:23.898637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfccbfd-3982-41d0-8dcf-aa6973127f76 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:23.941377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2ee1d-e478-4bdd-8d33-2a3556d9d07b None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:23.941587 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86a526-3e44-40c9-9b31-387d181a62fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:23.942159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2ee1d-e478-4bdd-8d33-2a3556d9d07b None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:23.942520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86a526-3e44-40c9-9b31-387d181a62fa None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:23.942946 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2ee1d-e478-4bdd-8d33-2a3556d9d07b None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:23.943090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86a526-3e44-40c9-9b31-387d181a62fa None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:23.943449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86a526-3e44-40c9-9b31-387d181a62fa None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:23.944776 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86a526-3e44-40c9-9b31-387d181a62fa None admin] RBAC: Adding query filter params (name=compute) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:23.945302 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86a526-3e44-40c9-9b31-387d181a62fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:23.945851 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 509/1011] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:23.947758 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86a526-3e44-40c9-9b31-387d181a62fa None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:23.952245 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66730ba-d281-439d-97a9-6a204ae615c2 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:23.952952 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66730ba-d281-439d-97a9-6a204ae615c2 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:23.953472 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66730ba-d281-439d-97a9-6a204ae615c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:23.953828 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 503/1012] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:19:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:23.960339 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86a526-3e44-40c9-9b31-387d181a62fa None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:23.975343 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-b66730ba-d281-439d-97a9-6a204ae615c2 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:23.977165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-b66730ba-d281-439d-97a9-6a204ae615c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:23.986024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:24.024404 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:24.054634 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 510/1013] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:23 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:19:24.075601 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66730ba-d281-439d-97a9-6a204ae615c2 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:24.079892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86a526-3e44-40c9-9b31-387d181a62fa None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:24.115907 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e84e3-cf38-4ebe-a34c-40dcedb41910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:24.116583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e84e3-cf38-4ebe-a34c-40dcedb41910 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:24.116867 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e84e3-cf38-4ebe-a34c-40dcedb41910 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:24.117138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e84e3-cf38-4ebe-a34c-40dcedb41910 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:24.118026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e84e3-cf38-4ebe-a34c-40dcedb41910 None admin] RBAC: Adding query filter params (type=compute) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:24.118148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e84e3-cf38-4ebe-a34c-40dcedb41910 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:24.119554 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e84e3-cf38-4ebe-a34c-40dcedb41910 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:24.125064 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 504/1014] 10.222.0.84 () {56 vars in 1112 bytes} [Thu May 19 20:19:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:24.130680 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e84e3-cf38-4ebe-a34c-40dcedb41910 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:24.187153 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66730ba-d281-439d-97a9-6a204ae615c2 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:24.220665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:24.225883 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26860ed2-cf16-4c08-819a-85421c1db05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:24.226738 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26860ed2-cf16-4c08-819a-85421c1db05a None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:24.227017 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26860ed2-cf16-4c08-819a-85421c1db05a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:24.227294 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26860ed2-cf16-4c08-819a-85421c1db05a None admin] PATH_INFO: `/v3/domains/default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:24.228505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26860ed2-cf16-4c08-819a-85421c1db05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:24.230030 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26860ed2-cf16-4c08-819a-85421c1db05a None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:24.231723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 511/1015] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:24.237646 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26860ed2-cf16-4c08-819a-85421c1db05a None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:24.239799 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e84e3-cf38-4ebe-a34c-40dcedb41910 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:24.279318 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51a4e3-6832-415b-b57e-0435f6d4d83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:24.280302 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51a4e3-6832-415b-b57e-0435f6d4d83b None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:24.280750 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51a4e3-6832-415b-b57e-0435f6d4d83b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:24.281187 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51a4e3-6832-415b-b57e-0435f6d4d83b None admin] PATH_INFO: `/v3/endpoints` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:24.282113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51a4e3-6832-415b-b57e-0435f6d4d83b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:24.283736 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51a4e3-6832-415b-b57e-0435f6d4d83b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:24.290589 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:24.308224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 505/1016] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:19:24 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:19:24.367086 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26860ed2-cf16-4c08-819a-85421c1db05a None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:24.415989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f53d4-d9fe-40a6-82ae-65b682a0186e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:24.419882 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f53d4-d9fe-40a6-82ae-65b682a0186e None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:24.419882 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f53d4-d9fe-40a6-82ae-65b682a0186e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:24.419882 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f53d4-d9fe-40a6-82ae-65b682a0186e None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:24.419882 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f53d4-d9fe-40a6-82ae-65b682a0186e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:24.419882 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f53d4-d9fe-40a6-82ae-65b682a0186e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:24.458392 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 512/1017] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:19:24 2022] POST /identity/v3/projects => generated 311 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:19:24.764393 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b5201-0eeb-4b9e-800c-e3050b4ed794 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:24.764955 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b5201-0eeb-4b9e-800c-e3050b4ed794 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:24.764955 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b5201-0eeb-4b9e-800c-e3050b4ed794 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:24.766500 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 506/1018] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:24.771781 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd4630-d6a0-4228-975d-630e5bea5c8e None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:24.772254 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd4630-d6a0-4228-975d-630e5bea5c8e None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:24.772549 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd4630-d6a0-4228-975d-630e5bea5c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:24.790573 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-84fd4630-d6a0-4228-975d-630e5bea5c8e None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:24.791778 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-84fd4630-d6a0-4228-975d-630e5bea5c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:24.867590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 513/1019] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:24 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:19:24.890871 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b5201-0eeb-4b9e-800c-e3050b4ed794 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:25.004872 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b5201-0eeb-4b9e-800c-e3050b4ed794 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:25.041704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53240319-9a62-444e-b73a-7ece6a998a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:25.042413 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53240319-9a62-444e-b73a-7ece6a998a5f None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:25.042704 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53240319-9a62-444e-b73a-7ece6a998a5f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:25.042981 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53240319-9a62-444e-b73a-7ece6a998a5f None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:25.043743 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53240319-9a62-444e-b73a-7ece6a998a5f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:25.045131 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53240319-9a62-444e-b73a-7ece6a998a5f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:25.055504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:25.070754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 507/1020] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:19:24 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:19:26.687784 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf3e8e-489b-4e33-a5d7-9b8a282f7592 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:26.688342 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf3e8e-489b-4e33-a5d7-9b8a282f7592 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:26.688342 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf3e8e-489b-4e33-a5d7-9b8a282f7592 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:26.690294 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 514/1021] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:26.695284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a41fb-d9df-498c-82d8-c9438f9a1bd8 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:26.695717 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a41fb-d9df-498c-82d8-c9438f9a1bd8 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:26.695896 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a41fb-d9df-498c-82d8-c9438f9a1bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:26.714162 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-ec1a41fb-d9df-498c-82d8-c9438f9a1bd8 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:26.715399 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-ec1a41fb-d9df-498c-82d8-c9438f9a1bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:26.756400 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175de15b-5ad2-47e8-a1be-c6d67f02296f None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:26.756621 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175de15b-5ad2-47e8-a1be-c6d67f02296f None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:26.756919 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175de15b-5ad2-47e8-a1be-c6d67f02296f None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:26.758684 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 515/1022] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:26.763713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06dcea-6353-4281-a24b-8c70dce609f3 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:26.763987 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06dcea-6353-4281-a24b-8c70dce609f3 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:26.764261 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06dcea-6353-4281-a24b-8c70dce609f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:26.782701 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-af06dcea-6353-4281-a24b-8c70dce609f3 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:26.784011 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-af06dcea-6353-4281-a24b-8c70dce609f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:26.794343 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 508/1023] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:26.802733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:26.817776 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a41fb-d9df-498c-82d8-c9438f9a1bd8 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:26.861470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 516/1024] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:19:26.886371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06dcea-6353-4281-a24b-8c70dce609f3 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:26.927040 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a41fb-d9df-498c-82d8-c9438f9a1bd8 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:26.953862 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:26.966406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700f87f-0694-4935-a220-c7bad2e44639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:26.967229 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700f87f-0694-4935-a220-c7bad2e44639 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:26.967506 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700f87f-0694-4935-a220-c7bad2e44639 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:26.967779 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700f87f-0694-4935-a220-c7bad2e44639 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:26.981164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700f87f-0694-4935-a220-c7bad2e44639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-a700f87f-0694-4935-a220-c7bad2e44639 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:19:26.992327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:26.993463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:26.994927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:26.994927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:26.994927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:26.994927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:26.994927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:19:26.994927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:19:26.994927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:19:26.994927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:19:26.994927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:19:26.994927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:19:26.994927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:19:26.994927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:26.995791 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06dcea-6353-4281-a24b-8c70dce609f3 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:26.996342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 509/1025] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 20:19:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:27.000132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700f87f-0694-4935-a220-c7bad2e44639 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:27.034281 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b72940-3a65-4a68-b5eb-3febd312cb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:27.035114 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b72940-3a65-4a68-b5eb-3febd312cb17 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:27.035391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b72940-3a65-4a68-b5eb-3febd312cb17 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:27.035661 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b72940-3a65-4a68-b5eb-3febd312cb17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:27.036898 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b72940-3a65-4a68-b5eb-3febd312cb17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:27.038517 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b72940-3a65-4a68-b5eb-3febd312cb17 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:27.042327 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 517/1026] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:27.045760 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b72940-3a65-4a68-b5eb-3febd312cb17 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:27.070814 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:27.109606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700f87f-0694-4935-a220-c7bad2e44639 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:27.144379 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86409d9a-be71-4204-bfb4-221fc0378214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:27.145149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86409d9a-be71-4204-bfb4-221fc0378214 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:27.145495 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86409d9a-be71-4204-bfb4-221fc0378214 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:27.145863 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86409d9a-be71-4204-bfb4-221fc0378214 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:27.146722 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86409d9a-be71-4204-bfb4-221fc0378214 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:27.147095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86409d9a-be71-4204-bfb4-221fc0378214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:27.148591 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86409d9a-be71-4204-bfb4-221fc0378214 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:27.162554 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 510/1027] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 20:19:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:27.162793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b72940-3a65-4a68-b5eb-3febd312cb17 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:27.164999 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86409d9a-be71-4204-bfb4-221fc0378214 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:27.201609 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403dae94-2e63-4d9c-bb60-e7dd064ffdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:27.202197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dae94-2e63-4d9c-bb60-e7dd064ffdb0 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:27.202508 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dae94-2e63-4d9c-bb60-e7dd064ffdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:27.202785 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dae94-2e63-4d9c-bb60-e7dd064ffdb0 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:27.203505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403dae94-2e63-4d9c-bb60-e7dd064ffdb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:27.205007 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403dae94-2e63-4d9c-bb60-e7dd064ffdb0 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:27.224521 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:27.228317 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 518/1028] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:19:27 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:19:27.295205 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86409d9a-be71-4204-bfb4-221fc0378214 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:27.334906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b503e-e37a-4347-b6f7-8c91b572d781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:27.335995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b503e-e37a-4347-b6f7-8c91b572d781 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:27.336452 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b503e-e37a-4347-b6f7-8c91b572d781 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:27.336886 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b503e-e37a-4347-b6f7-8c91b572d781 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:27.350452 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b503e-e37a-4347-b6f7-8c91b572d781 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:27.352205 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b503e-e37a-4347-b6f7-8c91b572d781 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:27.371102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70deec0a-4bd9-404a-9041-e88e81e0aa9d None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:27.371394 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70deec0a-4bd9-404a-9041-e88e81e0aa9d None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:27.371683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70deec0a-4bd9-404a-9041-e88e81e0aa9d None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:27.371927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-832b503e-e37a-4347-b6f7-8c91b572d781 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:27.371927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:27.371927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:27.371927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:27.371927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:27.371927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:27.371927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:27.371927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:27.371927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:27.371927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:27.371927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:27.371927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:27.371927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:27.371927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:27.371927 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:27.373061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:27.374352 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:27.374352 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:27.374352 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:27.374352 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:27.374639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 519/1029] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:19:27.380438 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 511/1030] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:27.380632 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb7143-0284-46f3-8742-0b5da4f7c96e None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:27.380632 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb7143-0284-46f3-8742-0b5da4f7c96e None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:27.380632 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb7143-0284-46f3-8742-0b5da4f7c96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:27.381311 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b503e-e37a-4347-b6f7-8c91b572d781 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:27.402831 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-c2eb7143-0284-46f3-8742-0b5da4f7c96e None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:27.404079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-c2eb7143-0284-46f3-8742-0b5da4f7c96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:27.495401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 520/1031] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:19:27.509142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:27.512581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b503e-e37a-4347-b6f7-8c91b572d781 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:27.519417 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb7143-0284-46f3-8742-0b5da4f7c96e None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:27.529034 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:27.551802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84335c-2f87-4b66-829f-1be9e7d58e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:27.552820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84335c-2f87-4b66-829f-1be9e7d58e08 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:27.553243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84335c-2f87-4b66-829f-1be9e7d58e08 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:27.553742 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84335c-2f87-4b66-829f-1be9e7d58e08 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:27.558307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84335c-2f87-4b66-829f-1be9e7d58e08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:27.558307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84335c-2f87-4b66-829f-1be9e7d58e08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:27.558307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84335c-2f87-4b66-829f-1be9e7d58e08 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:27.570161 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 512/1032] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:27.576195 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84335c-2f87-4b66-829f-1be9e7d58e08 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:27.634209 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb7143-0284-46f3-8742-0b5da4f7c96e None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:27.671247 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b840c8-3c8e-445d-bcc5-5372c9af4933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:27.672045 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b840c8-3c8e-445d-bcc5-5372c9af4933 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:27.672339 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b840c8-3c8e-445d-bcc5-5372c9af4933 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:27.672598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b840c8-3c8e-445d-bcc5-5372c9af4933 None admin] PATH_INFO: `/v3/services/image` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:27.673245 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b840c8-3c8e-445d-bcc5-5372c9af4933 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:27.674641 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b840c8-3c8e-445d-bcc5-5372c9af4933 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:27.683066 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84335c-2f87-4b66-829f-1be9e7d58e08 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-37b840c8-3c8e-445d-bcc5-5372c9af4933 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:27.686348 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:27.687525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:27.689277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:27.689277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:27.689277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:27.689277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:27.689277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:27.689277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:27.689277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:27.689277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:27.689277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:27.689277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:27.689277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:27.689277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:27.689277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:19:27.689277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:27.689277 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 521/1033] 10.222.0.84 () {56 vars in 1105 bytes} [Thu May 19 20:19:27 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:27.693471 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b840c8-3c8e-445d-bcc5-5372c9af4933 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:27.726736 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efe521-7377-4656-9762-f3b1646cee99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:27.727551 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efe521-7377-4656-9762-f3b1646cee99 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:27.727832 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efe521-7377-4656-9762-f3b1646cee99 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:27.728109 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efe521-7377-4656-9762-f3b1646cee99 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:27.735458 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efe521-7377-4656-9762-f3b1646cee99 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:27.736913 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efe521-7377-4656-9762-f3b1646cee99 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-e6efe521-7377-4656-9762-f3b1646cee99 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:19:27.751803 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:27.753045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:27.754209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:27.754209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:27.754209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:19:27.754209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:19:27.754209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:19:27.754209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:19:27.754209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:19:27.754209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:19:27.754209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:19:27.754209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:27.754209 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 513/1034] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:19:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:27.758871 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efe521-7377-4656-9762-f3b1646cee99 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:27.771876 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:27.781562 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:27.808367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b840c8-3c8e-445d-bcc5-5372c9af4933 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:27.849199 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3485f9-958c-42ae-82e5-d7ef356c4b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:27.849887 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3485f9-958c-42ae-82e5-d7ef356c4b71 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:27.850269 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3485f9-958c-42ae-82e5-d7ef356c4b71 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:27.850567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3485f9-958c-42ae-82e5-d7ef356c4b71 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:27.851283 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3485f9-958c-42ae-82e5-d7ef356c4b71 None admin] RBAC: Adding query filter params (name=image) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:27.851593 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3485f9-958c-42ae-82e5-d7ef356c4b71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:27.852979 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3485f9-958c-42ae-82e5-d7ef356c4b71 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:27.859166 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 522/1035] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:27.863292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efe521-7377-4656-9762-f3b1646cee99 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:27.864695 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3485f9-958c-42ae-82e5-d7ef356c4b71 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:27.901071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df1142-95a4-4fea-8fff-cc2a1bc7b02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:27.901859 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df1142-95a4-4fea-8fff-cc2a1bc7b02c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:27.902148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df1142-95a4-4fea-8fff-cc2a1bc7b02c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:27.902456 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df1142-95a4-4fea-8fff-cc2a1bc7b02c None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:27.903294 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df1142-95a4-4fea-8fff-cc2a1bc7b02c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:27.903601 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df1142-95a4-4fea-8fff-cc2a1bc7b02c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:27.905055 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df1142-95a4-4fea-8fff-cc2a1bc7b02c None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:27.924693 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 514/1036] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:19:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:27.930648 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df1142-95a4-4fea-8fff-cc2a1bc7b02c None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:27.975268 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3485f9-958c-42ae-82e5-d7ef356c4b71 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:28.013478 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51957b03-931e-452f-9f33-335e06e78b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:28.014078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51957b03-931e-452f-9f33-335e06e78b7e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:28.014612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:28.014757 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51957b03-931e-452f-9f33-335e06e78b7e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:28.014967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51957b03-931e-452f-9f33-335e06e78b7e None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:28.015975 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51957b03-931e-452f-9f33-335e06e78b7e None admin] RBAC: Adding query filter params (type=image) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:28.016256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51957b03-931e-452f-9f33-335e06e78b7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:28.018377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51957b03-931e-452f-9f33-335e06e78b7e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:28.020125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:28.031184 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 523/1037] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:28.032022 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51957b03-931e-452f-9f33-335e06e78b7e None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:28.040472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df1142-95a4-4fea-8fff-cc2a1bc7b02c None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:28.076501 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e710b5-12e8-426a-bf34-331d1519420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:28.077156 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e710b5-12e8-426a-bf34-331d1519420c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:28.077434 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e710b5-12e8-426a-bf34-331d1519420c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:28.077709 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e710b5-12e8-426a-bf34-331d1519420c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:28.141285 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51957b03-931e-452f-9f33-335e06e78b7e None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:28.154523 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e710b5-12e8-426a-bf34-331d1519420c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:28.156000 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e710b5-12e8-426a-bf34-331d1519420c None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:28.183377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e13df3-70a9-4592-bef5-aed1c5a2c54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:28.184459 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e13df3-70a9-4592-bef5-aed1c5a2c54c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:28.185021 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e13df3-70a9-4592-bef5-aed1c5a2c54c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:28.185309 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e13df3-70a9-4592-bef5-aed1c5a2c54c None admin] PATH_INFO: `/v3/endpoints` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:28.186390 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e13df3-70a9-4592-bef5-aed1c5a2c54c None admin] RBAC: Adding query filter params (interface=public, service_id=57bb1fc7370645218e00c8b6b8e0d3cb, region_id=RegionOne) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:28.187088 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e13df3-70a9-4592-bef5-aed1c5a2c54c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:28.188720 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e13df3-70a9-4592-bef5-aed1c5a2c54c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:28.195991 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 524/1038] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:19:28 2022] GET /identity/v3/endpoints?service_id=57bb1fc7370645218e00c8b6b8e0d3cb&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:28.201514 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e13df3-70a9-4592-bef5-aed1c5a2c54c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:28.237825 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:28.262938 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 515/1039] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:19:27 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:19:28.268673 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e710b5-12e8-426a-bf34-331d1519420c None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:28.272968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:28.324964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e13df3-70a9-4592-bef5-aed1c5a2c54c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:28.360488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc433d4-379d-4081-8575-799bb7aa2208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:28.361157 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc433d4-379d-4081-8575-799bb7aa2208 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:28.361435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc433d4-379d-4081-8575-799bb7aa2208 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:28.361705 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc433d4-379d-4081-8575-799bb7aa2208 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:28.362403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc433d4-379d-4081-8575-799bb7aa2208 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:28.363782 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc433d4-379d-4081-8575-799bb7aa2208 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:28.370771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 525/1040] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:19:28 2022] GET /identity/v3/services => generated 1957 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:19:28.380600 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e710b5-12e8-426a-bf34-331d1519420c None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:28.420426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93e1d9-1657-472d-9dfb-53ffdea8ccf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:28.421261 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93e1d9-1657-472d-9dfb-53ffdea8ccf6 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:28.421546 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93e1d9-1657-472d-9dfb-53ffdea8ccf6 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:28.421816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93e1d9-1657-472d-9dfb-53ffdea8ccf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:28.435224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93e1d9-1657-472d-9dfb-53ffdea8ccf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:28.436767 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93e1d9-1657-472d-9dfb-53ffdea8ccf6 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:28.456637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-9a93e1d9-1657-472d-9dfb-53ffdea8ccf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:28.456637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:28.456637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:28.456637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:28.456637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:28.456637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:28.456637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:28.456637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:28.456637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:28.456637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:28.456637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:28.456637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:28.456637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:28.456637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:28.456637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:28.457830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:28.459335 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:28.459335 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:28.459335 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:28.459335 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:28.462627 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 516/1041] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:28.464289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc433d4-379d-4081-8575-799bb7aa2208 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:28.580004 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc433d4-379d-4081-8575-799bb7aa2208 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:28.606679 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:28.616527 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8bde2-285a-4c5f-9f9d-c847c7fc8b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:28.617196 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8bde2-285a-4c5f-9f9d-c847c7fc8b94 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:28.617474 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8bde2-285a-4c5f-9f9d-c847c7fc8b94 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:28.617745 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8bde2-285a-4c5f-9f9d-c847c7fc8b94 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:28.618486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8bde2-285a-4c5f-9f9d-c847c7fc8b94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:28.618786 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8bde2-285a-4c5f-9f9d-c847c7fc8b94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:28.620148 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8bde2-285a-4c5f-9f9d-c847c7fc8b94 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:28.638318 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 526/1042] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:28.641115 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93e1d9-1657-472d-9dfb-53ffdea8ccf6 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:28.680608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:28.756985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93e1d9-1657-472d-9dfb-53ffdea8ccf6 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:28.792089 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76e899-1280-4719-bf6c-e2fbff3d12b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:28.792891 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76e899-1280-4719-bf6c-e2fbff3d12b1 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:28.793168 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76e899-1280-4719-bf6c-e2fbff3d12b1 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:28.793442 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76e899-1280-4719-bf6c-e2fbff3d12b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:28.800360 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76e899-1280-4719-bf6c-e2fbff3d12b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:28.801814 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76e899-1280-4719-bf6c-e2fbff3d12b1 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:28.815906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-af76e899-1280-4719-bf6c-e2fbff3d12b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:28.815906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:28.815906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:28.815906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:28.815906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:28.815906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:28.815906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:28.815906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:28.815906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:28.815906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:28.815906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:28.815906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:28.815906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:19:28.815906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:19:28.815906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:28.817017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:28.818185 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:19:28.818185 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:19:28.818185 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:19:28.818185 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:19:28.818185 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:19:28.818185 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:19:28.818185 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:28.818185 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:28.818185 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 517/1043] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:19:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:28.825024 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8bde2-285a-4c5f-9f9d-c847c7fc8b94 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:28.929305 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8bde2-285a-4c5f-9f9d-c847c7fc8b94 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:28.964392 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8830a4-caab-4fa7-b8fe-770acd3d26e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:28.965064 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8830a4-caab-4fa7-b8fe-770acd3d26e8 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:28.965341 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8830a4-caab-4fa7-b8fe-770acd3d26e8 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:28.965610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8830a4-caab-4fa7-b8fe-770acd3d26e8 None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:28.966427 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8830a4-caab-4fa7-b8fe-770acd3d26e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:28.966734 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8830a4-caab-4fa7-b8fe-770acd3d26e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:28.968162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8830a4-caab-4fa7-b8fe-770acd3d26e8 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:28.985242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 527/1044] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:19:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:28.986577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76e899-1280-4719-bf6c-e2fbff3d12b1 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:29.089064 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:29.100846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76e899-1280-4719-bf6c-e2fbff3d12b1 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:29.135503 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59cfb07-8e04-4306-b5fb-2f540fd81a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:29.136219 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cfb07-8e04-4306-b5fb-2f540fd81a20 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:29.136498 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cfb07-8e04-4306-b5fb-2f540fd81a20 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:29.136766 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cfb07-8e04-4306-b5fb-2f540fd81a20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:29.137597 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59cfb07-8e04-4306-b5fb-2f540fd81a20 None admin] RBAC: Adding query filter params (role.id=943b7648db36455b8073c12e3b45374d, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=edb1bcb781084f03a4ad2611a09bd77e) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:29.137908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59cfb07-8e04-4306-b5fb-2f540fd81a20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:29.139373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59cfb07-8e04-4306-b5fb-2f540fd81a20 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:29.150314 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 518/1045] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:19:28 2022] GET /identity/v3/role_assignments?user.id=edb1bcb781084f03a4ad2611a09bd77e&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=943b7648db36455b8073c12e3b45374d => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:29.554041 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e2b20-135e-47b3-80f6-5733be46306b None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:29.554440 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e2b20-135e-47b3-80f6-5733be46306b None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:29.554729 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e2b20-135e-47b3-80f6-5733be46306b None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:29.556074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 528/1046] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:29.561707 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c926f8c-461e-492f-a389-4a3b8b992d58 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:29.561970 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c926f8c-461e-492f-a389-4a3b8b992d58 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:29.562301 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c926f8c-461e-492f-a389-4a3b8b992d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:29.579939 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-2c926f8c-461e-492f-a389-4a3b8b992d58 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:29.581181 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-2c926f8c-461e-492f-a389-4a3b8b992d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:29.654149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 519/1047] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:29.677049 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e2b20-135e-47b3-80f6-5733be46306b None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:29.728585 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:29.785458 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e2b20-135e-47b3-80f6-5733be46306b None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:29.818970 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fac55-297f-422e-aa09-a17349a81b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:29.819767 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fac55-297f-422e-aa09-a17349a81b9e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:29.820046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fac55-297f-422e-aa09-a17349a81b9e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:29.820314 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fac55-297f-422e-aa09-a17349a81b9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:29.831141 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fac55-297f-422e-aa09-a17349a81b9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-f85fac55-297f-422e-aa09-a17349a81b9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:19:29.839079 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:29.840322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:29.841419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:29.841419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:29.841419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:29.841419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:29.841419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:19:29.841419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:19:29.841419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:19:29.841419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:19:29.841419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:19:29.841419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:19:29.841419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:19:29.841419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:29.848488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 529/1048] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:19:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:29.848715 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c926f8c-461e-492f-a389-4a3b8b992d58 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:29.939859 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:29.958166 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c926f8c-461e-492f-a389-4a3b8b992d58 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:29.993841 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693b206-b593-488e-a4bb-4ac987ed21c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:29.994467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693b206-b593-488e-a4bb-4ac987ed21c4 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:29.994746 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693b206-b593-488e-a4bb-4ac987ed21c4 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:29.995021 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693b206-b593-488e-a4bb-4ac987ed21c4 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:29.995763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693b206-b593-488e-a4bb-4ac987ed21c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:29.996069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693b206-b593-488e-a4bb-4ac987ed21c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:29.997446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693b206-b593-488e-a4bb-4ac987ed21c4 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:30.010289 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 520/1049] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:19:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:30.012323 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fac55-297f-422e-aa09-a17349a81b9e None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:30.123129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fac55-297f-422e-aa09-a17349a81b9e None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:30.131662 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:30.162758 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b753020-3a9b-4192-86eb-2209e32777dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:30.163661 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b753020-3a9b-4192-86eb-2209e32777dc None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:30.163906 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b753020-3a9b-4192-86eb-2209e32777dc None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:30.164203 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b753020-3a9b-4192-86eb-2209e32777dc None admin] PATH_INFO: `/v3/users/dd60ddb1b52548b6a347f3636750bc9f` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:30.179968 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b753020-3a9b-4192-86eb-2209e32777dc None admin] RBAC: Authorizing `identity:get_user(user_id=dd60ddb1b52548b6a347f3636750bc9f)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:30.181420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b753020-3a9b-4192-86eb-2209e32777dc None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:30.188118 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 530/1050] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:19:30 2022] GET /identity/v3/users/dd60ddb1b52548b6a347f3636750bc9f => generated 285 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:30.189410 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693b206-b593-488e-a4bb-4ac987ed21c4 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:30.311372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693b206-b593-488e-a4bb-4ac987ed21c4 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:30.348527 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce441b95-7c01-495a-89f1-2b0a814c6c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:30.349202 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce441b95-7c01-495a-89f1-2b0a814c6c42 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:30.349579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce441b95-7c01-495a-89f1-2b0a814c6c42 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:30.349796 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce441b95-7c01-495a-89f1-2b0a814c6c42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:30.368212 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:30.436593 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce441b95-7c01-495a-89f1-2b0a814c6c42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:30.438833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce441b95-7c01-495a-89f1-2b0a814c6c42 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:30.578122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 521/1051] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:19:30 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:19:30.579633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b753020-3a9b-4192-86eb-2209e32777dc None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:30.694172 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b753020-3a9b-4192-86eb-2209e32777dc None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:30.736000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d053f0-efbd-4a95-b454-74674cd155fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:30.736858 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d053f0-efbd-4a95-b454-74674cd155fa None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:30.737141 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d053f0-efbd-4a95-b454-74674cd155fa None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:30.737421 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d053f0-efbd-4a95-b454-74674cd155fa None admin] PATH_INFO: `/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:30.738653 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d053f0-efbd-4a95-b454-74674cd155fa None admin] RBAC: Authorizing `identity:get_project(project_id=e5e7d4494dfb4f96ae2fa5c5641fc6cc)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:30.740104 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d053f0-efbd-4a95-b454-74674cd155fa None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:30.742663 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 531/1052] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:19:30 2022] GET /identity/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:30.744550 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e51c7-c178-4388-a31b-bb5cb2002f26 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:30.744904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e51c7-c178-4388-a31b-bb5cb2002f26 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:30.745251 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e51c7-c178-4388-a31b-bb5cb2002f26 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:30.746805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 522/1053] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:30.748345 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d053f0-efbd-4a95-b454-74674cd155fa None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:30.754007 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439035d0-ef1b-4703-93d1-261cc59fda1a None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:30.754322 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439035d0-ef1b-4703-93d1-261cc59fda1a None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:30.754610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439035d0-ef1b-4703-93d1-261cc59fda1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:30.779572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-439035d0-ef1b-4703-93d1-261cc59fda1a None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:30.780913 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-439035d0-ef1b-4703-93d1-261cc59fda1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:30.788101 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:30.829600 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:30.859457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 523/1054] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:19:30.873388 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d053f0-efbd-4a95-b454-74674cd155fa None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:30.883128 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439035d0-ef1b-4703-93d1-261cc59fda1a None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:30.924132 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a786a-1842-468b-a1fa-a96d446497de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:30.924801 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a786a-1842-468b-a1fa-a96d446497de None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:30.925081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a786a-1842-468b-a1fa-a96d446497de None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:30.925352 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a786a-1842-468b-a1fa-a96d446497de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:30.926602 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a786a-1842-468b-a1fa-a96d446497de None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=e5e7d4494dfb4f96ae2fa5c5641fc6cc, user.id=dd60ddb1b52548b6a347f3636750bc9f) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:30.926986 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a786a-1842-468b-a1fa-a96d446497de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:30.928521 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a786a-1842-468b-a1fa-a96d446497de None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:30.944389 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 532/1055] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:19:30 2022] GET /identity/v3/role_assignments?user.id=dd60ddb1b52548b6a347f3636750bc9f&scope.project.id=e5e7d4494dfb4f96ae2fa5c5641fc6cc&role.id=b239f941cb724331bf28dc489b70ad1d => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:31.018928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439035d0-ef1b-4703-93d1-261cc59fda1a None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:31.072631 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73425cdd-c4d3-4ad6-9678-4a70f0cce9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:31.073461 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73425cdd-c4d3-4ad6-9678-4a70f0cce9ce None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:31.073794 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73425cdd-c4d3-4ad6-9678-4a70f0cce9ce None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:31.074081 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73425cdd-c4d3-4ad6-9678-4a70f0cce9ce None admin] PATH_INFO: `/v3/services/image` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:31.074786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73425cdd-c4d3-4ad6-9678-4a70f0cce9ce None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:31.076205 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73425cdd-c4d3-4ad6-9678-4a70f0cce9ce None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-73425cdd-c4d3-4ad6-9678-4a70f0cce9ce None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:31.093096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:31.099082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:31.100297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:31.100297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:31.100297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:31.100297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:31.100297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:31.100297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:31.100297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:31.100297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:31.100297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:31.100297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:31.100297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:19:31.100297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:19:31.100297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:19:31.100297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:31.100297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 524/1056] 10.222.0.84 () {56 vars in 1105 bytes} [Thu May 19 20:19:30 2022] GET /identity/v3/services/image => generated 86 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:31.104152 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a786a-1842-468b-a1fa-a96d446497de None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:31.208831 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:31.216736 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a786a-1842-468b-a1fa-a96d446497de None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:31.252246 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac30e3e-1484-48d5-9a89-8ce1d2b588a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:31.252923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac30e3e-1484-48d5-9a89-8ce1d2b588a1 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:31.253203 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac30e3e-1484-48d5-9a89-8ce1d2b588a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:31.253474 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac30e3e-1484-48d5-9a89-8ce1d2b588a1 None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:31.254178 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac30e3e-1484-48d5-9a89-8ce1d2b588a1 None admin] RBAC: Adding query filter params (name=image) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:31.254506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac30e3e-1484-48d5-9a89-8ce1d2b588a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:31.255874 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac30e3e-1484-48d5-9a89-8ce1d2b588a1 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:31.261760 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 533/1057] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:31.267408 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73425cdd-c4d3-4ad6-9678-4a70f0cce9ce None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:31.311065 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:31.374912 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73425cdd-c4d3-4ad6-9678-4a70f0cce9ce None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:31.412727 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1cfce-e440-47d6-8f03-87582c006cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:31.413582 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1cfce-e440-47d6-8f03-87582c006cba None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:31.413866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1cfce-e440-47d6-8f03-87582c006cba None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:31.414136 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1cfce-e440-47d6-8f03-87582c006cba None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:31.414868 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1cfce-e440-47d6-8f03-87582c006cba None admin] RBAC: Adding query filter params (type=image) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:31.415172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1cfce-e440-47d6-8f03-87582c006cba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:31.418390 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1cfce-e440-47d6-8f03-87582c006cba None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:31.430520 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 525/1058] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:31.432342 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac30e3e-1484-48d5-9a89-8ce1d2b588a1 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:31.537165 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac30e3e-1484-48d5-9a89-8ce1d2b588a1 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:31.551665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ed94b-6496-4eea-9dff-acba8bfb6b7e None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:31.552037 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ed94b-6496-4eea-9dff-acba8bfb6b7e None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:31.552404 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ed94b-6496-4eea-9dff-acba8bfb6b7e None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:31.554156 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 526/1059] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:31.559158 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fce88a-d5ef-48cb-abf9-54cd369e4192 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:31.559437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fce88a-d5ef-48cb-abf9-54cd369e4192 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:31.559708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fce88a-d5ef-48cb-abf9-54cd369e4192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:31.576322 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f9ace-fb4b-42ef-9225-c95d22fd1fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:31.576646 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-64fce88a-d5ef-48cb-abf9-54cd369e4192 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:31.576985 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f9ace-fb4b-42ef-9225-c95d22fd1fbe None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:31.577262 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f9ace-fb4b-42ef-9225-c95d22fd1fbe None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:31.577532 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f9ace-fb4b-42ef-9225-c95d22fd1fbe None admin] PATH_INFO: `/v3/endpoints` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:31.577789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-64fce88a-d5ef-48cb-abf9-54cd369e4192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:31.578287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f9ace-fb4b-42ef-9225-c95d22fd1fbe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:31.579709 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f9ace-fb4b-42ef-9225-c95d22fd1fbe None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:31.602668 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 534/1060] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:19:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:19:31.670216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 527/1061] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:31.692864 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f9ace-fb4b-42ef-9225-c95d22fd1fbe None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:31.719510 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:31.816264 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f9ace-fb4b-42ef-9225-c95d22fd1fbe None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:31.853534 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1255dd-86fc-498b-a282-099be5e07c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:31.854394 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1255dd-86fc-498b-a282-099be5e07c3d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:31.854672 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1255dd-86fc-498b-a282-099be5e07c3d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:31.854945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1255dd-86fc-498b-a282-099be5e07c3d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:31.865663 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1255dd-86fc-498b-a282-099be5e07c3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-bc1255dd-86fc-498b-a282-099be5e07c3d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:19:31.874304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:31.875813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:31.876809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:31.876809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:31.876809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:31.876809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:31.876809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:19:31.876809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:19:31.876809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:19:31.876809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:19:31.876809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:19:31.876809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:19:31.876809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:19:31.876809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:31.876809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 535/1062] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 20:19:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:31.882065 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fce88a-d5ef-48cb-abf9-54cd369e4192 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:31.913061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:31.987025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fce88a-d5ef-48cb-abf9-54cd369e4192 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:32.021417 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82a4af-a70d-4a46-b77c-e0e7c6aaa4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:32.022281 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82a4af-a70d-4a46-b77c-e0e7c6aaa4ef None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:32.022575 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82a4af-a70d-4a46-b77c-e0e7c6aaa4ef None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:32.022853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82a4af-a70d-4a46-b77c-e0e7c6aaa4ef None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:32.023598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82a4af-a70d-4a46-b77c-e0e7c6aaa4ef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:32.023902 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82a4af-a70d-4a46-b77c-e0e7c6aaa4ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:32.025285 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82a4af-a70d-4a46-b77c-e0e7c6aaa4ef None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:32.033841 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 528/1063] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 20:19:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:32.039542 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1255dd-86fc-498b-a282-099be5e07c3d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:32.123346 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:32.148097 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1255dd-86fc-498b-a282-099be5e07c3d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:32.183678 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6c642-1eff-4178-98a0-3dab8c5d0eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:32.184480 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6c642-1eff-4178-98a0-3dab8c5d0eae None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:32.184756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6c642-1eff-4178-98a0-3dab8c5d0eae None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:32.185025 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6c642-1eff-4178-98a0-3dab8c5d0eae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:32.196122 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6c642-1eff-4178-98a0-3dab8c5d0eae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:32.197568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6c642-1eff-4178-98a0-3dab8c5d0eae None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:32.215434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-74a6c642-1eff-4178-98a0-3dab8c5d0eae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:32.215434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:32.215434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:32.215434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:32.215434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:32.215434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:32.215434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:32.215434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:32.215434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:32.215434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:32.215434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:32.215434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:32.215434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:32.215434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:32.215434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:32.216536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:32.222347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:32.222347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:32.222347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:32.222347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:32.222347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 536/1064] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:32.224132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82a4af-a70d-4a46-b77c-e0e7c6aaa4ef None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:32.321000 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:32.331593 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82a4af-a70d-4a46-b77c-e0e7c6aaa4ef None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:32.365914 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b837c1f-4d2b-4ae6-9e94-244858bd4165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:32.366772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b837c1f-4d2b-4ae6-9e94-244858bd4165 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:32.367068 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b837c1f-4d2b-4ae6-9e94-244858bd4165 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:32.367340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b837c1f-4d2b-4ae6-9e94-244858bd4165 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:32.368073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b837c1f-4d2b-4ae6-9e94-244858bd4165 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:32.368375 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b837c1f-4d2b-4ae6-9e94-244858bd4165 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:32.369811 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b837c1f-4d2b-4ae6-9e94-244858bd4165 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:32.383320 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 529/1065] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:32.387548 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6c642-1eff-4178-98a0-3dab8c5d0eae None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:32.506182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6c642-1eff-4178-98a0-3dab8c5d0eae None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:32.533368 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:32.543090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a54461-3666-49dc-bcd8-63ba840f39d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:32.543887 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a54461-3666-49dc-bcd8-63ba840f39d8 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:32.544166 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a54461-3666-49dc-bcd8-63ba840f39d8 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:32.544437 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a54461-3666-49dc-bcd8-63ba840f39d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:32.552236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a54461-3666-49dc-bcd8-63ba840f39d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:32.554392 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a54461-3666-49dc-bcd8-63ba840f39d8 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-23a54461-3666-49dc-bcd8-63ba840f39d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:19:32.568422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:32.569567 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:32.575788 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:32.575788 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:32.575788 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:19:32.575788 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:19:32.575788 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:19:32.575788 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:19:32.575788 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:19:32.575788 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:19:32.575788 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:19:32.575788 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:32.575788 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 537/1066] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:19:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:32.577952 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b837c1f-4d2b-4ae6-9e94-244858bd4165 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:32.684891 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b837c1f-4d2b-4ae6-9e94-244858bd4165 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:32.730951 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2acea-1cfc-450f-a336-4fc869a67942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:32.731679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2acea-1cfc-450f-a336-4fc869a67942 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:32.731938 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2acea-1cfc-450f-a336-4fc869a67942 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:32.732298 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2acea-1cfc-450f-a336-4fc869a67942 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:32.733136 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2acea-1cfc-450f-a336-4fc869a67942 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:32.733486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2acea-1cfc-450f-a336-4fc869a67942 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:32.734967 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2acea-1cfc-450f-a336-4fc869a67942 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:32.753464 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 530/1067] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:19:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:32.759283 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a54461-3666-49dc-bcd8-63ba840f39d8 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:32.873178 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a54461-3666-49dc-bcd8-63ba840f39d8 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:32.909893 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5571b9-ef6a-44ad-bc22-5375fa693602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:32.910763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5571b9-ef6a-44ad-bc22-5375fa693602 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:32.911008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5571b9-ef6a-44ad-bc22-5375fa693602 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:32.911283 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5571b9-ef6a-44ad-bc22-5375fa693602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:32.922881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5571b9-ef6a-44ad-bc22-5375fa693602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:32.924345 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5571b9-ef6a-44ad-bc22-5375fa693602 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:32.942657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-cf5571b9-ef6a-44ad-bc22-5375fa693602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:32.942657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:32.942657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:32.942657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:32.942657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:32.942657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:32.942657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:32.942657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:32.942657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:32.942657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:32.942657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:32.942657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:32.942657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:32.942657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:32.942657 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:32.943891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:32.944954 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:32.944954 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:32.944954 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:32.944954 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:32.945310 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 538/1068] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:32.951551 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2acea-1cfc-450f-a336-4fc869a67942 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:32.967136 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:33.070142 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2acea-1cfc-450f-a336-4fc869a67942 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:33.114499 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2543e-f702-4a62-8478-22cc4b3697a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:33.114499 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2543e-f702-4a62-8478-22cc4b3697a4 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:33.114499 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2543e-f702-4a62-8478-22cc4b3697a4 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:33.114499 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2543e-f702-4a62-8478-22cc4b3697a4 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:33.115047 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2543e-f702-4a62-8478-22cc4b3697a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:33.115047 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2543e-f702-4a62-8478-22cc4b3697a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:33.115047 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2543e-f702-4a62-8478-22cc4b3697a4 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:33.131992 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 531/1069] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:33.137587 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5571b9-ef6a-44ad-bc22-5375fa693602 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:33.200708 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:33.217387 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8d098-d531-4439-a7f8-6f919bb164a6 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:33.217755 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8d098-d531-4439-a7f8-6f919bb164a6 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:33.217948 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8d098-d531-4439-a7f8-6f919bb164a6 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:33.219611 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 532/1070] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:19:33.224943 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5d16e-1594-4273-8a2f-f78a34673989 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:33.225295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5d16e-1594-4273-8a2f-f78a34673989 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:33.225655 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5d16e-1594-4273-8a2f-f78a34673989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:33.242795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-56c5d16e-1594-4273-8a2f-f78a34673989 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:33.244081 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-56c5d16e-1594-4273-8a2f-f78a34673989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:33.281768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5571b9-ef6a-44ad-bc22-5375fa693602 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:33.320797 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1df24e-7faf-4b45-b886-781f15676517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:33.321613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1df24e-7faf-4b45-b886-781f15676517 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:33.321891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1df24e-7faf-4b45-b886-781f15676517 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:33.322169 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1df24e-7faf-4b45-b886-781f15676517 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:33.329223 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:33.329455 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1df24e-7faf-4b45-b886-781f15676517 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:33.331180 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1df24e-7faf-4b45-b886-781f15676517 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:33.336947 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 533/1071] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:19:33.354523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-df1df24e-7faf-4b45-b886-781f15676517 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:33.354523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:33.354523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:33.354523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:33.354523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:33.354523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:33.354523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:33.354523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:33.354523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:33.354523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:33.354523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:33.354523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:33.354523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:19:33.354523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:19:33.354523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:33.355633 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:33.356740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:19:33.356740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:19:33.356740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:19:33.356740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:19:33.356740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:19:33.356740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:19:33.356740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:33.356740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:33.357332 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 539/1072] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:19:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:33.362591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1df24e-7faf-4b45-b886-781f15676517 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:33.362754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5d16e-1594-4273-8a2f-f78a34673989 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:33.479238 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5d16e-1594-4273-8a2f-f78a34673989 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:33.488038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:33.492044 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1df24e-7faf-4b45-b886-781f15676517 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:33.516361 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f7bd2-62a3-4bef-8d44-ad9c725b84ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:33.517074 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f7bd2-62a3-4bef-8d44-ad9c725b84ba None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:33.517305 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f7bd2-62a3-4bef-8d44-ad9c725b84ba None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:33.517586 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f7bd2-62a3-4bef-8d44-ad9c725b84ba None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:33.518571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f7bd2-62a3-4bef-8d44-ad9c725b84ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:33.518705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f7bd2-62a3-4bef-8d44-ad9c725b84ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:33.520165 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f7bd2-62a3-4bef-8d44-ad9c725b84ba None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:33.530743 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2d3a9-9955-4096-a510-6e4a62ea1761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:33.531582 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2d3a9-9955-4096-a510-6e4a62ea1761 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:33.531868 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2d3a9-9955-4096-a510-6e4a62ea1761 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:33.532145 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2d3a9-9955-4096-a510-6e4a62ea1761 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:33.542609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 534/1073] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:19:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:33.543987 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2d3a9-9955-4096-a510-6e4a62ea1761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:19:33.548360 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f7bd2-62a3-4bef-8d44-ad9c725b84ba None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-2ab2d3a9-9955-4096-a510-6e4a62ea1761 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:19:33.551945 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:33.553080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:33.554110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:33.554110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:33.554110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:33.554110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:33.554110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:19:33.554110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:19:33.554110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:19:33.554110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:19:33.554110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:19:33.554110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:19:33.554110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:19:33.554110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:33.554110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 540/1074] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:19:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:33.559343 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2d3a9-9955-4096-a510-6e4a62ea1761 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:33.614997 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:33.659917 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f7bd2-62a3-4bef-8d44-ad9c725b84ba None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:33.672951 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2d3a9-9955-4096-a510-6e4a62ea1761 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:33.695151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd4e7c-6478-4061-ac01-57e670ad0fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:33.695986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd4e7c-6478-4061-ac01-57e670ad0fd3 None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:33.696270 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd4e7c-6478-4061-ac01-57e670ad0fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:33.696551 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd4e7c-6478-4061-ac01-57e670ad0fd3 None admin] PATH_INFO: `/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/edb1bcb781084f03a4ad2611a09bd77e/roles/943b7648db36455b8073c12e3b45374d` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:33.698880 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd4e7c-6478-4061-ac01-57e670ad0fd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=59555e7adac54eddb47d1ef18ef870c4, user_id=edb1bcb781084f03a4ad2611a09bd77e, role_id=943b7648db36455b8073c12e3b45374d)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:33.700356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd4e7c-6478-4061-ac01-57e670ad0fd3 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:33.715268 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64cee98-9539-44dd-89a7-1afdd13095a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:33.715927 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cee98-9539-44dd-89a7-1afdd13095a5 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:33.716208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cee98-9539-44dd-89a7-1afdd13095a5 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:33.716485 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cee98-9539-44dd-89a7-1afdd13095a5 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:33.717299 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64cee98-9539-44dd-89a7-1afdd13095a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:33.717603 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64cee98-9539-44dd-89a7-1afdd13095a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:33.726362 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64cee98-9539-44dd-89a7-1afdd13095a5 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:33.726606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 535/1075] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:19:33 2022] PUT /identity/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/edb1bcb781084f03a4ad2611a09bd77e/roles/943b7648db36455b8073c12e3b45374d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:19:33.736094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 541/1076] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:19:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:33.742412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd4e7c-6478-4061-ac01-57e670ad0fd3 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:33.860772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd4e7c-6478-4061-ac01-57e670ad0fd3 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:33.884039 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:33.901276 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aff5f2-2565-437a-a7ac-a4b91c428b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:33.902167 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aff5f2-2565-437a-a7ac-a4b91c428b26 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:33.902468 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aff5f2-2565-437a-a7ac-a4b91c428b26 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:33.902728 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aff5f2-2565-437a-a7ac-a4b91c428b26 None admin] PATH_INFO: `/v3/users/dd60ddb1b52548b6a347f3636750bc9f` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:33.904452 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aff5f2-2565-437a-a7ac-a4b91c428b26 None admin] RBAC: Authorizing `identity:get_user(user_id=dd60ddb1b52548b6a347f3636750bc9f)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:33.905844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aff5f2-2565-437a-a7ac-a4b91c428b26 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:33.907256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 536/1077] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:19:33 2022] GET /identity/v3/users/dd60ddb1b52548b6a347f3636750bc9f => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:33.913043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64cee98-9539-44dd-89a7-1afdd13095a5 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:33.941100 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:33.955882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af64fc-34d6-4592-b84f-722212186422 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:33.956405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af64fc-34d6-4592-b84f-722212186422 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:33.956405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af64fc-34d6-4592-b84f-722212186422 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:33.957873 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 537/1078] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:33.962980 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc5488-6d51-4b57-b3e3-6ab17c668d26 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:33.963647 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc5488-6d51-4b57-b3e3-6ab17c668d26 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:33.963918 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc5488-6d51-4b57-b3e3-6ab17c668d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:33.982454 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-80cc5488-6d51-4b57-b3e3-6ab17c668d26 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:33.983854 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-80cc5488-6d51-4b57-b3e3-6ab17c668d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:34.023897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64cee98-9539-44dd-89a7-1afdd13095a5 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:34.063030 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c282ce-4838-4325-933e-064928f6091d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:34.064208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c282ce-4838-4325-933e-064928f6091d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:34.064483 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c282ce-4838-4325-933e-064928f6091d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:34.064805 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c282ce-4838-4325-933e-064928f6091d None admin] PATH_INFO: `/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:34.066008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c282ce-4838-4325-933e-064928f6091d None admin] RBAC: Authorizing `identity:get_project(project_id=e5e7d4494dfb4f96ae2fa5c5641fc6cc)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:34.074076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 538/1079] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:19:34.074290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c282ce-4838-4325-933e-064928f6091d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:34.074290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 542/1080] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:19:33 2022] GET /identity/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:34.075709 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc5488-6d51-4b57-b3e3-6ab17c668d26 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:34.090140 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c282ce-4838-4325-933e-064928f6091d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:34.184100 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:34.191657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc5488-6d51-4b57-b3e3-6ab17c668d26 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:34.196940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c282ce-4838-4325-933e-064928f6091d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:34.212465 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:34.233500 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc603d7-046b-421a-85ac-3a12e075afc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:34.234529 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc603d7-046b-421a-85ac-3a12e075afc7 None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:34.234882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc603d7-046b-421a-85ac-3a12e075afc7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:34.235163 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc603d7-046b-421a-85ac-3a12e075afc7 None admin] PATH_INFO: `/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc/users/dd60ddb1b52548b6a347f3636750bc9f/roles/b239f941cb724331bf28dc489b70ad1d` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:34.237174 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc603d7-046b-421a-85ac-3a12e075afc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e7d4494dfb4f96ae2fa5c5641fc6cc, user_id=dd60ddb1b52548b6a347f3636750bc9f, role_id=b239f941cb724331bf28dc489b70ad1d)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:34.238678 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc603d7-046b-421a-85ac-3a12e075afc7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:34.240779 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96827c95-b24d-49ce-999f-c49996df998c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:34.242144 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96827c95-b24d-49ce-999f-c49996df998c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:34.242623 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96827c95-b24d-49ce-999f-c49996df998c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:34.242896 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96827c95-b24d-49ce-999f-c49996df998c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:34.255490 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 539/1081] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:19:34 2022] PUT /identity/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc/users/dd60ddb1b52548b6a347f3636750bc9f/roles/b239f941cb724331bf28dc489b70ad1d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:19:34.323773 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96827c95-b24d-49ce-999f-c49996df998c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:34.325261 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96827c95-b24d-49ce-999f-c49996df998c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:34.433168 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:34.446674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 543/1082] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:19:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:19:34.451767 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc603d7-046b-421a-85ac-3a12e075afc7 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:34.559617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc603d7-046b-421a-85ac-3a12e075afc7 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:34.596196 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9368b-b664-4c4e-a98c-0271bd80c525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:34.597002 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9368b-b664-4c4e-a98c-0271bd80c525 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:34.597279 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9368b-b664-4c4e-a98c-0271bd80c525 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:34.597551 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9368b-b664-4c4e-a98c-0271bd80c525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:34.608130 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9368b-b664-4c4e-a98c-0271bd80c525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:34.609588 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9368b-b664-4c4e-a98c-0271bd80c525 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:34.627144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-0de9368b-b664-4c4e-a98c-0271bd80c525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:34.627144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:34.627144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:34.627144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:34.627144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:34.627144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:34.627144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:34.627144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:34.627144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:34.627144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:34.627144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:34.627144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:34.627144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:34.627144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:34.627144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:34.628231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:34.629333 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:34.629333 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:34.629333 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:34.629333 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:34.629333 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 540/1083] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:34.634694 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96827c95-b24d-49ce-999f-c49996df998c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:34.738032 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96827c95-b24d-49ce-999f-c49996df998c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:34.774141 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84257cf-724f-45f1-8e35-60dee2afd2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:34.774835 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84257cf-724f-45f1-8e35-60dee2afd2d6 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:34.775118 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84257cf-724f-45f1-8e35-60dee2afd2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:34.775384 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84257cf-724f-45f1-8e35-60dee2afd2d6 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:34.776094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84257cf-724f-45f1-8e35-60dee2afd2d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:34.776393 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84257cf-724f-45f1-8e35-60dee2afd2d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:34.779479 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84257cf-724f-45f1-8e35-60dee2afd2d6 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:34.798351 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 544/1084] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:35.948570 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990b0f3-472f-4960-a50c-aec11919ba24 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:35.949173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990b0f3-472f-4960-a50c-aec11919ba24 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:35.949173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990b0f3-472f-4960-a50c-aec11919ba24 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:35.950531 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 541/1085] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:35.955980 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87797c-73c7-48cf-acc5-cff93292fe44 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:35.956433 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87797c-73c7-48cf-acc5-cff93292fe44 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:35.956541 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87797c-73c7-48cf-acc5-cff93292fe44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:35.974284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-ae87797c-73c7-48cf-acc5-cff93292fe44 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:35.975536 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-ae87797c-73c7-48cf-acc5-cff93292fe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:36.024076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:36.046975 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 545/1086] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:36.071344 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990b0f3-472f-4960-a50c-aec11919ba24 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:36.104859 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:36.184895 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990b0f3-472f-4960-a50c-aec11919ba24 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:36.222418 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9335faf-8073-4316-84b2-ace02a3936dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:36.223087 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9335faf-8073-4316-84b2-ace02a3936dc None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:36.223369 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9335faf-8073-4316-84b2-ace02a3936dc None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:36.223644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9335faf-8073-4316-84b2-ace02a3936dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:36.234915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9335faf-8073-4316-84b2-ace02a3936dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-e9335faf-8073-4316-84b2-ace02a3936dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:19:36.243718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:36.244830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:36.245842 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:36.245842 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:36.245842 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:36.245842 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:36.245842 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:19:36.245842 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:19:36.245842 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:19:36.245842 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:19:36.245842 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:19:36.245842 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:19:36.245842 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:19:36.245842 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:36.245842 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 542/1087] 10.222.0.84 () {56 vars in 1120 bytes} [Thu May 19 20:19:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:36.251144 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87797c-73c7-48cf-acc5-cff93292fe44 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:36.360525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87797c-73c7-48cf-acc5-cff93292fe44 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:36.397182 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494463b-04a3-4cc8-aacc-dce0d1a6b474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:36.397848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494463b-04a3-4cc8-aacc-dce0d1a6b474 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:36.398127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494463b-04a3-4cc8-aacc-dce0d1a6b474 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:36.398422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494463b-04a3-4cc8-aacc-dce0d1a6b474 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:36.399163 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494463b-04a3-4cc8-aacc-dce0d1a6b474 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:36.399465 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494463b-04a3-4cc8-aacc-dce0d1a6b474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:36.400836 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494463b-04a3-4cc8-aacc-dce0d1a6b474 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:36.409064 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 546/1088] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 20:19:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:36.415220 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9335faf-8073-4316-84b2-ace02a3936dc None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:36.507604 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:36.510404 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6375c1-6547-40a8-9462-eb4fbb37412c None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:36.510713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6375c1-6547-40a8-9462-eb4fbb37412c None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:36.510991 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6375c1-6547-40a8-9462-eb4fbb37412c None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:36.512522 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 547/1089] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:19:36.517582 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f9359-89fb-4fed-8ba3-9c1fafbddc14 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:36.517850 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f9359-89fb-4fed-8ba3-9c1fafbddc14 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:36.518125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f9359-89fb-4fed-8ba3-9c1fafbddc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:36.519135 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9335faf-8073-4316-84b2-ace02a3936dc None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:36.534718 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-333f9359-89fb-4fed-8ba3-9c1fafbddc14 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:36.535908 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-333f9359-89fb-4fed-8ba3-9c1fafbddc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:36.555249 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb5bd2-059a-4e0d-9024-02c4c71d2936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:36.556071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb5bd2-059a-4e0d-9024-02c4c71d2936 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:36.556353 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb5bd2-059a-4e0d-9024-02c4c71d2936 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:36.556631 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb5bd2-059a-4e0d-9024-02c4c71d2936 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:36.567405 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb5bd2-059a-4e0d-9024-02c4c71d2936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:36.569084 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb5bd2-059a-4e0d-9024-02c4c71d2936 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:36.572591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:36.587025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-edcb5bd2-059a-4e0d-9024-02c4c71d2936 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:36.587025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:36.587025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:36.587025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:36.587025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:36.587025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:36.587025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:36.587025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:36.587025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:36.587025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:36.587025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:36.587025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:36.587025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:36.587025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:36.587025 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:36.588467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:36.589734 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:36.589734 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:36.589734 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:36.589734 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:36.589734 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 543/1090] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:36.594270 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb5bd2-059a-4e0d-9024-02c4c71d2936 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:36.608643 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 548/1091] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:19:36.632627 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f9359-89fb-4fed-8ba3-9c1fafbddc14 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:36.696797 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb5bd2-059a-4e0d-9024-02c4c71d2936 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:36.732067 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881ece3-1df7-4ab4-9c36-4942f8ce13c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:36.732296 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f9359-89fb-4fed-8ba3-9c1fafbddc14 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:36.732735 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881ece3-1df7-4ab4-9c36-4942f8ce13c7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:36.733016 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881ece3-1df7-4ab4-9c36-4942f8ce13c7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:36.733289 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881ece3-1df7-4ab4-9c36-4942f8ce13c7 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:36.734009 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881ece3-1df7-4ab4-9c36-4942f8ce13c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:36.734960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881ece3-1df7-4ab4-9c36-4942f8ce13c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:36.735731 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881ece3-1df7-4ab4-9c36-4942f8ce13c7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:36.747597 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 544/1092] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:36.753198 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881ece3-1df7-4ab4-9c36-4942f8ce13c7 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:36.755946 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:36.767276 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af11a5-35e7-48c6-a994-0f517c400621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:36.768180 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af11a5-35e7-48c6-a994-0f517c400621 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:36.768459 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af11a5-35e7-48c6-a994-0f517c400621 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:36.768732 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af11a5-35e7-48c6-a994-0f517c400621 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:36.779262 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af11a5-35e7-48c6-a994-0f517c400621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-82af11a5-35e7-48c6-a994-0f517c400621 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:19:36.787043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:36.788367 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:36.789467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:36.789467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:36.789467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:36.789467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:36.789467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:19:36.789467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:19:36.789467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:19:36.789467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:19:36.789467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:19:36.789467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:19:36.789467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:19:36.789467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:36.789467 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 549/1093] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:19:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:36.794434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af11a5-35e7-48c6-a994-0f517c400621 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:36.826797 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:36.857364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881ece3-1df7-4ab4-9c36-4942f8ce13c7 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:36.891673 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a371b19-2291-463b-8f04-969f2677f7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:36.892471 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a371b19-2291-463b-8f04-969f2677f7f7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:36.892749 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a371b19-2291-463b-8f04-969f2677f7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:36.893020 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a371b19-2291-463b-8f04-969f2677f7f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:36.899214 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af11a5-35e7-48c6-a994-0f517c400621 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:36.901087 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a371b19-2291-463b-8f04-969f2677f7f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:36.901903 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a371b19-2291-463b-8f04-969f2677f7f7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-3a371b19-2291-463b-8f04-969f2677f7f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:19:36.916406 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:36.917581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:36.918768 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:36.918768 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:36.918768 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:19:36.918768 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:19:36.918768 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:19:36.918768 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:19:36.918768 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:19:36.918768 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:19:36.918768 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:19:36.918768 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:36.918768 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 545/1094] 10.222.0.84 () {56 vars in 1093 bytes} [Thu May 19 20:19:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:19:36.923398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a371b19-2291-463b-8f04-969f2677f7f7 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:36.934660 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d68e5-7641-4cd2-8607-67ba854dcca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:36.935352 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d68e5-7641-4cd2-8607-67ba854dcca1 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:36.935677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d68e5-7641-4cd2-8607-67ba854dcca1 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:36.935988 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d68e5-7641-4cd2-8607-67ba854dcca1 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:36.936681 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d68e5-7641-4cd2-8607-67ba854dcca1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:36.936996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d68e5-7641-4cd2-8607-67ba854dcca1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:36.938405 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d68e5-7641-4cd2-8607-67ba854dcca1 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:36.947848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 550/1095] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:19:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:36.953380 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d68e5-7641-4cd2-8607-67ba854dcca1 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:36.988210 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:37.029470 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a371b19-2291-463b-8f04-969f2677f7f7 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:37.047193 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:37.054963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d68e5-7641-4cd2-8607-67ba854dcca1 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:37.063017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07314dcc-48bb-4409-a358-f6ff8b62cca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:37.063699 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07314dcc-48bb-4409-a358-f6ff8b62cca2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:37.063981 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07314dcc-48bb-4409-a358-f6ff8b62cca2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:37.064288 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07314dcc-48bb-4409-a358-f6ff8b62cca2 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:37.065115 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07314dcc-48bb-4409-a358-f6ff8b62cca2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:37.065392 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07314dcc-48bb-4409-a358-f6ff8b62cca2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:37.066957 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07314dcc-48bb-4409-a358-f6ff8b62cca2 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:37.083561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 546/1096] 10.222.0.84 () {56 vars in 1133 bytes} [Thu May 19 20:19:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:37.088995 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bbdfc-9c4d-40c1-88a8-9c63e1aba4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:37.089888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bbdfc-9c4d-40c1-88a8-9c63e1aba4f9 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:37.090010 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07314dcc-48bb-4409-a358-f6ff8b62cca2 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:37.090326 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bbdfc-9c4d-40c1-88a8-9c63e1aba4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:37.090628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bbdfc-9c4d-40c1-88a8-9c63e1aba4f9 None admin] PATH_INFO: `/v3/users/dd60ddb1b52548b6a347f3636750bc9f` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:37.091951 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bbdfc-9c4d-40c1-88a8-9c63e1aba4f9 None admin] RBAC: Authorizing `identity:get_user(user_id=dd60ddb1b52548b6a347f3636750bc9f)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:37.093408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bbdfc-9c4d-40c1-88a8-9c63e1aba4f9 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:37.095077 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 551/1097] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:19:36 2022] GET /identity/v3/users/dd60ddb1b52548b6a347f3636750bc9f => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:37.100904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bbdfc-9c4d-40c1-88a8-9c63e1aba4f9 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:37.192509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07314dcc-48bb-4409-a358-f6ff8b62cca2 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:37.207658 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bbdfc-9c4d-40c1-88a8-9c63e1aba4f9 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:37.214234 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:37.230320 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0ca53-061e-4e9d-a5d1-a65dc329c24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:37.231331 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0ca53-061e-4e9d-a5d1-a65dc329c24b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:37.231849 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0ca53-061e-4e9d-a5d1-a65dc329c24b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:37.232350 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0ca53-061e-4e9d-a5d1-a65dc329c24b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:37.243933 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6a23c-0715-44c8-996d-184326e93b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:37.244597 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6a23c-0715-44c8-996d-184326e93b26 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:37.244885 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6a23c-0715-44c8-996d-184326e93b26 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:37.245188 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6a23c-0715-44c8-996d-184326e93b26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:37.275565 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:37.305060 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0ca53-061e-4e9d-a5d1-a65dc329c24b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:37.306880 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0ca53-061e-4e9d-a5d1-a65dc329c24b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:37.315190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6a23c-0715-44c8-996d-184326e93b26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:37.316663 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6a23c-0715-44c8-996d-184326e93b26 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:37.411528 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 547/1098] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:19:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:19:37.417572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0ca53-061e-4e9d-a5d1-a65dc329c24b None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:37.423401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 552/1099] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:19:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:19:37.424489 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6a23c-0715-44c8-996d-184326e93b26 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:37.439318 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:37.484723 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:37.519118 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0ca53-061e-4e9d-a5d1-a65dc329c24b None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:37.524948 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6a23c-0715-44c8-996d-184326e93b26 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:37.554056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be844cfc-0b4f-469b-914c-60b8d4f38810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:37.554897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be844cfc-0b4f-469b-914c-60b8d4f38810 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:37.555233 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be844cfc-0b4f-469b-914c-60b8d4f38810 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:37.555462 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be844cfc-0b4f-469b-914c-60b8d4f38810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:37.560303 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38dbd9-e5f9-449d-a754-21e9bcd02931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:37.561168 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38dbd9-e5f9-449d-a754-21e9bcd02931 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:37.561449 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38dbd9-e5f9-449d-a754-21e9bcd02931 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:37.561727 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38dbd9-e5f9-449d-a754-21e9bcd02931 None admin] PATH_INFO: `/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:37.562949 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38dbd9-e5f9-449d-a754-21e9bcd02931 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e7d4494dfb4f96ae2fa5c5641fc6cc)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:37.564434 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38dbd9-e5f9-449d-a754-21e9bcd02931 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:37.566164 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 553/1100] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:19:37 2022] GET /identity/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:37.566814 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be844cfc-0b4f-469b-914c-60b8d4f38810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:37.568273 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be844cfc-0b4f-469b-914c-60b8d4f38810 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:37.572923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38dbd9-e5f9-449d-a754-21e9bcd02931 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:37.586669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-be844cfc-0b4f-469b-914c-60b8d4f38810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:37.586669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:37.586669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:37.586669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:37.586669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:37.586669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:37.586669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:37.586669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:37.586669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:37.586669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:37.586669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:37.586669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:37.586669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:19:37.586669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:19:37.586669 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:37.587965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:37.589354 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:19:37.589354 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:19:37.589354 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:19:37.589354 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:37.589354 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 548/1101] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:37.593832 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be844cfc-0b4f-469b-914c-60b8d4f38810 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:37.672396 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:37.687583 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38dbd9-e5f9-449d-a754-21e9bcd02931 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:37.703655 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be844cfc-0b4f-469b-914c-60b8d4f38810 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:37.718800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:37.725192 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53bf55-ed87-4a6c-aa2c-407b69b3fc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:37.725860 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53bf55-ed87-4a6c-aa2c-407b69b3fc9e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:37.726135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53bf55-ed87-4a6c-aa2c-407b69b3fc9e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:37.726448 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53bf55-ed87-4a6c-aa2c-407b69b3fc9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:37.727263 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53bf55-ed87-4a6c-aa2c-407b69b3fc9e None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=e5e7d4494dfb4f96ae2fa5c5641fc6cc, user.id=dd60ddb1b52548b6a347f3636750bc9f) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:37.727570 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53bf55-ed87-4a6c-aa2c-407b69b3fc9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:37.728991 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53bf55-ed87-4a6c-aa2c-407b69b3fc9e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:37.734981 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 554/1102] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:19:37 2022] GET /identity/v3/role_assignments?user.id=dd60ddb1b52548b6a347f3636750bc9f&scope.project.id=e5e7d4494dfb4f96ae2fa5c5641fc6cc&role.id=b239f941cb724331bf28dc489b70ad1d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:37.740996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff0878-18c5-430e-884b-3712a6f9d2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:37.741774 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff0878-18c5-430e-884b-3712a6f9d2e2 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:37.742062 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff0878-18c5-430e-884b-3712a6f9d2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:37.742342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff0878-18c5-430e-884b-3712a6f9d2e2 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:37.743087 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff0878-18c5-430e-884b-3712a6f9d2e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:37.743391 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff0878-18c5-430e-884b-3712a6f9d2e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:37.744814 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff0878-18c5-430e-884b-3712a6f9d2e2 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:37.757820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 549/1103] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:19:37.764040 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53bf55-ed87-4a6c-aa2c-407b69b3fc9e None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:37.874771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53bf55-ed87-4a6c-aa2c-407b69b3fc9e None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:37.912595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0dc10-8e32-477d-b6f6-d1fc4d331ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:37.913506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0dc10-8e32-477d-b6f6-d1fc4d331ff9 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:37.913787 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0dc10-8e32-477d-b6f6-d1fc4d331ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:37.914064 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0dc10-8e32-477d-b6f6-d1fc4d331ff9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:37.921940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0dc10-8e32-477d-b6f6-d1fc4d331ff9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:37.923461 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0dc10-8e32-477d-b6f6-d1fc4d331ff9 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:37.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-3cf0dc10-8e32-477d-b6f6-d1fc4d331ff9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:37.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:37.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:37.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:37.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:37.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:37.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:37.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:37.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:37.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:37.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:37.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:37.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:19:37.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:19:37.938935 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:37.940237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:37.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:19:37.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:19:37.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:19:37.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:19:37.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:19:37.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:19:37.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:19:37.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:19:37.941557 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 555/1104] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:19:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:37.947404 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff0878-18c5-430e-884b-3712a6f9d2e2 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:38.054921 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff0878-18c5-430e-884b-3712a6f9d2e2 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:38.085644 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:38.092070 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99d736-3963-4d50-b261-80acec7750de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:38.092746 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99d736-3963-4d50-b261-80acec7750de None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:38.093027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99d736-3963-4d50-b261-80acec7750de None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:38.093302 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99d736-3963-4d50-b261-80acec7750de None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:38.094096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99d736-3963-4d50-b261-80acec7750de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:38.094448 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99d736-3963-4d50-b261-80acec7750de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:38.095945 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99d736-3963-4d50-b261-80acec7750de None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:38.108887 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 550/1105] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:19:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:38.115081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0dc10-8e32-477d-b6f6-d1fc4d331ff9 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:38.174829 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:38.221830 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0dc10-8e32-477d-b6f6-d1fc4d331ff9 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:38.256116 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bd4ed-f4ee-43b0-9e5e-b88e15301728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:38.256958 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bd4ed-f4ee-43b0-9e5e-b88e15301728 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:38.257242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bd4ed-f4ee-43b0-9e5e-b88e15301728 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:38.257516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bd4ed-f4ee-43b0-9e5e-b88e15301728 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:38.258376 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017bd4ed-f4ee-43b0-9e5e-b88e15301728 None admin] RBAC: Adding query filter params (role.id=943b7648db36455b8073c12e3b45374d, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=edb1bcb781084f03a4ad2611a09bd77e) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:38.258689 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017bd4ed-f4ee-43b0-9e5e-b88e15301728 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:38.260123 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017bd4ed-f4ee-43b0-9e5e-b88e15301728 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:38.266274 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 556/1106] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:19:38 2022] GET /identity/v3/role_assignments?user.id=edb1bcb781084f03a4ad2611a09bd77e&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=943b7648db36455b8073c12e3b45374d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:40.019108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b21b13-cc38-476b-9fb1-e25928807e88 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:40.019732 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b21b13-cc38-476b-9fb1-e25928807e88 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:40.019732 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b21b13-cc38-476b-9fb1-e25928807e88 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:40.021045 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 551/1107] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:40.028360 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09f725-65c1-4497-abb3-c05f0f919d3c None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:40.029158 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09f725-65c1-4497-abb3-c05f0f919d3c None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:40.029366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09f725-65c1-4497-abb3-c05f0f919d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:40.056443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-ba09f725-65c1-4497-abb3-c05f0f919d3c None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:40.057628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-ba09f725-65c1-4497-abb3-c05f0f919d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:40.138292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 557/1108] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:40.163121 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b21b13-cc38-476b-9fb1-e25928807e88 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:40.274713 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b21b13-cc38-476b-9fb1-e25928807e88 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:40.313563 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18c279-961f-42bc-bf2a-058250061bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:40.314433 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18c279-961f-42bc-bf2a-058250061bc7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:40.314719 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18c279-961f-42bc-bf2a-058250061bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:40.314997 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18c279-961f-42bc-bf2a-058250061bc7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:40.316339 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18c279-961f-42bc-bf2a-058250061bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:40.317833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18c279-961f-42bc-bf2a-058250061bc7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:40.319739 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 552/1109] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:40.325853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09f725-65c1-4497-abb3-c05f0f919d3c None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:40.379795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:40.434016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09f725-65c1-4497-abb3-c05f0f919d3c None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:40.471399 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf8ca4-ce43-4093-873c-096ecf0b34a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:40.472097 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf8ca4-ce43-4093-873c-096ecf0b34a6 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:40.472381 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf8ca4-ce43-4093-873c-096ecf0b34a6 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:40.472659 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf8ca4-ce43-4093-873c-096ecf0b34a6 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:40.473372 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf8ca4-ce43-4093-873c-096ecf0b34a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:40.474937 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf8ca4-ce43-4093-873c-096ecf0b34a6 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:40.500179 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 558/1110] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:19:40 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:19:42.749613 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07179e71-d87b-4f16-8d12-dd555ca4877c None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:42.750184 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07179e71-d87b-4f16-8d12-dd555ca4877c None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:42.750184 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07179e71-d87b-4f16-8d12-dd555ca4877c None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:42.751474 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 553/1111] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:42.758620 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6130a6-9e41-4f91-b8b0-6caecc0568e7 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:42.759432 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6130a6-9e41-4f91-b8b0-6caecc0568e7 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:42.759609 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6130a6-9e41-4f91-b8b0-6caecc0568e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:42.788895 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-da6130a6-9e41-4f91-b8b0-6caecc0568e7 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:42.790125 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-da6130a6-9e41-4f91-b8b0-6caecc0568e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:42.864046 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 559/1112] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:42.889157 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07179e71-d87b-4f16-8d12-dd555ca4877c None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:42.958514 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:43.003806 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07179e71-d87b-4f16-8d12-dd555ca4877c None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:43.046945 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ba7d4-c502-4801-95f0-383348f63c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:43.048252 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ba7d4-c502-4801-95f0-383348f63c14 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:43.048654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ba7d4-c502-4801-95f0-383348f63c14 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:43.049049 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ba7d4-c502-4801-95f0-383348f63c14 None admin] PATH_INFO: `/v3/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:43.051656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ba7d4-c502-4801-95f0-383348f63c14 None admin] RBAC: Authorizing `identity:get_role(role_id=6d25238fbc0e4d11bac1bcdc7e50d2e2)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:43.053880 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ba7d4-c502-4801-95f0-383348f63c14 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:43.055312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 554/1113] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:19:42 2022] GET /identity/v3/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 223 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:43.064672 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6130a6-9e41-4f91-b8b0-6caecc0568e7 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:43.108832 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:43.181905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6130a6-9e41-4f91-b8b0-6caecc0568e7 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:43.217788 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efec4fa-188c-4584-be4e-8cdfc28a58e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:43.218697 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efec4fa-188c-4584-be4e-8cdfc28a58e1 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:43.218961 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efec4fa-188c-4584-be4e-8cdfc28a58e1 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:43.219237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efec4fa-188c-4584-be4e-8cdfc28a58e1 None admin] PATH_INFO: `/v3/users/faad8c651164443fb8143577ee996f7e` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:43.234698 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efec4fa-188c-4584-be4e-8cdfc28a58e1 None admin] RBAC: Authorizing `identity:get_user(user_id=faad8c651164443fb8143577ee996f7e)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:43.236549 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efec4fa-188c-4584-be4e-8cdfc28a58e1 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:43.238290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 560/1114] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:19:43 2022] GET /identity/v3/users/faad8c651164443fb8143577ee996f7e => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:43.245511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ba7d4-c502-4801-95f0-383348f63c14 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:43.356636 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ba7d4-c502-4801-95f0-383348f63c14 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:43.384418 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:43.393381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fb98e-d1ff-4353-88b0-8ee529e9ede8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:43.394043 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fb98e-d1ff-4353-88b0-8ee529e9ede8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:43.394354 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fb98e-d1ff-4353-88b0-8ee529e9ede8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:43.394637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fb98e-d1ff-4353-88b0-8ee529e9ede8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:43.469638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735fb98e-d1ff-4353-88b0-8ee529e9ede8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:43.471148 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735fb98e-d1ff-4353-88b0-8ee529e9ede8 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:43.576362 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 555/1115] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:19:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:19:43.586339 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efec4fa-188c-4584-be4e-8cdfc28a58e1 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:43.700380 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:43.714412 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efec4fa-188c-4584-be4e-8cdfc28a58e1 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:43.753410 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c779f17-b114-420a-8f4f-fd9cd404e951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:43.754287 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c779f17-b114-420a-8f4f-fd9cd404e951 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:43.754591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c779f17-b114-420a-8f4f-fd9cd404e951 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:43.754866 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c779f17-b114-420a-8f4f-fd9cd404e951 None admin] PATH_INFO: `/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:43.756135 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c779f17-b114-420a-8f4f-fd9cd404e951 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e7d4494dfb4f96ae2fa5c5641fc6cc)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:43.757765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c779f17-b114-420a-8f4f-fd9cd404e951 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:43.759568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 561/1116] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:19:43 2022] GET /identity/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:43.769908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fb98e-d1ff-4353-88b0-8ee529e9ede8 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:43.799161 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:43.874064 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fb98e-d1ff-4353-88b0-8ee529e9ede8 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:43.917182 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c934dc-6ea9-4881-b045-051270e011d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:43.918188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c934dc-6ea9-4881-b045-051270e011d0 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:43.918627 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c934dc-6ea9-4881-b045-051270e011d0 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:43.919049 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c934dc-6ea9-4881-b045-051270e011d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:43.920380 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c934dc-6ea9-4881-b045-051270e011d0 None admin] RBAC: Adding query filter params (role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2, scope.project.id=e5e7d4494dfb4f96ae2fa5c5641fc6cc, user.id=faad8c651164443fb8143577ee996f7e) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:43.920867 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c934dc-6ea9-4881-b045-051270e011d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:43.923258 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c934dc-6ea9-4881-b045-051270e011d0 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:43.931413 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 556/1117] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:19:43 2022] GET /identity/v3/role_assignments?user.id=faad8c651164443fb8143577ee996f7e&scope.project.id=e5e7d4494dfb4f96ae2fa5c5641fc6cc&role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:46.213871 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bd41b-300b-412e-bcfa-66662c7d8fc4 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:46.214813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bd41b-300b-412e-bcfa-66662c7d8fc4 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:46.214813 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bd41b-300b-412e-bcfa-66662c7d8fc4 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:46.217074 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 562/1118] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:46.223637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8536b1-1939-46aa-b23b-cd77edb99e13 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:46.224327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8536b1-1939-46aa-b23b-cd77edb99e13 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:46.224987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8536b1-1939-46aa-b23b-cd77edb99e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:46.248566 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-ac8536b1-1939-46aa-b23b-cd77edb99e13 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:46.249751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-ac8536b1-1939-46aa-b23b-cd77edb99e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:46.323745 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 557/1119] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:46.347692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bd41b-300b-412e-bcfa-66662c7d8fc4 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:46.459174 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bd41b-300b-412e-bcfa-66662c7d8fc4 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:46.496033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cba742-f653-477d-b710-e89ea230d277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:46.496818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cba742-f653-477d-b710-e89ea230d277 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:46.497100 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cba742-f653-477d-b710-e89ea230d277 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:46.497378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cba742-f653-477d-b710-e89ea230d277 None admin] PATH_INFO: `/v3/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:46.499007 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cba742-f653-477d-b710-e89ea230d277 None admin] RBAC: Authorizing `identity:get_role(role_id=6d25238fbc0e4d11bac1bcdc7e50d2e2)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:46.500397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cba742-f653-477d-b710-e89ea230d277 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:46.501802 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 563/1120] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:19:46 2022] GET /identity/v3/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:46.508446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8536b1-1939-46aa-b23b-cd77edb99e13 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:46.534786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:46.630299 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8536b1-1939-46aa-b23b-cd77edb99e13 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:46.666913 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc865d37-4be4-42d4-b301-f7109cb4c1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:46.667795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc865d37-4be4-42d4-b301-f7109cb4c1ea None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:46.668076 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc865d37-4be4-42d4-b301-f7109cb4c1ea None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:46.668350 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc865d37-4be4-42d4-b301-f7109cb4c1ea None admin] PATH_INFO: `/v3/users/faad8c651164443fb8143577ee996f7e` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:46.669569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc865d37-4be4-42d4-b301-f7109cb4c1ea None admin] RBAC: Authorizing `identity:get_user(user_id=faad8c651164443fb8143577ee996f7e)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:46.671057 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc865d37-4be4-42d4-b301-f7109cb4c1ea None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:46.672698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 558/1121] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:19:46 2022] GET /identity/v3/users/faad8c651164443fb8143577ee996f7e => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:46.680599 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cba742-f653-477d-b710-e89ea230d277 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:46.710517 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:46.787105 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cba742-f653-477d-b710-e89ea230d277 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:46.824838 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5aaab1-2315-4422-9cd0-af68370802fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:46.825682 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5aaab1-2315-4422-9cd0-af68370802fa None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:46.825959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5aaab1-2315-4422-9cd0-af68370802fa None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:46.826231 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5aaab1-2315-4422-9cd0-af68370802fa None admin] PATH_INFO: `/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:46.827464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5aaab1-2315-4422-9cd0-af68370802fa None admin] RBAC: Authorizing `identity:get_project(project_id=e5e7d4494dfb4f96ae2fa5c5641fc6cc)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:46.828916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5aaab1-2315-4422-9cd0-af68370802fa None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:46.830667 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 564/1122] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:19:46 2022] GET /identity/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:46.836895 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc865d37-4be4-42d4-b301-f7109cb4c1ea None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:46.935721 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:46.946848 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc865d37-4be4-42d4-b301-f7109cb4c1ea None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:46.982531 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c02d64-124b-4bdb-a584-32d9df32d5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:46.983431 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c02d64-124b-4bdb-a584-32d9df32d5e6 None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:46.983711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c02d64-124b-4bdb-a584-32d9df32d5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:46.983976 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c02d64-124b-4bdb-a584-32d9df32d5e6 None admin] PATH_INFO: `/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc/users/faad8c651164443fb8143577ee996f7e/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:46.986281 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c02d64-124b-4bdb-a584-32d9df32d5e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e7d4494dfb4f96ae2fa5c5641fc6cc, user_id=faad8c651164443fb8143577ee996f7e, role_id=6d25238fbc0e4d11bac1bcdc7e50d2e2)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:46.987748 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c02d64-124b-4bdb-a584-32d9df32d5e6 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:47.007355 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 559/1123] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:19:46 2022] PUT /identity/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc/users/faad8c651164443fb8143577ee996f7e/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:19:49.197342 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501f9c0-29ca-4a10-93cc-8860d102252a None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:49.197927 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501f9c0-29ca-4a10-93cc-8860d102252a None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:49.197927 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501f9c0-29ca-4a10-93cc-8860d102252a None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:49.199316 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 565/1124] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:49.205874 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d7fa4-fa31-4966-8041-c29f23d7ac07 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:49.206703 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d7fa4-fa31-4966-8041-c29f23d7ac07 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:49.206868 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d7fa4-fa31-4966-8041-c29f23d7ac07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:49.227990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-2f3d7fa4-fa31-4966-8041-c29f23d7ac07 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:49.229187 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-2f3d7fa4-fa31-4966-8041-c29f23d7ac07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:49.305260 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 560/1125] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:49.335280 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501f9c0-29ca-4a10-93cc-8860d102252a None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:49.443472 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:49.451933 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501f9c0-29ca-4a10-93cc-8860d102252a None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:49.492703 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34c54f-dcb8-4aa9-9bb1-1708dd8b2626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:49.493559 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34c54f-dcb8-4aa9-9bb1-1708dd8b2626 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:49.493843 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34c54f-dcb8-4aa9-9bb1-1708dd8b2626 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:49.494119 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34c54f-dcb8-4aa9-9bb1-1708dd8b2626 None admin] PATH_INFO: `/v3/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:49.495762 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34c54f-dcb8-4aa9-9bb1-1708dd8b2626 None admin] RBAC: Authorizing `identity:get_role(role_id=6d25238fbc0e4d11bac1bcdc7e50d2e2)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:49.497141 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34c54f-dcb8-4aa9-9bb1-1708dd8b2626 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:49.498572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 566/1126] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:19:49 2022] GET /identity/v3/roles/6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 223 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:49.505256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d7fa4-fa31-4966-8041-c29f23d7ac07 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:49.591575 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:49.612437 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d7fa4-fa31-4966-8041-c29f23d7ac07 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:49.648492 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b75c6-d2d8-4428-bf59-563c6f0600bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:49.649363 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b75c6-d2d8-4428-bf59-563c6f0600bd None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:49.649640 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b75c6-d2d8-4428-bf59-563c6f0600bd None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:49.649919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b75c6-d2d8-4428-bf59-563c6f0600bd None admin] PATH_INFO: `/v3/users/faad8c651164443fb8143577ee996f7e` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:49.651204 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b75c6-d2d8-4428-bf59-563c6f0600bd None admin] RBAC: Authorizing `identity:get_user(user_id=faad8c651164443fb8143577ee996f7e)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:49.652678 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b75c6-d2d8-4428-bf59-563c6f0600bd None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:49.654381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 561/1127] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:19:49 2022] GET /identity/v3/users/faad8c651164443fb8143577ee996f7e => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:49.660452 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34c54f-dcb8-4aa9-9bb1-1708dd8b2626 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:49.770403 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34c54f-dcb8-4aa9-9bb1-1708dd8b2626 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:49.806106 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6bf3bd-89cf-445b-98a4-288f75c8cf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:49.806785 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6bf3bd-89cf-445b-98a4-288f75c8cf12 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:49.807072 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6bf3bd-89cf-445b-98a4-288f75c8cf12 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:49.807349 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6bf3bd-89cf-445b-98a4-288f75c8cf12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:49.839996 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:49.878535 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6bf3bd-89cf-445b-98a4-288f75c8cf12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:49.880000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6bf3bd-89cf-445b-98a4-288f75c8cf12 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:49.989485 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 567/1128] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:19:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:19:49.996248 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b75c6-d2d8-4428-bf59-563c6f0600bd None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:50.116875 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b75c6-d2d8-4428-bf59-563c6f0600bd None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:50.155336 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e51e3-4f05-4d51-89d5-c3eb51788aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:50.156206 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e51e3-4f05-4d51-89d5-c3eb51788aa1 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:50.156486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e51e3-4f05-4d51-89d5-c3eb51788aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:50.156765 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e51e3-4f05-4d51-89d5-c3eb51788aa1 None admin] PATH_INFO: `/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:50.158034 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e51e3-4f05-4d51-89d5-c3eb51788aa1 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e7d4494dfb4f96ae2fa5c5641fc6cc)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:50.159548 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e51e3-4f05-4d51-89d5-c3eb51788aa1 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:50.161267 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 562/1129] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:19:49 2022] GET /identity/v3/projects/e5e7d4494dfb4f96ae2fa5c5641fc6cc => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:50.168006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6bf3bd-89cf-445b-98a4-288f75c8cf12 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:50.234661 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:50.275153 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6bf3bd-89cf-445b-98a4-288f75c8cf12 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:50.311078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b3c1e-e094-49c2-9ef1-06b2687d26ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:50.311736 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b3c1e-e094-49c2-9ef1-06b2687d26ae None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:50.312015 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b3c1e-e094-49c2-9ef1-06b2687d26ae None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:50.312296 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b3c1e-e094-49c2-9ef1-06b2687d26ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:50.313120 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b3c1e-e094-49c2-9ef1-06b2687d26ae None admin] RBAC: Adding query filter params (role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2, scope.project.id=e5e7d4494dfb4f96ae2fa5c5641fc6cc, user.id=faad8c651164443fb8143577ee996f7e) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:50.313437 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b3c1e-e094-49c2-9ef1-06b2687d26ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:50.314902 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b3c1e-e094-49c2-9ef1-06b2687d26ae None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:50.321793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 568/1130] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:19:50 2022] GET /identity/v3/role_assignments?user.id=faad8c651164443fb8143577ee996f7e&scope.project.id=e5e7d4494dfb4f96ae2fa5c5641fc6cc&role.id=6d25238fbc0e4d11bac1bcdc7e50d2e2 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:52.557453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6549b-b4a1-48d4-9157-70a545549000 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:52.558048 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6549b-b4a1-48d4-9157-70a545549000 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:52.558048 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6549b-b4a1-48d4-9157-70a545549000 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:52.559418 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 563/1131] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:52.565979 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1f7fb-3ef6-4734-a4f9-8b289f5cd87c None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:52.566766 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1f7fb-3ef6-4734-a4f9-8b289f5cd87c None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:52.566986 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1f7fb-3ef6-4734-a4f9-8b289f5cd87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:52.585704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-c0f1f7fb-3ef6-4734-a4f9-8b289f5cd87c None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:52.586938 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-c0f1f7fb-3ef6-4734-a4f9-8b289f5cd87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:52.659677 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 569/1132] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:52.683778 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6549b-b4a1-48d4-9157-70a545549000 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:52.705856 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:52.805989 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6549b-b4a1-48d4-9157-70a545549000 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:52.842181 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89928a9-8ec1-42ec-99a5-55a6bb900744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:52.843026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89928a9-8ec1-42ec-99a5-55a6bb900744 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:52.843283 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89928a9-8ec1-42ec-99a5-55a6bb900744 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:52.843558 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89928a9-8ec1-42ec-99a5-55a6bb900744 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:52.844866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89928a9-8ec1-42ec-99a5-55a6bb900744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:52.846329 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89928a9-8ec1-42ec-99a5-55a6bb900744 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:52.847992 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 564/1133] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:52.854911 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1f7fb-3ef6-4734-a4f9-8b289f5cd87c None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:52.916642 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:52.959565 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1f7fb-3ef6-4734-a4f9-8b289f5cd87c None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:52.997613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dda146-8bfd-4a11-8bdf-e938a310d48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:52.998299 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dda146-8bfd-4a11-8bdf-e938a310d48e None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:52.998578 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dda146-8bfd-4a11-8bdf-e938a310d48e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:52.998846 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dda146-8bfd-4a11-8bdf-e938a310d48e None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:52.999547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dda146-8bfd-4a11-8bdf-e938a310d48e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:53.000992 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dda146-8bfd-4a11-8bdf-e938a310d48e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:53.031782 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 570/1134] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:19:52 2022] POST /identity/v3/projects => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:19:55.298316 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93908b1-e1b3-4594-ba42-3ca90fa1e080 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:55.298896 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93908b1-e1b3-4594-ba42-3ca90fa1e080 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:55.298896 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93908b1-e1b3-4594-ba42-3ca90fa1e080 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:55.300261 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 565/1135] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:55.306775 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493df93-d082-4e03-a7d4-cf85f08fb964 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:55.307543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493df93-d082-4e03-a7d4-cf85f08fb964 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:55.307725 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493df93-d082-4e03-a7d4-cf85f08fb964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:55.329653 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-6493df93-d082-4e03-a7d4-cf85f08fb964 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:55.330889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-6493df93-d082-4e03-a7d4-cf85f08fb964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:55.402753 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 571/1136] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:55.432309 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93908b1-e1b3-4594-ba42-3ca90fa1e080 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:55.527172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:55.544979 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93908b1-e1b3-4594-ba42-3ca90fa1e080 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:55.589656 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b41389-7bb6-474c-af07-95a496bf9d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:55.590833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b41389-7bb6-474c-af07-95a496bf9d51 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:55.591158 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b41389-7bb6-474c-af07-95a496bf9d51 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:55.591536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b41389-7bb6-474c-af07-95a496bf9d51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:55.593574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b41389-7bb6-474c-af07-95a496bf9d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:55.595953 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b41389-7bb6-474c-af07-95a496bf9d51 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:55.598594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 566/1137] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:19:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:55.604832 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493df93-d082-4e03-a7d4-cf85f08fb964 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:55.650081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:55.717613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493df93-d082-4e03-a7d4-cf85f08fb964 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:55.756512 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27f7d4-bed5-47b2-8588-9d93bbabe777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:55.757190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27f7d4-bed5-47b2-8588-9d93bbabe777 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:55.757480 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27f7d4-bed5-47b2-8588-9d93bbabe777 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:55.757745 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27f7d4-bed5-47b2-8588-9d93bbabe777 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:55.758484 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27f7d4-bed5-47b2-8588-9d93bbabe777 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:55.759925 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27f7d4-bed5-47b2-8588-9d93bbabe777 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:55.783658 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 572/1138] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:19:55 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:19:58.052755 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d965dc-ffcf-4f68-bf37-2d2c4de61155 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:58.053665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d965dc-ffcf-4f68-bf37-2d2c4de61155 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:58.053665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d965dc-ffcf-4f68-bf37-2d2c4de61155 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:58.055883 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 567/1139] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:19:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:58.065009 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8ca7b-8433-4d5d-8ac3-86f52bd097c9 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:58.065445 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8ca7b-8433-4d5d-8ac3-86f52bd097c9 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:58.065583 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8ca7b-8433-4d5d-8ac3-86f52bd097c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:58.091101 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-44d8ca7b-8433-4d5d-8ac3-86f52bd097c9 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:58.092296 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-44d8ca7b-8433-4d5d-8ac3-86f52bd097c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:58.164542 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 573/1140] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:19:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:19:58.188210 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d965dc-ffcf-4f68-bf37-2d2c4de61155 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:58.296655 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d965dc-ffcf-4f68-bf37-2d2c4de61155 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:58.341115 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b0345-f0cb-4170-bfd1-c68ef963690a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:58.341841 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b0345-f0cb-4170-bfd1-c68ef963690a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:58.342122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b0345-f0cb-4170-bfd1-c68ef963690a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:58.342428 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b0345-f0cb-4170-bfd1-c68ef963690a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:58.354373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b0345-f0cb-4170-bfd1-c68ef963690a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-948b0345-f0cb-4170-bfd1-c68ef963690a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:19:58.362075 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:19:58.363373 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:19:58.364384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:19:58.364384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:19:58.364384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:19:58.364384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:19:58.364384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:19:58.364384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:19:58.364384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:19:58.364384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:19:58.364384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:19:58.364384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:19:58.364384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:19:58.364384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:19:58.364384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 568/1141] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:19:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:19:58.369787 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8ca7b-8433-4d5d-8ac3-86f52bd097c9 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:58.401095 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:58.476967 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8ca7b-8433-4d5d-8ac3-86f52bd097c9 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:58.512922 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458494e-30bd-4718-9299-74ef7c7439a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:58.513590 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458494e-30bd-4718-9299-74ef7c7439a4 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:58.513867 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458494e-30bd-4718-9299-74ef7c7439a4 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:58.514142 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458494e-30bd-4718-9299-74ef7c7439a4 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:58.514918 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458494e-30bd-4718-9299-74ef7c7439a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:58.515221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458494e-30bd-4718-9299-74ef7c7439a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:58.516601 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458494e-30bd-4718-9299-74ef7c7439a4 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:58.525146 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 574/1142] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:19:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:58.531869 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b0345-f0cb-4170-bfd1-c68ef963690a None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:58.550079 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:58.639725 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b0345-f0cb-4170-bfd1-c68ef963690a None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:58.675810 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6e2d5-d0bc-4f11-bd7d-46b0879f05f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:58.676685 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6e2d5-d0bc-4f11-bd7d-46b0879f05f5 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:58.676967 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6e2d5-d0bc-4f11-bd7d-46b0879f05f5 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:58.677353 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6e2d5-d0bc-4f11-bd7d-46b0879f05f5 None admin] PATH_INFO: `/v3/users/f090ff54b64e462ca333e4c9c532ba36` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:58.691575 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6e2d5-d0bc-4f11-bd7d-46b0879f05f5 None admin] RBAC: Authorizing `identity:get_user(user_id=f090ff54b64e462ca333e4c9c532ba36)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:58.693527 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6e2d5-d0bc-4f11-bd7d-46b0879f05f5 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:58.695557 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 569/1143] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:19:58 2022] GET /identity/v3/users/f090ff54b64e462ca333e4c9c532ba36 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:58.701424 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458494e-30bd-4718-9299-74ef7c7439a4 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:58.793647 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:58.808366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458494e-30bd-4718-9299-74ef7c7439a4 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:58.844313 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e181712-d101-4245-90dd-e5ab7a6d701c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:58.844958 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e181712-d101-4245-90dd-e5ab7a6d701c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:58.845242 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e181712-d101-4245-90dd-e5ab7a6d701c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:58.845518 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e181712-d101-4245-90dd-e5ab7a6d701c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:58.914991 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e181712-d101-4245-90dd-e5ab7a6d701c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:58.916366 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e181712-d101-4245-90dd-e5ab7a6d701c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:59.015902 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:59.030708 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 575/1144] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:19:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:19:59.037525 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6e2d5-d0bc-4f11-bd7d-46b0879f05f5 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:59.111552 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:59.147685 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6e2d5-d0bc-4f11-bd7d-46b0879f05f5 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:59.194053 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dfaaf-6b1f-4ab1-b2ef-08fdeb0bd54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:59.194898 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dfaaf-6b1f-4ab1-b2ef-08fdeb0bd54f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:59.195179 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dfaaf-6b1f-4ab1-b2ef-08fdeb0bd54f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:59.195453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dfaaf-6b1f-4ab1-b2ef-08fdeb0bd54f None admin] PATH_INFO: `/v3/projects/e05db95889c24bda9c5c13dd8d4992bd` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:59.196697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981dfaaf-6b1f-4ab1-b2ef-08fdeb0bd54f None admin] RBAC: Authorizing `identity:get_project(project_id=e05db95889c24bda9c5c13dd8d4992bd)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:59.198156 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981dfaaf-6b1f-4ab1-b2ef-08fdeb0bd54f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:59.199895 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 570/1145] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:19:59 2022] GET /identity/v3/projects/e05db95889c24bda9c5c13dd8d4992bd => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:19:59.209794 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e181712-d101-4245-90dd-e5ab7a6d701c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:59.319487 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e181712-d101-4245-90dd-e5ab7a6d701c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:59.355516 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e16a8-7afa-4634-a78e-a86b802a8112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:59.356169 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e16a8-7afa-4634-a78e-a86b802a8112 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:59.356443 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e16a8-7afa-4634-a78e-a86b802a8112 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:59.356715 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e16a8-7afa-4634-a78e-a86b802a8112 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:59.357512 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e16a8-7afa-4634-a78e-a86b802a8112 None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=e05db95889c24bda9c5c13dd8d4992bd, user.id=f090ff54b64e462ca333e4c9c532ba36) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:19:59.357818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e16a8-7afa-4634-a78e-a86b802a8112 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:59.359298 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e16a8-7afa-4634-a78e-a86b802a8112 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:59.365222 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 576/1146] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:19:59 2022] GET /identity/v3/role_assignments?user.id=f090ff54b64e462ca333e4c9c532ba36&scope.project.id=e05db95889c24bda9c5c13dd8d4992bd&role.id=b239f941cb724331bf28dc489b70ad1d => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:01.535642 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade02764-0b03-44ad-8972-d14b9894ff00 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:01.536602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade02764-0b03-44ad-8972-d14b9894ff00 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:01.536602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade02764-0b03-44ad-8972-d14b9894ff00 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:01.538836 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 571/1147] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:20:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:20:01.546920 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dec44c-6c39-4dad-a14f-3112c3be93d4 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:01.547409 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dec44c-6c39-4dad-a14f-3112c3be93d4 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:01.547409 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dec44c-6c39-4dad-a14f-3112c3be93d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:01.566015 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-77dec44c-6c39-4dad-a14f-3112c3be93d4 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:01.567291 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-77dec44c-6c39-4dad-a14f-3112c3be93d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:01.616244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:01.650747 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 577/1148] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:20:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:20:01.673541 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade02764-0b03-44ad-8972-d14b9894ff00 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:01.790597 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade02764-0b03-44ad-8972-d14b9894ff00 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:01.828057 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa9c17-f9de-43fc-a953-2bd306a7b9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:01.828852 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa9c17-f9de-43fc-a953-2bd306a7b9ca None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:01.829132 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa9c17-f9de-43fc-a953-2bd306a7b9ca None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:01.829410 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa9c17-f9de-43fc-a953-2bd306a7b9ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:01.837174 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:01.841909 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa9c17-f9de-43fc-a953-2bd306a7b9ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-daaa9c17-f9de-43fc-a953-2bd306a7b9ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:20:01.850073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:20:01.851950 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:20:01.854032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:20:01.854032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:20:01.854032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:20:01.854032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:20:01.854032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:20:01.854032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:20:01.854032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:20:01.854032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:20:01.854032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:20:01.854032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:20:01.854032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:20:01.854032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:20:01.854032 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 572/1149] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:20:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:20:01.860822 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dec44c-6c39-4dad-a14f-3112c3be93d4 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:01.972610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dec44c-6c39-4dad-a14f-3112c3be93d4 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:02.010078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cd935-a894-4c5c-af68-ebbf376e27d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:02.010798 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cd935-a894-4c5c-af68-ebbf376e27d3 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:02.011077 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cd935-a894-4c5c-af68-ebbf376e27d3 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:02.011358 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cd935-a894-4c5c-af68-ebbf376e27d3 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:02.012119 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cd935-a894-4c5c-af68-ebbf376e27d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:20:02.012424 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cd935-a894-4c5c-af68-ebbf376e27d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:02.013836 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cd935-a894-4c5c-af68-ebbf376e27d3 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:02.022650 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 578/1150] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:20:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:20:02.030041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa9c17-f9de-43fc-a953-2bd306a7b9ca None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:02.142783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa9c17-f9de-43fc-a953-2bd306a7b9ca None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:02.182293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcce81-a24f-4062-81bb-3009b10f756e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:02.183190 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcce81-a24f-4062-81bb-3009b10f756e None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:02.183471 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcce81-a24f-4062-81bb-3009b10f756e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:02.183742 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcce81-a24f-4062-81bb-3009b10f756e None admin] PATH_INFO: `/v3/users/f090ff54b64e462ca333e4c9c532ba36` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:02.185163 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fcce81-a24f-4062-81bb-3009b10f756e None admin] RBAC: Authorizing `identity:get_user(user_id=f090ff54b64e462ca333e4c9c532ba36)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:02.186673 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fcce81-a24f-4062-81bb-3009b10f756e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:02.188319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 573/1151] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:20:02 2022] GET /identity/v3/users/f090ff54b64e462ca333e4c9c532ba36 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:02.194495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cd935-a894-4c5c-af68-ebbf376e27d3 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:02.255239 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:02.315080 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cd935-a894-4c5c-af68-ebbf376e27d3 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:02.351194 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41657f3d-e4c3-4bc9-a2d3-96f6c6d71c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:02.352062 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41657f3d-e4c3-4bc9-a2d3-96f6c6d71c73 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:02.352342 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41657f3d-e4c3-4bc9-a2d3-96f6c6d71c73 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:02.352615 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41657f3d-e4c3-4bc9-a2d3-96f6c6d71c73 None admin] PATH_INFO: `/v3/projects/e05db95889c24bda9c5c13dd8d4992bd` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:02.353928 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41657f3d-e4c3-4bc9-a2d3-96f6c6d71c73 None admin] RBAC: Authorizing `identity:get_project(project_id=e05db95889c24bda9c5c13dd8d4992bd)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:02.355396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41657f3d-e4c3-4bc9-a2d3-96f6c6d71c73 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:02.357109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 579/1152] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:20:02 2022] GET /identity/v3/projects/e05db95889c24bda9c5c13dd8d4992bd => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:02.366946 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcce81-a24f-4062-81bb-3009b10f756e None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:02.438572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:02.479527 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcce81-a24f-4062-81bb-3009b10f756e None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:02.515924 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba0f2d-9d29-48fb-a3df-14ca0ef644b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:02.516817 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba0f2d-9d29-48fb-a3df-14ca0ef644b4 None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:02.517074 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba0f2d-9d29-48fb-a3df-14ca0ef644b4 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:02.517359 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba0f2d-9d29-48fb-a3df-14ca0ef644b4 None admin] PATH_INFO: `/v3/projects/e05db95889c24bda9c5c13dd8d4992bd/users/f090ff54b64e462ca333e4c9c532ba36/roles/b239f941cb724331bf28dc489b70ad1d` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:02.519444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba0f2d-9d29-48fb-a3df-14ca0ef644b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e05db95889c24bda9c5c13dd8d4992bd, user_id=f090ff54b64e462ca333e4c9c532ba36, role_id=b239f941cb724331bf28dc489b70ad1d)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:02.520902 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba0f2d-9d29-48fb-a3df-14ca0ef644b4 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:02.539502 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 574/1153] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:20:02 2022] PUT /identity/v3/projects/e05db95889c24bda9c5c13dd8d4992bd/users/f090ff54b64e462ca333e4c9c532ba36/roles/b239f941cb724331bf28dc489b70ad1d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:20:04.725358 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0ad38-55b7-4679-86d4-2f4fd9836b6e None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:04.726227 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0ad38-55b7-4679-86d4-2f4fd9836b6e None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:04.726227 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0ad38-55b7-4679-86d4-2f4fd9836b6e None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:04.728386 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 580/1154] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:20:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:20:04.737149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad36e7-19ad-4f66-929f-fc0bc36a2260 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:04.737585 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad36e7-19ad-4f66-929f-fc0bc36a2260 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:04.737802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad36e7-19ad-4f66-929f-fc0bc36a2260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:04.757918 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-d9ad36e7-19ad-4f66-929f-fc0bc36a2260 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:04.759168 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-d9ad36e7-19ad-4f66-929f-fc0bc36a2260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:04.836793 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 575/1155] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:20:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:20:04.872665 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0ad38-55b7-4679-86d4-2f4fd9836b6e None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:04.989426 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0ad38-55b7-4679-86d4-2f4fd9836b6e None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:05.002775 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:05.025846 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d145a75-ead6-4413-b961-d14eae0d3cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:05.026765 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d145a75-ead6-4413-b961-d14eae0d3cd2 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:05.027040 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d145a75-ead6-4413-b961-d14eae0d3cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:05.027320 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d145a75-ead6-4413-b961-d14eae0d3cd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:05.039618 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d145a75-ead6-4413-b961-d14eae0d3cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-7d145a75-ead6-4413-b961-d14eae0d3cd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:20:05.047361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:20:05.048613 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:20:05.050076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:20:05.050076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:20:05.050076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:20:05.050076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:20:05.050076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:20:05.050076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:20:05.050076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:20:05.050076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:20:05.050076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:20:05.050076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:20:05.050076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:20:05.050076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:20:05.050076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 581/1156] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:20:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:20:05.057947 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad36e7-19ad-4f66-929f-fc0bc36a2260 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:05.135188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:05.187069 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad36e7-19ad-4f66-929f-fc0bc36a2260 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:05.229424 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e888b-b5bb-4053-8c72-a4f9ae9f6501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:05.230088 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e888b-b5bb-4053-8c72-a4f9ae9f6501 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:05.230395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e888b-b5bb-4053-8c72-a4f9ae9f6501 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:05.230668 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e888b-b5bb-4053-8c72-a4f9ae9f6501 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:05.231414 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e888b-b5bb-4053-8c72-a4f9ae9f6501 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:20:05.231721 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e888b-b5bb-4053-8c72-a4f9ae9f6501 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:05.233110 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e888b-b5bb-4053-8c72-a4f9ae9f6501 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:05.241573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 576/1157] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:20:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:05.248153 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d145a75-ead6-4413-b961-d14eae0d3cd2 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:05.362811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d145a75-ead6-4413-b961-d14eae0d3cd2 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:05.402194 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf55c2b-c72c-49cc-aea3-b5436a00ef05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:05.403110 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf55c2b-c72c-49cc-aea3-b5436a00ef05 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:05.403391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf55c2b-c72c-49cc-aea3-b5436a00ef05 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:05.403666 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf55c2b-c72c-49cc-aea3-b5436a00ef05 None admin] PATH_INFO: `/v3/users/f090ff54b64e462ca333e4c9c532ba36` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:05.405086 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf55c2b-c72c-49cc-aea3-b5436a00ef05 None admin] RBAC: Authorizing `identity:get_user(user_id=f090ff54b64e462ca333e4c9c532ba36)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:05.406577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf55c2b-c72c-49cc-aea3-b5436a00ef05 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:05.408248 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 582/1158] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:20:05 2022] GET /identity/v3/users/f090ff54b64e462ca333e4c9c532ba36 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:05.414210 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e888b-b5bb-4053-8c72-a4f9ae9f6501 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:05.532597 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e888b-b5bb-4053-8c72-a4f9ae9f6501 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:05.550185 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:05.571774 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edc8fd-6e68-4568-a5ed-a983adeb6745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:05.572427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edc8fd-6e68-4568-a5ed-a983adeb6745 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:05.572715 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edc8fd-6e68-4568-a5ed-a983adeb6745 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:05.572988 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edc8fd-6e68-4568-a5ed-a983adeb6745 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:05.646103 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edc8fd-6e68-4568-a5ed-a983adeb6745 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:05.647707 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edc8fd-6e68-4568-a5ed-a983adeb6745 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:05.750594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 577/1159] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:20:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:20:05.756898 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf55c2b-c72c-49cc-aea3-b5436a00ef05 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:05.803744 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:05.865822 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf55c2b-c72c-49cc-aea3-b5436a00ef05 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:05.924538 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68f98c-c9bf-4637-8d09-58ca7f553303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:05.925404 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68f98c-c9bf-4637-8d09-58ca7f553303 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:05.925686 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68f98c-c9bf-4637-8d09-58ca7f553303 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:05.925962 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68f98c-c9bf-4637-8d09-58ca7f553303 None admin] PATH_INFO: `/v3/projects/e05db95889c24bda9c5c13dd8d4992bd` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:05.927236 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68f98c-c9bf-4637-8d09-58ca7f553303 None admin] RBAC: Authorizing `identity:get_project(project_id=e05db95889c24bda9c5c13dd8d4992bd)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:05.928715 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68f98c-c9bf-4637-8d09-58ca7f553303 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:05.930505 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 583/1160] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:20:05 2022] GET /identity/v3/projects/e05db95889c24bda9c5c13dd8d4992bd => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:20:05.940126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edc8fd-6e68-4568-a5ed-a983adeb6745 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:05.964070 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:06.049127 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edc8fd-6e68-4568-a5ed-a983adeb6745 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:06.086117 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ece05-144f-48c3-b46c-9f2740dabbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:06.086727 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ece05-144f-48c3-b46c-9f2740dabbd9 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:06.087021 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ece05-144f-48c3-b46c-9f2740dabbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:06.087297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ece05-144f-48c3-b46c-9f2740dabbd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:06.088446 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ece05-144f-48c3-b46c-9f2740dabbd9 None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=e05db95889c24bda9c5c13dd8d4992bd, user.id=f090ff54b64e462ca333e4c9c532ba36) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:20:06.088756 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ece05-144f-48c3-b46c-9f2740dabbd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:06.090170 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ece05-144f-48c3-b46c-9f2740dabbd9 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:06.096172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 578/1161] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:20:05 2022] GET /identity/v3/role_assignments?user.id=f090ff54b64e462ca333e4c9c532ba36&scope.project.id=e05db95889c24bda9c5c13dd8d4992bd&role.id=b239f941cb724331bf28dc489b70ad1d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:08.341984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54da58a-8cc1-4d3d-973c-7b4a8d598187 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:08.342644 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54da58a-8cc1-4d3d-973c-7b4a8d598187 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:08.342644 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54da58a-8cc1-4d3d-973c-7b4a8d598187 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:08.343979 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 584/1162] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:20:08.350796 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85bf02-a49e-4999-87fd-ce095491ccac None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:08.351567 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85bf02-a49e-4999-87fd-ce095491ccac None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:08.351567 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85bf02-a49e-4999-87fd-ce095491ccac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:08.375194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-cb85bf02-a49e-4999-87fd-ce095491ccac None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:08.376417 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-cb85bf02-a49e-4999-87fd-ce095491ccac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:08.448368 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 579/1163] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:20:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:20:08.471320 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54da58a-8cc1-4d3d-973c-7b4a8d598187 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:08.577295 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54da58a-8cc1-4d3d-973c-7b4a8d598187 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:08.589015 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:08.621621 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cedcc5-8cbd-4308-a751-ed616b29936f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:08.622520 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cedcc5-8cbd-4308-a751-ed616b29936f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:08.622907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cedcc5-8cbd-4308-a751-ed616b29936f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:08.623273 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cedcc5-8cbd-4308-a751-ed616b29936f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:08.703976 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cedcc5-8cbd-4308-a751-ed616b29936f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:08.705417 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cedcc5-8cbd-4308-a751-ed616b29936f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:08.810455 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 585/1164] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:20:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:20:08.817468 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85bf02-a49e-4999-87fd-ce095491ccac None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:08.833103 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:08.935799 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85bf02-a49e-4999-87fd-ce095491ccac None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:08.976107 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ad378-75de-48f8-9859-8a0ae4cea1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:08.976888 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ad378-75de-48f8-9859-8a0ae4cea1be None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:08.977174 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ad378-75de-48f8-9859-8a0ae4cea1be None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:08.977451 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ad378-75de-48f8-9859-8a0ae4cea1be None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:08.984433 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ad378-75de-48f8-9859-8a0ae4cea1be None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:08.985913 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ad378-75de-48f8-9859-8a0ae4cea1be None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-429ad378-75de-48f8-9859-8a0ae4cea1be None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:20:09.000657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:20:09.002026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:20:09.003126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:20:09.003126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:20:09.003126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:20:09.003126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:20:09.003126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:20:09.003126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:20:09.003126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:20:09.003126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:20:09.003126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:20:09.003126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:20:09.003126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 20:20:09.003126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:20:09.003126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 20:20:09.003126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:20:09.003126 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 580/1165] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:20:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:20:09.009119 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cedcc5-8cbd-4308-a751-ed616b29936f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:09.019769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:09.114345 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cedcc5-8cbd-4308-a751-ed616b29936f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:09.155159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf718bfe-1f99-4750-860c-485646ba7dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:09.156136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf718bfe-1f99-4750-860c-485646ba7dd7 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:09.156564 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf718bfe-1f99-4750-860c-485646ba7dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:09.156973 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf718bfe-1f99-4750-860c-485646ba7dd7 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:09.158116 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf718bfe-1f99-4750-860c-485646ba7dd7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:20:09.158556 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf718bfe-1f99-4750-860c-485646ba7dd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:09.160775 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf718bfe-1f99-4750-860c-485646ba7dd7 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:09.172933 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 586/1166] 10.222.0.84 () {56 vars in 1115 bytes} [Thu May 19 20:20:09 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:11.437978 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd4b64-a486-47e8-b269-704377edcd6f None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:11.438753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd4b64-a486-47e8-b269-704377edcd6f None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:11.438919 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd4b64-a486-47e8-b269-704377edcd6f None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:11.441028 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 581/1167] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:20:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:20:11.446629 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efcf43-832a-4441-9595-b75ff50d3cd3 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:11.447086 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efcf43-832a-4441-9595-b75ff50d3cd3 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:11.447196 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efcf43-832a-4441-9595-b75ff50d3cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:11.466456 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-b7efcf43-832a-4441-9595-b75ff50d3cd3 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:11.467654 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-b7efcf43-832a-4441-9595-b75ff50d3cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:11.544150 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 587/1168] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:20:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:20:11.570540 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd4b64-a486-47e8-b269-704377edcd6f None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:11.667063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:11.700789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd4b64-a486-47e8-b269-704377edcd6f None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:11.737606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd4c87-1187-41a3-be65-0ce8891fd575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:11.738298 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd4c87-1187-41a3-be65-0ce8891fd575 None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:11.738584 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd4c87-1187-41a3-be65-0ce8891fd575 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:11.738863 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd4c87-1187-41a3-be65-0ce8891fd575 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:11.739613 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd4c87-1187-41a3-be65-0ce8891fd575 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:11.741038 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd4c87-1187-41a3-be65-0ce8891fd575 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:11.761466 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 582/1169] 10.222.0.84 () {60 vars in 1138 bytes} [Thu May 19 20:20:11 2022] POST /identity/v3/domains => generated 267 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:20:14.067889 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca804a-3d57-475f-9171-65ffce24051c None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:14.068806 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca804a-3d57-475f-9171-65ffce24051c None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:14.068806 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca804a-3d57-475f-9171-65ffce24051c None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:14.070623 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 588/1170] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:20:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:20:14.077261 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280e11c-151c-455e-937a-61905a073e56 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:14.077731 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280e11c-151c-455e-937a-61905a073e56 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:14.077836 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280e11c-151c-455e-937a-61905a073e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:14.103846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-9280e11c-151c-455e-937a-61905a073e56 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:14.105047 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-9280e11c-151c-455e-937a-61905a073e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:14.178134 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 583/1171] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:20:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:20:14.202107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca804a-3d57-475f-9171-65ffce24051c None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:14.205534 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:14.327189 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca804a-3d57-475f-9171-65ffce24051c None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:14.363460 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81aa0b5-81cb-42bd-bac0-f3f3358ad884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:14.364327 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81aa0b5-81cb-42bd-bac0-f3f3358ad884 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:14.364601 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81aa0b5-81cb-42bd-bac0-f3f3358ad884 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:14.364882 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81aa0b5-81cb-42bd-bac0-f3f3358ad884 None admin] PATH_INFO: `/v3/domains/e8a3dc1b2ad04947a05169a6e55459cf` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:14.376193 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81aa0b5-81cb-42bd-bac0-f3f3358ad884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e8a3dc1b2ad04947a05169a6e55459cf)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:14.377628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81aa0b5-81cb-42bd-bac0-f3f3358ad884 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:14.379303 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 589/1172] 10.222.0.84 () {56 vars in 1183 bytes} [Thu May 19 20:20:14 2022] GET /identity/v3/domains/e8a3dc1b2ad04947a05169a6e55459cf => generated 267 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:14.385526 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280e11c-151c-455e-937a-61905a073e56 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:14.450862 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:14.491831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280e11c-151c-455e-937a-61905a073e56 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:14.537930 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31da75-9b2c-4c64-84fa-b2c3ff997fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:14.538952 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31da75-9b2c-4c64-84fa-b2c3ff997fab None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:14.539378 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31da75-9b2c-4c64-84fa-b2c3ff997fab None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:14.539798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31da75-9b2c-4c64-84fa-b2c3ff997fab None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:14.540902 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31da75-9b2c-4c64-84fa-b2c3ff997fab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:14.543242 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31da75-9b2c-4c64-84fa-b2c3ff997fab None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:14.580347 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 584/1173] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:20:14 2022] POST /identity/v3/projects => generated 361 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:20:16.816040 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6be2e-1b7e-4c4d-a077-a24b479b6d69 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:16.816627 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6be2e-1b7e-4c4d-a077-a24b479b6d69 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:16.816627 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6be2e-1b7e-4c4d-a077-a24b479b6d69 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:16.817992 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 590/1174] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:20:16.824586 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13727f-7a78-48d1-bc21-0004c01a8ad8 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:16.825368 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13727f-7a78-48d1-bc21-0004c01a8ad8 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:16.825536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13727f-7a78-48d1-bc21-0004c01a8ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:16.848653 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-4a13727f-7a78-48d1-bc21-0004c01a8ad8 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:16.849866 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-4a13727f-7a78-48d1-bc21-0004c01a8ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:16.930895 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 585/1175] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:20:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:20:16.969114 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6be2e-1b7e-4c4d-a077-a24b479b6d69 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:17.041610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:17.095847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6be2e-1b7e-4c4d-a077-a24b479b6d69 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:17.132894 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d5700-9bab-4d66-8d78-8eb28ee68ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:17.133771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d5700-9bab-4d66-8d78-8eb28ee68ae0 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:17.134058 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d5700-9bab-4d66-8d78-8eb28ee68ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:17.134335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d5700-9bab-4d66-8d78-8eb28ee68ae0 None admin] PATH_INFO: `/v3/domains/e8a3dc1b2ad04947a05169a6e55459cf` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:17.135678 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d5700-9bab-4d66-8d78-8eb28ee68ae0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e8a3dc1b2ad04947a05169a6e55459cf)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:17.137122 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d5700-9bab-4d66-8d78-8eb28ee68ae0 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:17.138804 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 591/1176] 10.222.0.84 () {56 vars in 1183 bytes} [Thu May 19 20:20:16 2022] GET /identity/v3/domains/e8a3dc1b2ad04947a05169a6e55459cf => generated 267 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:17.148054 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13727f-7a78-48d1-bc21-0004c01a8ad8 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:17.203964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:17.267493 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13727f-7a78-48d1-bc21-0004c01a8ad8 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:17.311144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d1f01-e383-41da-8b6f-fca48f87f657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:17.312106 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d1f01-e383-41da-8b6f-fca48f87f657 None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:17.312486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d1f01-e383-41da-8b6f-fca48f87f657 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:17.312861 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d1f01-e383-41da-8b6f-fca48f87f657 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:17.313915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d1f01-e383-41da-8b6f-fca48f87f657 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:17.316237 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d1f01-e383-41da-8b6f-fca48f87f657 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:17.340266 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 586/1177] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:20:17 2022] POST /identity/v3/users => generated 311 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:20:19.641088 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cbf53-8d49-44ee-8540-0f3c687dc913 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:19.641698 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cbf53-8d49-44ee-8540-0f3c687dc913 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:19.641698 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cbf53-8d49-44ee-8540-0f3c687dc913 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:19.643093 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 592/1178] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:20:19.650252 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f0042-c5d5-47c4-86e3-7c016b767e37 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:19.650990 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f0042-c5d5-47c4-86e3-7c016b767e37 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:19.651152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f0042-c5d5-47c4-86e3-7c016b767e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:19.679790 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-d94f0042-c5d5-47c4-86e3-7c016b767e37 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:19.681050 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-d94f0042-c5d5-47c4-86e3-7c016b767e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:19.753847 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 587/1179] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:20:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:20:19.786134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430cbf53-8d49-44ee-8540-0f3c687dc913 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:19.905972 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430cbf53-8d49-44ee-8540-0f3c687dc913 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:19.938401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:19.946775 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f6d72-fe69-4578-87dc-a5c8bd5f726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:19.947972 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f6d72-fe69-4578-87dc-a5c8bd5f726e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:19.948299 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f6d72-fe69-4578-87dc-a5c8bd5f726e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:19.948690 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f6d72-fe69-4578-87dc-a5c8bd5f726e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:19.960476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f6d72-fe69-4578-87dc-a5c8bd5f726e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-666f6d72-fe69-4578-87dc-a5c8bd5f726e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:20:19.968591 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:20:19.969818 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:20:19.970930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:20:19.970930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:20:19.970930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:20:19.970930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:20:19.970930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:20:19.970930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:20:19.970930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:20:19.970930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:20:19.970930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:20:19.970930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:20:19.970930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:20:19.970930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:20:19.970930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 593/1180] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:20:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:20:19.976909 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f0042-c5d5-47c4-86e3-7c016b767e37 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:20.019620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:20.100904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f0042-c5d5-47c4-86e3-7c016b767e37 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:20.136155 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f99e91-4186-42ba-a7fd-21c910382e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:20.136820 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f99e91-4186-42ba-a7fd-21c910382e0b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:20.137097 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f99e91-4186-42ba-a7fd-21c910382e0b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:20.137372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f99e91-4186-42ba-a7fd-21c910382e0b None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:20.138231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f99e91-4186-42ba-a7fd-21c910382e0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:20:20.138565 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f99e91-4186-42ba-a7fd-21c910382e0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:20.139969 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f99e91-4186-42ba-a7fd-21c910382e0b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:20.148521 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 588/1181] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:20:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:20.155457 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f6d72-fe69-4578-87dc-a5c8bd5f726e None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:20.276293 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f6d72-fe69-4578-87dc-a5c8bd5f726e None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:20.312571 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a5031-f0a4-45d5-a7a5-063579acf639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:20.313442 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a5031-f0a4-45d5-a7a5-063579acf639 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:20.313740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a5031-f0a4-45d5-a7a5-063579acf639 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:20.313995 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a5031-f0a4-45d5-a7a5-063579acf639 None admin] PATH_INFO: `/v3/users/c16e79c2512142c8bbdb422f45644379` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:20.327632 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a5031-f0a4-45d5-a7a5-063579acf639 None admin] RBAC: Authorizing `identity:get_user(user_id=c16e79c2512142c8bbdb422f45644379)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:20.329093 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a5031-f0a4-45d5-a7a5-063579acf639 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:20.330821 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 594/1182] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:20:20 2022] GET /identity/v3/users/c16e79c2512142c8bbdb422f45644379 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:20.340668 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f99e91-4186-42ba-a7fd-21c910382e0b None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:20.452948 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:20.464079 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f99e91-4186-42ba-a7fd-21c910382e0b None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:20.499749 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36829bf3-8a75-4435-b1f5-e3bef19da9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:20.500431 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36829bf3-8a75-4435-b1f5-e3bef19da9b8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:20.500659 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36829bf3-8a75-4435-b1f5-e3bef19da9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:20.500971 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36829bf3-8a75-4435-b1f5-e3bef19da9b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:20.577883 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36829bf3-8a75-4435-b1f5-e3bef19da9b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:20.579429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36829bf3-8a75-4435-b1f5-e3bef19da9b8 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:20.674497 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:20.686749 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 589/1183] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:20:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:20:20.693069 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a5031-f0a4-45d5-a7a5-063579acf639 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:20.752229 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:20.808595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a5031-f0a4-45d5-a7a5-063579acf639 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:20.844649 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79180ce-0477-4932-9761-13d0d47b7ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:20.845440 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79180ce-0477-4932-9761-13d0d47b7ac3 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:20.845709 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79180ce-0477-4932-9761-13d0d47b7ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:20.845974 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79180ce-0477-4932-9761-13d0d47b7ac3 None admin] PATH_INFO: `/v3/projects/9fb394fabac14675b4d9f217c7023102` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:20.847234 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79180ce-0477-4932-9761-13d0d47b7ac3 None admin] RBAC: Authorizing `identity:get_project(project_id=9fb394fabac14675b4d9f217c7023102)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:20.848664 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79180ce-0477-4932-9761-13d0d47b7ac3 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:20.850481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 595/1184] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:20:20 2022] GET /identity/v3/projects/9fb394fabac14675b4d9f217c7023102 => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:20.860172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36829bf3-8a75-4435-b1f5-e3bef19da9b8 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:20.983708 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36829bf3-8a75-4435-b1f5-e3bef19da9b8 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:21.019041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf23d89-9dd8-4ea8-b387-d7635ae2af94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:21.019687 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf23d89-9dd8-4ea8-b387-d7635ae2af94 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:21.019966 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf23d89-9dd8-4ea8-b387-d7635ae2af94 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:21.020239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf23d89-9dd8-4ea8-b387-d7635ae2af94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:21.021071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf23d89-9dd8-4ea8-b387-d7635ae2af94 None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=9fb394fabac14675b4d9f217c7023102, user.id=c16e79c2512142c8bbdb422f45644379) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:20:21.021379 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf23d89-9dd8-4ea8-b387-d7635ae2af94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:21.022926 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf23d89-9dd8-4ea8-b387-d7635ae2af94 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:21.028539 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 590/1185] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:20:20 2022] GET /identity/v3/role_assignments?user.id=c16e79c2512142c8bbdb422f45644379&scope.project.id=9fb394fabac14675b4d9f217c7023102&role.id=b239f941cb724331bf28dc489b70ad1d => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:23.193003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1354d-0c37-4d62-b586-b95bb28ee5f9 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:23.194094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1354d-0c37-4d62-b586-b95bb28ee5f9 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:23.194094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1354d-0c37-4d62-b586-b95bb28ee5f9 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:23.196261 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 596/1186] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:20:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:20:23.205140 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7c4c6-74c8-4f19-a919-aef23e94b030 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:23.205571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7c4c6-74c8-4f19-a919-aef23e94b030 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:23.205818 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7c4c6-74c8-4f19-a919-aef23e94b030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:23.232026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-79b7c4c6-74c8-4f19-a919-aef23e94b030 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:23.233216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-79b7c4c6-74c8-4f19-a919-aef23e94b030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:23.288450 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:23.317680 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 591/1187] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:20:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:20:23.341397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1354d-0c37-4d62-b586-b95bb28ee5f9 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:23.457920 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1354d-0c37-4d62-b586-b95bb28ee5f9 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:23.477655 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:23.498697 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eaf1d8-257f-4590-bfed-199d998626d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:23.499493 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eaf1d8-257f-4590-bfed-199d998626d4 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:23.499778 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eaf1d8-257f-4590-bfed-199d998626d4 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:23.500057 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eaf1d8-257f-4590-bfed-199d998626d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:23.511052 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eaf1d8-257f-4590-bfed-199d998626d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-d5eaf1d8-257f-4590-bfed-199d998626d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:20:23.518756 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:20:23.519959 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:20:23.521159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:20:23.521159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:20:23.521159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:20:23.521159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:20:23.521159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:20:23.521159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:20:23.521159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:20:23.521159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:20:23.521159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:20:23.521159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:20:23.521159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:20:23.521159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:20:23.522336 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 597/1188] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:20:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:20:23.526983 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7c4c6-74c8-4f19-a919-aef23e94b030 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:23.635356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7c4c6-74c8-4f19-a919-aef23e94b030 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:23.671179 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c01a91-8e0f-4f6e-9551-e6a004236192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:23.671836 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c01a91-8e0f-4f6e-9551-e6a004236192 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:23.672110 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c01a91-8e0f-4f6e-9551-e6a004236192 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:23.672384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c01a91-8e0f-4f6e-9551-e6a004236192 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:23.673164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c01a91-8e0f-4f6e-9551-e6a004236192 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:20:23.673467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c01a91-8e0f-4f6e-9551-e6a004236192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:23.674883 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c01a91-8e0f-4f6e-9551-e6a004236192 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:23.683347 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 592/1189] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:20:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:23.690210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eaf1d8-257f-4590-bfed-199d998626d4 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:23.806550 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eaf1d8-257f-4590-bfed-199d998626d4 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:23.843318 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0878986-47db-4a96-ba29-ef8956aaef1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:23.844196 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0878986-47db-4a96-ba29-ef8956aaef1c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:23.844476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0878986-47db-4a96-ba29-ef8956aaef1c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:23.844751 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0878986-47db-4a96-ba29-ef8956aaef1c None admin] PATH_INFO: `/v3/users/c16e79c2512142c8bbdb422f45644379` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:23.846008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0878986-47db-4a96-ba29-ef8956aaef1c None admin] RBAC: Authorizing `identity:get_user(user_id=c16e79c2512142c8bbdb422f45644379)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:23.847490 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0878986-47db-4a96-ba29-ef8956aaef1c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:23.849133 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 598/1190] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:20:23 2022] GET /identity/v3/users/c16e79c2512142c8bbdb422f45644379 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:23.855973 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c01a91-8e0f-4f6e-9551-e6a004236192 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:23.920619 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:23.975156 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c01a91-8e0f-4f6e-9551-e6a004236192 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:24.010805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733dbd4-b8d2-43ca-9bb6-e228de20471c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:24.011665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733dbd4-b8d2-43ca-9bb6-e228de20471c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:24.011940 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733dbd4-b8d2-43ca-9bb6-e228de20471c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:24.012210 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733dbd4-b8d2-43ca-9bb6-e228de20471c None admin] PATH_INFO: `/v3/projects/9fb394fabac14675b4d9f217c7023102` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:24.013534 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733dbd4-b8d2-43ca-9bb6-e228de20471c None admin] RBAC: Authorizing `identity:get_project(project_id=9fb394fabac14675b4d9f217c7023102)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:24.015027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733dbd4-b8d2-43ca-9bb6-e228de20471c None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:24.016712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 593/1191] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:20:23 2022] GET /identity/v3/projects/9fb394fabac14675b4d9f217c7023102 => generated 361 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:24.027354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0878986-47db-4a96-ba29-ef8956aaef1c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:24.083394 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:24.152127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0878986-47db-4a96-ba29-ef8956aaef1c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:24.188106 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fddb6e9-31bc-4dd1-b960-3a126f01405b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:24.188989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddb6e9-31bc-4dd1-b960-3a126f01405b None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:24.189268 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddb6e9-31bc-4dd1-b960-3a126f01405b None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:24.189538 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddb6e9-31bc-4dd1-b960-3a126f01405b None admin] PATH_INFO: `/v3/projects/9fb394fabac14675b4d9f217c7023102/users/c16e79c2512142c8bbdb422f45644379/roles/b239f941cb724331bf28dc489b70ad1d` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:24.191700 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fddb6e9-31bc-4dd1-b960-3a126f01405b None admin] RBAC: Authorizing `identity:create_grant(project_id=9fb394fabac14675b4d9f217c7023102, user_id=c16e79c2512142c8bbdb422f45644379, role_id=b239f941cb724331bf28dc489b70ad1d)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:24.193137 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fddb6e9-31bc-4dd1-b960-3a126f01405b None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:24.211450 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 599/1192] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:20:24 2022] PUT /identity/v3/projects/9fb394fabac14675b4d9f217c7023102/users/c16e79c2512142c8bbdb422f45644379/roles/b239f941cb724331bf28dc489b70ad1d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:20:26.372479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083a755-a594-4ec4-889d-03c777c6ce03 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:26.373404 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083a755-a594-4ec4-889d-03c777c6ce03 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:26.373404 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083a755-a594-4ec4-889d-03c777c6ce03 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:26.375696 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 594/1193] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:20:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:20:26.381561 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483de3bc-dd9f-4266-910d-e2eb034127e0 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:26.382017 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483de3bc-dd9f-4266-910d-e2eb034127e0 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:26.382290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483de3bc-dd9f-4266-910d-e2eb034127e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:26.400915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-483de3bc-dd9f-4266-910d-e2eb034127e0 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:26.402090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-483de3bc-dd9f-4266-910d-e2eb034127e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:26.476189 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 600/1194] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:20:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:20:26.500816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083a755-a594-4ec4-889d-03c777c6ce03 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:26.622900 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083a755-a594-4ec4-889d-03c777c6ce03 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:26.639439 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:26.659383 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7a1de-cdd2-414e-8d94-c6e0e762f80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:26.660178 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7a1de-cdd2-414e-8d94-c6e0e762f80d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:26.660456 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7a1de-cdd2-414e-8d94-c6e0e762f80d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:26.660728 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7a1de-cdd2-414e-8d94-c6e0e762f80d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:26.671587 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7a1de-cdd2-414e-8d94-c6e0e762f80d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-85f7a1de-cdd2-414e-8d94-c6e0e762f80d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:20:26.679239 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:20:26.680472 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:20:26.682094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:20:26.682094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:20:26.682094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:20:26.682094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:20:26.682094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:20:26.682094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:20:26.682094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:20:26.682094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:20:26.682094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:20:26.682094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:20:26.682094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:20:26.682094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:20:26.682094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 595/1195] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:20:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:20:26.690017 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483de3bc-dd9f-4266-910d-e2eb034127e0 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:26.732597 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:26.810576 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483de3bc-dd9f-4266-910d-e2eb034127e0 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:26.853767 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11088d61-e8c4-45cb-8b0a-924eff141288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:26.854465 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11088d61-e8c4-45cb-8b0a-924eff141288 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:26.854748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11088d61-e8c4-45cb-8b0a-924eff141288 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:26.855021 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11088d61-e8c4-45cb-8b0a-924eff141288 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:26.855768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11088d61-e8c4-45cb-8b0a-924eff141288 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:20:26.856077 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11088d61-e8c4-45cb-8b0a-924eff141288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:26.857455 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11088d61-e8c4-45cb-8b0a-924eff141288 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:26.865991 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 601/1196] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:20:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:26.872699 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7a1de-cdd2-414e-8d94-c6e0e762f80d None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:26.987817 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7a1de-cdd2-414e-8d94-c6e0e762f80d None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:27.027961 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59452a-c283-4182-8c7e-61279998ab9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:27.028833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59452a-c283-4182-8c7e-61279998ab9c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:27.029111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59452a-c283-4182-8c7e-61279998ab9c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:27.029357 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59452a-c283-4182-8c7e-61279998ab9c None admin] PATH_INFO: `/v3/users/c16e79c2512142c8bbdb422f45644379` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:27.030640 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59452a-c283-4182-8c7e-61279998ab9c None admin] RBAC: Authorizing `identity:get_user(user_id=c16e79c2512142c8bbdb422f45644379)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:27.032088 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59452a-c283-4182-8c7e-61279998ab9c None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:27.033761 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 596/1197] 10.222.0.84 () {56 vars in 1177 bytes} [Thu May 19 20:20:26 2022] GET /identity/v3/users/c16e79c2512142c8bbdb422f45644379 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:27.041237 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11088d61-e8c4-45cb-8b0a-924eff141288 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:27.146223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:27.153039 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11088d61-e8c4-45cb-8b0a-924eff141288 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:27.191786 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d6b0d-deea-4722-bd68-54bf64ca6a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:27.192442 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d6b0d-deea-4722-bd68-54bf64ca6a6e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:27.192722 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d6b0d-deea-4722-bd68-54bf64ca6a6e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:27.193002 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d6b0d-deea-4722-bd68-54bf64ca6a6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:27.270431 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d6b0d-deea-4722-bd68-54bf64ca6a6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:27.271894 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d6b0d-deea-4722-bd68-54bf64ca6a6e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:27.375833 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:27.383940 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 602/1198] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:20:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:20:27.392628 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59452a-c283-4182-8c7e-61279998ab9c None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:27.444424 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:27.509302 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59452a-c283-4182-8c7e-61279998ab9c None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:27.550297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bca159-b947-46a2-b38c-fb247c5aeb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:27.551180 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bca159-b947-46a2-b38c-fb247c5aeb89 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:27.551456 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bca159-b947-46a2-b38c-fb247c5aeb89 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:27.551725 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bca159-b947-46a2-b38c-fb247c5aeb89 None admin] PATH_INFO: `/v3/projects/9fb394fabac14675b4d9f217c7023102` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:27.553104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bca159-b947-46a2-b38c-fb247c5aeb89 None admin] RBAC: Authorizing `identity:get_project(project_id=9fb394fabac14675b4d9f217c7023102)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:27.554584 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bca159-b947-46a2-b38c-fb247c5aeb89 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:27.556284 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 597/1199] 10.222.0.84 () {56 vars in 1186 bytes} [Thu May 19 20:20:27 2022] GET /identity/v3/projects/9fb394fabac14675b4d9f217c7023102 => generated 361 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:27.562856 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d6b0d-deea-4722-bd68-54bf64ca6a6e None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:27.668732 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d6b0d-deea-4722-bd68-54bf64ca6a6e None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:27.705603 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31af39-45fe-4335-b480-0cc6b18ac4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:27.706294 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31af39-45fe-4335-b480-0cc6b18ac4ef None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:27.706585 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31af39-45fe-4335-b480-0cc6b18ac4ef None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:27.706862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31af39-45fe-4335-b480-0cc6b18ac4ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:27.707710 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31af39-45fe-4335-b480-0cc6b18ac4ef None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=9fb394fabac14675b4d9f217c7023102, user.id=c16e79c2512142c8bbdb422f45644379) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:20:27.708016 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31af39-45fe-4335-b480-0cc6b18ac4ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:27.709470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31af39-45fe-4335-b480-0cc6b18ac4ef None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:27.716603 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 603/1200] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:20:27 2022] GET /identity/v3/role_assignments?user.id=c16e79c2512142c8bbdb422f45644379&scope.project.id=9fb394fabac14675b4d9f217c7023102&role.id=b239f941cb724331bf28dc489b70ad1d => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:20:37.544523 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=44 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:37.713520 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=70 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:26.849448 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a1c5e-bd5a-42dc-95f6-bbf444558ce8 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:26.850082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a1c5e-bd5a-42dc-95f6-bbf444558ce8 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:26.850082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a1c5e-bd5a-42dc-95f6-bbf444558ce8 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:26.851588 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 598/1201] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:21:26.857270 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8fd3e-30b1-468d-8365-c71263583974 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:26.857718 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8fd3e-30b1-468d-8365-c71263583974 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:26.857929 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8fd3e-30b1-468d-8365-c71263583974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:26.874799 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-86b8fd3e-30b1-468d-8365-c71263583974 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:21:26.875989 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-86b8fd3e-30b1-468d-8365-c71263583974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:21:26.955457 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 604/1202] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:21:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:21:26.982457 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a1c5e-bd5a-42dc-95f6-bbf444558ce8 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:27.110303 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a1c5e-bd5a-42dc-95f6-bbf444558ce8 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:27.146685 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43bfa1-1494-44d2-8358-73d5120d221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:27.147494 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43bfa1-1494-44d2-8358-73d5120d221c None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:27.147781 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43bfa1-1494-44d2-8358-73d5120d221c None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:27.148059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43bfa1-1494-44d2-8358-73d5120d221c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:27.158349 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43bfa1-1494-44d2-8358-73d5120d221c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:27.159829 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43bfa1-1494-44d2-8358-73d5120d221c None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:27.178824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-ec43bfa1-1494-44d2-8358-73d5120d221c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:27.178824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:27.178824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:27.178824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:27.178824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:27.178824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:27.178824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:27.178824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:27.178824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:27.178824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:27.178824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:27.178824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:27.178824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:21:27.178824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:21:27.178824 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:27.180063 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:27.181429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:21:27.181429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:21:27.181429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:27.181429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:27.181429 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 599/1203] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:27.188218 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8fd3e-30b1-468d-8365-c71263583974 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:27.317647 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8fd3e-30b1-468d-8365-c71263583974 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:27.354545 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:27.362351 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e795a10-2855-46ff-b447-72278930ace7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:27.363040 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e795a10-2855-46ff-b447-72278930ace7 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:27.363347 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e795a10-2855-46ff-b447-72278930ace7 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:27.363755 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e795a10-2855-46ff-b447-72278930ace7 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:27.364745 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e795a10-2855-46ff-b447-72278930ace7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:27.365043 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e795a10-2855-46ff-b447-72278930ace7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:27.366565 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e795a10-2855-46ff-b447-72278930ace7 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:27.380174 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 605/1204] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:21:27.386791 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43bfa1-1494-44d2-8358-73d5120d221c None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:27.455247 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:27.521706 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43bfa1-1494-44d2-8358-73d5120d221c None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:27.569194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b37c9-aae1-49e3-9b13-816428031f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:27.569891 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b37c9-aae1-49e3-9b13-816428031f00 None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:27.570174 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b37c9-aae1-49e3-9b13-816428031f00 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:27.570487 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b37c9-aae1-49e3-9b13-816428031f00 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:27.571216 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b37c9-aae1-49e3-9b13-816428031f00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:27.572674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b37c9-aae1-49e3-9b13-816428031f00 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:27.610293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 600/1205] 10.222.0.84 () {60 vars in 1132 bytes} [Thu May 19 20:21:27 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:21:29.946222 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9729b4-fc54-4459-b274-40035a92e11c None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:29.947020 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9729b4-fc54-4459-b274-40035a92e11c None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:29.947020 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9729b4-fc54-4459-b274-40035a92e11c None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:29.948342 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 606/1206] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:21:29.954077 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea15285-b153-4147-8446-02746bd3da8a None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:29.954630 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea15285-b153-4147-8446-02746bd3da8a None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:29.954780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea15285-b153-4147-8446-02746bd3da8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:29.973485 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-2ea15285-b153-4147-8446-02746bd3da8a None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:21:29.974709 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-2ea15285-b153-4147-8446-02746bd3da8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:21:30.052804 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 601/1207] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:21:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:21:30.080506 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9729b4-fc54-4459-b274-40035a92e11c None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:30.231490 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9729b4-fc54-4459-b274-40035a92e11c None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:30.274540 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ba694-4575-4eaa-813f-e166682df8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:30.275460 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ba694-4575-4eaa-813f-e166682df8e7 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:30.275748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ba694-4575-4eaa-813f-e166682df8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:30.276026 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ba694-4575-4eaa-813f-e166682df8e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:30.293168 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ba694-4575-4eaa-813f-e166682df8e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-d19ba694-4575-4eaa-813f-e166682df8e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:21:30.304624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:30.305879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:30.307356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:30.307356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:30.307356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:30.307356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:30.307356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:21:30.307356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:21:30.307356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:21:30.307356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:21:30.307356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:21:30.307356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:21:30.307356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:21:30.307356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:30.307356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 607/1208] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:21:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:30.312936 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea15285-b153-4147-8446-02746bd3da8a None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:30.376116 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:30.467021 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea15285-b153-4147-8446-02746bd3da8a None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:30.511340 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8815a-2fd8-4214-98e7-d31427222564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:30.511896 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8815a-2fd8-4214-98e7-d31427222564 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:30.512211 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8815a-2fd8-4214-98e7-d31427222564 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:30.512492 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8815a-2fd8-4214-98e7-d31427222564 None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:30.513230 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8815a-2fd8-4214-98e7-d31427222564 None admin] RBAC: Adding query filter params (name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:30.513541 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8815a-2fd8-4214-98e7-d31427222564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:30.514980 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8815a-2fd8-4214-98e7-d31427222564 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:30.524013 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 602/1209] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:21:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:30.531278 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ba694-4575-4eaa-813f-e166682df8e7 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:30.598159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:30.676223 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ba694-4575-4eaa-813f-e166682df8e7 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:30.728692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4fd76-e188-4c15-9cc4-cde8fc29cc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:30.729520 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4fd76-e188-4c15-9cc4-cde8fc29cc49 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:30.729806 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4fd76-e188-4c15-9cc4-cde8fc29cc49 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:30.730085 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4fd76-e188-4c15-9cc4-cde8fc29cc49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:30.742789 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4fd76-e188-4c15-9cc4-cde8fc29cc49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:30.744423 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4fd76-e188-4c15-9cc4-cde8fc29cc49 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:30.770447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-80f4fd76-e188-4c15-9cc4-cde8fc29cc49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:30.770447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:30.770447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:30.770447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:30.770447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:30.770447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:30.770447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:30.770447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:30.770447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:30.770447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:30.770447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:30.770447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:30.770447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:21:30.770447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:21:30.770447 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:30.771771 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:30.773221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:21:30.773221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:21:30.773221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:30.773221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:30.773221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 608/1210] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:21:30.778666 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8815a-2fd8-4214-98e7-d31427222564 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:30.896502 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:30.901369 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8815a-2fd8-4214-98e7-d31427222564 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:30.960341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f02af7-6b83-49d4-92c1-c39efdc221bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:30.961694 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f02af7-6b83-49d4-92c1-c39efdc221bf None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:30.962620 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f02af7-6b83-49d4-92c1-c39efdc221bf None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:30.963338 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f02af7-6b83-49d4-92c1-c39efdc221bf None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:30.965147 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f02af7-6b83-49d4-92c1-c39efdc221bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:30.965802 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f02af7-6b83-49d4-92c1-c39efdc221bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:30.968643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f02af7-6b83-49d4-92c1-c39efdc221bf None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:30.985957 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 603/1211] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:21:30.992734 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4fd76-e188-4c15-9cc4-cde8fc29cc49 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:31.115396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4fd76-e188-4c15-9cc4-cde8fc29cc49 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:31.117190 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:31.155941 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a0260-89bf-4c15-a4d4-faf3bd1ca207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:31.156876 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a0260-89bf-4c15-a4d4-faf3bd1ca207 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:31.157234 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a0260-89bf-4c15-a4d4-faf3bd1ca207 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:31.157674 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a0260-89bf-4c15-a4d4-faf3bd1ca207 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:31.166661 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a0260-89bf-4c15-a4d4-faf3bd1ca207 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:31.168358 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a0260-89bf-4c15-a4d4-faf3bd1ca207 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-7a4a0260-89bf-4c15-a4d4-faf3bd1ca207 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:21:31.184900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:31.186420 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:31.187853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:31.187853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:31.187853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:21:31.187853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:21:31.187853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:21:31.187853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:21:31.187853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:21:31.187853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:21:31.187853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:21:31.187853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:31.190445 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 609/1212] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:31.194832 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f02af7-6b83-49d4-92c1-c39efdc221bf None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:31.314299 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f02af7-6b83-49d4-92c1-c39efdc221bf None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:31.356419 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd8a8c-c557-4345-9058-fa635244b7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:31.357133 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd8a8c-c557-4345-9058-fa635244b7e1 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:31.357438 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd8a8c-c557-4345-9058-fa635244b7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:31.357734 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd8a8c-c557-4345-9058-fa635244b7e1 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:31.358764 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd8a8c-c557-4345-9058-fa635244b7e1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:31.359104 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd8a8c-c557-4345-9058-fa635244b7e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:31.360675 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd8a8c-c557-4345-9058-fa635244b7e1 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:31.385838 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 604/1213] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 20:21:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:21:31.387116 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a0260-89bf-4c15-a4d4-faf3bd1ca207 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:31.505763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a0260-89bf-4c15-a4d4-faf3bd1ca207 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:31.543494 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d239401-9bab-4fe2-b425-3052187f107b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:31.544169 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d239401-9bab-4fe2-b425-3052187f107b None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:31.544476 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d239401-9bab-4fe2-b425-3052187f107b None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:31.544883 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d239401-9bab-4fe2-b425-3052187f107b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:31.588210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:31.637288 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d239401-9bab-4fe2-b425-3052187f107b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:31.642308 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d239401-9bab-4fe2-b425-3052187f107b None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:31.754220 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 610/1214] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:21:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:21:31.760221 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd8a8c-c557-4345-9058-fa635244b7e1 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:31.781367 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:31.875729 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd8a8c-c557-4345-9058-fa635244b7e1 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:31.917027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea837a5-edf0-4fc3-ad08-8611a39894e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:31.917882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea837a5-edf0-4fc3-ad08-8611a39894e0 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:31.918199 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea837a5-edf0-4fc3-ad08-8611a39894e0 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:31.918509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea837a5-edf0-4fc3-ad08-8611a39894e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:31.932306 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea837a5-edf0-4fc3-ad08-8611a39894e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:31.933796 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea837a5-edf0-4fc3-ad08-8611a39894e0 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:31.952146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-bea837a5-edf0-4fc3-ad08-8611a39894e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:31.952146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:31.952146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:31.952146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:31.952146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:31.952146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:31.952146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:31.952146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:31.952146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:31.952146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:31.952146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:31.952146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:31.952146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:21:31.952146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:21:31.952146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:31.953538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:31.954892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:21:31.954892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:21:31.954892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:31.954892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:31.954892 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 605/1215] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:21:31.960084 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d239401-9bab-4fe2-b425-3052187f107b None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:32.036612 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:32.081594 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d239401-9bab-4fe2-b425-3052187f107b None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:32.124862 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afc92b-9c7e-474b-bbc5-d816e8971562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:32.125508 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afc92b-9c7e-474b-bbc5-d816e8971562 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:32.125766 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afc92b-9c7e-474b-bbc5-d816e8971562 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:32.126042 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afc92b-9c7e-474b-bbc5-d816e8971562 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:32.126867 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afc92b-9c7e-474b-bbc5-d816e8971562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:32.127227 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afc92b-9c7e-474b-bbc5-d816e8971562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:32.130315 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afc92b-9c7e-474b-bbc5-d816e8971562 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:32.146539 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 611/1216] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:21:32.151999 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea837a5-edf0-4fc3-ad08-8611a39894e0 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:32.236243 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:32.267571 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea837a5-edf0-4fc3-ad08-8611a39894e0 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:32.310918 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6b960-22af-40ff-b697-ce1f43b4d904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:32.310918 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6b960-22af-40ff-b697-ce1f43b4d904 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:32.310918 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6b960-22af-40ff-b697-ce1f43b4d904 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:32.310918 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6b960-22af-40ff-b697-ce1f43b4d904 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:32.317938 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6b960-22af-40ff-b697-ce1f43b4d904 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:32.319547 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6b960-22af-40ff-b697-ce1f43b4d904 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:32.334786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-56d6b960-22af-40ff-b697-ce1f43b4d904 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:21:32.334786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:32.334786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:32.334786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:32.334786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:32.334786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:32.334786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:32.334786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:32.334786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:32.334786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:32.334786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:32.334786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:32.334786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:21:32.334786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:21:32.334786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:32.341246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:32.342569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:21:32.342569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:21:32.342569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:21:32.342569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:21:32.342569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:21:32.342569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:21:32.342569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:21:32.342569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:32.342569 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 606/1217] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:21:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:21:32.343310 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afc92b-9c7e-474b-bbc5-d816e8971562 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:32.474939 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afc92b-9c7e-474b-bbc5-d816e8971562 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:32.506932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:32.527634 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b490ad-1fd9-4564-bdee-ae8ad7833f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:32.528933 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b490ad-1fd9-4564-bdee-ae8ad7833f7f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:32.529628 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b490ad-1fd9-4564-bdee-ae8ad7833f7f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:32.529986 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b490ad-1fd9-4564-bdee-ae8ad7833f7f None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:32.531281 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b490ad-1fd9-4564-bdee-ae8ad7833f7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:32.531766 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b490ad-1fd9-4564-bdee-ae8ad7833f7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:32.533879 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b490ad-1fd9-4564-bdee-ae8ad7833f7f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:32.556417 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 612/1218] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:21:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:32.557208 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6b960-22af-40ff-b697-ce1f43b4d904 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:32.669874 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6b960-22af-40ff-b697-ce1f43b4d904 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:32.704683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:32.713314 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b77286-0a2c-4cf5-9c7c-7f5ab1e7ac81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:32.714029 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b77286-0a2c-4cf5-9c7c-7f5ab1e7ac81 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:32.714327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b77286-0a2c-4cf5-9c7c-7f5ab1e7ac81 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:32.714625 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b77286-0a2c-4cf5-9c7c-7f5ab1e7ac81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:32.715597 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b77286-0a2c-4cf5-9c7c-7f5ab1e7ac81 None admin] RBAC: Adding query filter params (role.id=c9206a4a0d49473f87a45d6c8abb2e51, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=8cd315b75b1f4be5a788dc70ae07194d) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:32.715775 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b77286-0a2c-4cf5-9c7c-7f5ab1e7ac81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:32.717238 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b77286-0a2c-4cf5-9c7c-7f5ab1e7ac81 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:32.724169 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 607/1219] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:21:32 2022] GET /identity/v3/role_assignments?user.id=8cd315b75b1f4be5a788dc70ae07194d&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=c9206a4a0d49473f87a45d6c8abb2e51 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:35.036854 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0199684-f18f-4b74-a99a-bf7f135f1ab6 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:35.037606 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0199684-f18f-4b74-a99a-bf7f135f1ab6 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:35.037606 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0199684-f18f-4b74-a99a-bf7f135f1ab6 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:35.039060 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 613/1220] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:21:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:21:35.044807 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba1021-28fd-45ef-bb44-8b87658c565f None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:35.045278 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba1021-28fd-45ef-bb44-8b87658c565f None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:35.045396 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba1021-28fd-45ef-bb44-8b87658c565f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:35.064227 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-f3ba1021-28fd-45ef-bb44-8b87658c565f None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:21:35.065435 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-f3ba1021-28fd-45ef-bb44-8b87658c565f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:21:35.140409 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 608/1221] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:21:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:21:35.164330 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0199684-f18f-4b74-a99a-bf7f135f1ab6 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:35.274721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0199684-f18f-4b74-a99a-bf7f135f1ab6 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:35.311563 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e90ae-40df-47fb-a23d-92d3c4911a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:35.312279 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e90ae-40df-47fb-a23d-92d3c4911a22 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:35.312570 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e90ae-40df-47fb-a23d-92d3c4911a22 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:35.312881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e90ae-40df-47fb-a23d-92d3c4911a22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:35.323654 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e90ae-40df-47fb-a23d-92d3c4911a22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-562e90ae-40df-47fb-a23d-92d3c4911a22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:21:35.331495 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:35.333704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:35.334978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:35.334978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:35.334978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:35.334978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:35.334978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:21:35.334978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:21:35.334978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:21:35.334978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:21:35.334978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:21:35.334978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:21:35.334978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:21:35.334978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:35.334978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 614/1222] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:21:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:35.339603 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba1021-28fd-45ef-bb44-8b87658c565f None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:35.442698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba1021-28fd-45ef-bb44-8b87658c565f None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:35.468100 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:35.480101 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93ea59-35e6-4def-9b48-789d2e837c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:35.480781 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93ea59-35e6-4def-9b48-789d2e837c6e None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:35.481065 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93ea59-35e6-4def-9b48-789d2e837c6e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:35.481341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93ea59-35e6-4def-9b48-789d2e837c6e None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:35.482146 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93ea59-35e6-4def-9b48-789d2e837c6e None admin] RBAC: Adding query filter params (name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:35.482451 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93ea59-35e6-4def-9b48-789d2e837c6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:35.483834 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93ea59-35e6-4def-9b48-789d2e837c6e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:35.493240 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 609/1223] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:21:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:21:35.499246 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e90ae-40df-47fb-a23d-92d3c4911a22 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:35.546411 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:35.616469 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e90ae-40df-47fb-a23d-92d3c4911a22 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:35.660137 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c8bde-dc2e-4e72-a7f0-4482fe09bd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:35.661391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c8bde-dc2e-4e72-a7f0-4482fe09bd80 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:35.661891 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c8bde-dc2e-4e72-a7f0-4482fe09bd80 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:35.662354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c8bde-dc2e-4e72-a7f0-4482fe09bd80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:35.675503 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c8bde-dc2e-4e72-a7f0-4482fe09bd80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:35.677305 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c8bde-dc2e-4e72-a7f0-4482fe09bd80 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:35.696740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-9b9c8bde-dc2e-4e72-a7f0-4482fe09bd80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:35.696740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:35.696740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:35.696740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:35.696740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:35.696740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:35.696740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:35.696740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:35.696740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:35.696740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:35.696740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:35.696740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:35.696740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:21:35.696740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:21:35.696740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:35.700881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:35.703396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:21:35.703396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:21:35.703396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:35.703396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:35.703396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 615/1224] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:21:35.704026 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93ea59-35e6-4def-9b48-789d2e837c6e None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:35.810183 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93ea59-35e6-4def-9b48-789d2e837c6e None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:35.846884 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbc5cd-ad1c-4e04-bc9a-ab10ab94d9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:35.847840 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbc5cd-ad1c-4e04-bc9a-ab10ab94d9f8 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:35.848300 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbc5cd-ad1c-4e04-bc9a-ab10ab94d9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:35.848566 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbc5cd-ad1c-4e04-bc9a-ab10ab94d9f8 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:35.849335 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbc5cd-ad1c-4e04-bc9a-ab10ab94d9f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:35.849698 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbc5cd-ad1c-4e04-bc9a-ab10ab94d9f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:35.851098 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbc5cd-ad1c-4e04-bc9a-ab10ab94d9f8 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:35.864164 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 610/1225] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:35.870139 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c8bde-dc2e-4e72-a7f0-4482fe09bd80 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:35.976892 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:35.983998 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c8bde-dc2e-4e72-a7f0-4482fe09bd80 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:36.022808 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f54cf1-bf59-4f3f-92a2-b8addc28aea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:36.023721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f54cf1-bf59-4f3f-92a2-b8addc28aea7 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:36.024076 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f54cf1-bf59-4f3f-92a2-b8addc28aea7 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:36.024415 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f54cf1-bf59-4f3f-92a2-b8addc28aea7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:36.036278 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f54cf1-bf59-4f3f-92a2-b8addc28aea7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:36.037823 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f54cf1-bf59-4f3f-92a2-b8addc28aea7 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-f6f54cf1-bf59-4f3f-92a2-b8addc28aea7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:21:36.054071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:36.055687 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:36.057191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:36.057191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:36.057191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:21:36.057191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:21:36.057191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:21:36.057191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:21:36.057191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:21:36.057191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:21:36.057191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:21:36.057191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:36.057191 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 616/1226] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:36.061197 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbc5cd-ad1c-4e04-bc9a-ab10ab94d9f8 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:36.116663 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:36.179572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbc5cd-ad1c-4e04-bc9a-ab10ab94d9f8 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:36.219977 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70032486-fb7a-4ffc-a83c-cf905b7e72b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:36.220652 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70032486-fb7a-4ffc-a83c-cf905b7e72b0 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:36.220935 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70032486-fb7a-4ffc-a83c-cf905b7e72b0 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:36.221208 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70032486-fb7a-4ffc-a83c-cf905b7e72b0 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:36.222044 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70032486-fb7a-4ffc-a83c-cf905b7e72b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:36.222381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70032486-fb7a-4ffc-a83c-cf905b7e72b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:36.223869 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70032486-fb7a-4ffc-a83c-cf905b7e72b0 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:36.241774 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 611/1227] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 20:21:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:36.248464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f54cf1-bf59-4f3f-92a2-b8addc28aea7 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:36.350710 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f54cf1-bf59-4f3f-92a2-b8addc28aea7 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:36.386020 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e12007-4ec6-4eb1-96cb-73288dc189de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:36.386849 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e12007-4ec6-4eb1-96cb-73288dc189de None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:36.387129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e12007-4ec6-4eb1-96cb-73288dc189de None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:36.387396 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e12007-4ec6-4eb1-96cb-73288dc189de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:36.399099 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e12007-4ec6-4eb1-96cb-73288dc189de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:36.400576 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e12007-4ec6-4eb1-96cb-73288dc189de None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:36.420102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-03e12007-4ec6-4eb1-96cb-73288dc189de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:36.420102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:36.420102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:36.420102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:36.420102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:36.420102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:36.420102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:36.420102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:36.420102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:36.420102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:36.420102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:36.420102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:36.420102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:21:36.420102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:21:36.420102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:36.426543 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:36.427952 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:21:36.427952 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:21:36.427952 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:36.427952 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:36.427952 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 617/1228] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:36.428479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70032486-fb7a-4ffc-a83c-cf905b7e72b0 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:36.535410 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70032486-fb7a-4ffc-a83c-cf905b7e72b0 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:36.538173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:36.571616 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31c5a6-c97a-4c2f-9087-8d250a1f2994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:36.572654 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31c5a6-c97a-4c2f-9087-8d250a1f2994 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:36.573106 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31c5a6-c97a-4c2f-9087-8d250a1f2994 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:36.573533 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31c5a6-c97a-4c2f-9087-8d250a1f2994 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:36.574727 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31c5a6-c97a-4c2f-9087-8d250a1f2994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:36.575200 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31c5a6-c97a-4c2f-9087-8d250a1f2994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:36.577398 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31c5a6-c97a-4c2f-9087-8d250a1f2994 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:36.595194 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 612/1229] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:36.601538 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e12007-4ec6-4eb1-96cb-73288dc189de None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:36.614169 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:36.712714 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e12007-4ec6-4eb1-96cb-73288dc189de None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:36.750895 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f36d6-fe48-48f4-a294-9acb91ddf96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:36.751720 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f36d6-fe48-48f4-a294-9acb91ddf96d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:36.752004 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f36d6-fe48-48f4-a294-9acb91ddf96d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:36.752280 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f36d6-fe48-48f4-a294-9acb91ddf96d None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:36.759913 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f36d6-fe48-48f4-a294-9acb91ddf96d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:36.761397 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f36d6-fe48-48f4-a294-9acb91ddf96d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:36.777006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-163f36d6-fe48-48f4-a294-9acb91ddf96d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:21:36.777006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:36.777006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:36.777006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:36.777006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:36.777006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:36.777006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:36.777006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:36.777006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:36.777006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:36.777006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:36.777006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:36.777006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:21:36.777006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:21:36.777006 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:36.784568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:36.786507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:21:36.786507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:21:36.786507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:21:36.786507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:21:36.786507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:21:36.786507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:21:36.786507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:21:36.786507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:36.786507 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 618/1230] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:21:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:21:36.787351 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31c5a6-c97a-4c2f-9087-8d250a1f2994 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:36.907667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31c5a6-c97a-4c2f-9087-8d250a1f2994 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:36.945423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f100bc3-5247-4b11-9173-b7c11a942156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:36.946149 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f100bc3-5247-4b11-9173-b7c11a942156 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:36.946501 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f100bc3-5247-4b11-9173-b7c11a942156 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:36.946780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f100bc3-5247-4b11-9173-b7c11a942156 None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:36.947674 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f100bc3-5247-4b11-9173-b7c11a942156 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:36.948084 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f100bc3-5247-4b11-9173-b7c11a942156 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:36.949747 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f100bc3-5247-4b11-9173-b7c11a942156 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:36.965453 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 613/1231] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:21:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:36.972731 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f36d6-fe48-48f4-a294-9acb91ddf96d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:37.046680 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:37.084985 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f36d6-fe48-48f4-a294-9acb91ddf96d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:37.122866 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef510e1b-63c8-45cf-8139-8c1e01e333f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:37.123795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef510e1b-63c8-45cf-8139-8c1e01e333f7 None admin] REQUEST_METHOD: `PUT` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:37.124054 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef510e1b-63c8-45cf-8139-8c1e01e333f7 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:37.124356 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef510e1b-63c8-45cf-8139-8c1e01e333f7 None admin] PATH_INFO: `/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/8cd315b75b1f4be5a788dc70ae07194d/roles/c9206a4a0d49473f87a45d6c8abb2e51` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:37.141053 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef510e1b-63c8-45cf-8139-8c1e01e333f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=59555e7adac54eddb47d1ef18ef870c4, user_id=8cd315b75b1f4be5a788dc70ae07194d, role_id=c9206a4a0d49473f87a45d6c8abb2e51)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:37.142588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef510e1b-63c8-45cf-8139-8c1e01e333f7 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:37.157105 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 619/1232] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:21:36 2022] PUT /identity/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/8cd315b75b1f4be5a788dc70ae07194d/roles/c9206a4a0d49473f87a45d6c8abb2e51 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:21:39.537363 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb7ebc-737d-49f7-9c26-d1af5ea096d6 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:39.537952 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb7ebc-737d-49f7-9c26-d1af5ea096d6 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:39.537952 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb7ebc-737d-49f7-9c26-d1af5ea096d6 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:39.539344 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 614/1233] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:21:39.544994 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38663a2b-7e6a-43ed-a5b5-7d388351e302 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:39.545448 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38663a2b-7e6a-43ed-a5b5-7d388351e302 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:39.545683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38663a2b-7e6a-43ed-a5b5-7d388351e302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:39.564873 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-38663a2b-7e6a-43ed-a5b5-7d388351e302 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:21:39.566100 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-38663a2b-7e6a-43ed-a5b5-7d388351e302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:21:39.643713 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 620/1234] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:21:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:21:39.669652 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb7ebc-737d-49f7-9c26-d1af5ea096d6 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:39.698036 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:39.788636 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb7ebc-737d-49f7-9c26-d1af5ea096d6 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:39.827082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4520b54-83ec-4ecc-861d-d2f64c2b375a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:39.827893 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4520b54-83ec-4ecc-861d-d2f64c2b375a None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:39.828173 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4520b54-83ec-4ecc-861d-d2f64c2b375a None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:39.828447 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4520b54-83ec-4ecc-861d-d2f64c2b375a None admin] PATH_INFO: `/v3/roles/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:39.840241 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4520b54-83ec-4ecc-861d-d2f64c2b375a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-f4520b54-83ec-4ecc-861d-d2f64c2b375a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:21:39.848218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:39.849595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:39.850885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:39.850885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:39.850885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:39.850885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:39.850885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:21:39.850885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:21:39.850885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:21:39.850885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:21:39.850885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:21:39.850885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:21:39.850885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:21:39.850885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:39.854508 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 615/1235] 10.222.0.84 () {56 vars in 1102 bytes} [Thu May 19 20:21:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:39.855736 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38663a2b-7e6a-43ed-a5b5-7d388351e302 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:39.905565 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:39.960666 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38663a2b-7e6a-43ed-a5b5-7d388351e302 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:39.997391 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6cc8e-099f-46b4-8390-17a84cf8b6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:39.998564 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6cc8e-099f-46b4-8390-17a84cf8b6d1 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:39.999075 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6cc8e-099f-46b4-8390-17a84cf8b6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:39.999359 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6cc8e-099f-46b4-8390-17a84cf8b6d1 None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:40.000286 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6cc8e-099f-46b4-8390-17a84cf8b6d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:40.000606 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6cc8e-099f-46b4-8390-17a84cf8b6d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:40.002240 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6cc8e-099f-46b4-8390-17a84cf8b6d1 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:40.013136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 621/1236] 10.222.0.84 () {56 vars in 1103 bytes} [Thu May 19 20:21:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:40.019625 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4520b54-83ec-4ecc-861d-d2f64c2b375a None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:40.110348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:40.130097 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4520b54-83ec-4ecc-861d-d2f64c2b375a None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:40.165259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f6d9d-6314-4f18-a23b-06d43eefe722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:40.166177 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f6d9d-6314-4f18-a23b-06d43eefe722 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:40.166494 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f6d9d-6314-4f18-a23b-06d43eefe722 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:40.166794 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f6d9d-6314-4f18-a23b-06d43eefe722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:40.179130 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f6d9d-6314-4f18-a23b-06d43eefe722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:40.180609 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f6d9d-6314-4f18-a23b-06d43eefe722 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:40.198138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-767f6d9d-6314-4f18-a23b-06d43eefe722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:40.198138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:40.198138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:40.198138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:40.198138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:40.198138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:40.198138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:40.198138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:40.198138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:40.198138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:40.198138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:40.198138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:40.198138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:21:40.198138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:21:40.198138 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:40.199574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:40.200896 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:21:40.200896 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:21:40.200896 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:40.200896 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:40.202711 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 616/1237] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:40.205741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6cc8e-099f-46b4-8390-17a84cf8b6d1 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:40.316071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6cc8e-099f-46b4-8390-17a84cf8b6d1 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:40.328816 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:40.357268 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110017d-b104-4e54-9699-0091cb0bd88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:40.358070 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110017d-b104-4e54-9699-0091cb0bd88d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:40.358385 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110017d-b104-4e54-9699-0091cb0bd88d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:40.358693 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110017d-b104-4e54-9699-0091cb0bd88d None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:40.359416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110017d-b104-4e54-9699-0091cb0bd88d None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:40.359845 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110017d-b104-4e54-9699-0091cb0bd88d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:40.361480 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110017d-b104-4e54-9699-0091cb0bd88d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:40.374676 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 622/1238] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:40.381036 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f6d9d-6314-4f18-a23b-06d43eefe722 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:40.492229 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f6d9d-6314-4f18-a23b-06d43eefe722 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:40.529362 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7b24c-575b-44f7-84fa-64ba109fe200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:40.530182 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7b24c-575b-44f7-84fa-64ba109fe200 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:40.530501 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7b24c-575b-44f7-84fa-64ba109fe200 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:40.530772 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7b24c-575b-44f7-84fa-64ba109fe200 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:40.534113 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:40.539805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7b24c-575b-44f7-84fa-64ba109fe200 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:40.541548 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7b24c-575b-44f7-84fa-64ba109fe200 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-58c7b24c-575b-44f7-84fa-64ba109fe200 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:21:40.556960 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:40.558426 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:40.559730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:40.559730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:40.559730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:21:40.559730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:21:40.559730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:21:40.559730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:21:40.559730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:21:40.559730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:21:40.559730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:21:40.559730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:40.559730 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 617/1239] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:21:40.564371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110017d-b104-4e54-9699-0091cb0bd88d None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:40.671580 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110017d-b104-4e54-9699-0091cb0bd88d None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:40.710419 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058028db-52ec-41dc-a232-9ab454de39de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:40.711284 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058028db-52ec-41dc-a232-9ab454de39de None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:40.711603 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058028db-52ec-41dc-a232-9ab454de39de None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:40.712334 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058028db-52ec-41dc-a232-9ab454de39de None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:40.713159 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058028db-52ec-41dc-a232-9ab454de39de None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:40.713529 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058028db-52ec-41dc-a232-9ab454de39de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:40.715139 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058028db-52ec-41dc-a232-9ab454de39de None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:40.735117 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 623/1240] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 20:21:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:40.741203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7b24c-575b-44f7-84fa-64ba109fe200 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:40.855583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7b24c-575b-44f7-84fa-64ba109fe200 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:40.892356 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce898c4f-c739-490f-817a-b2974b9aebc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:40.893008 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce898c4f-c739-490f-817a-b2974b9aebc5 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:40.893288 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce898c4f-c739-490f-817a-b2974b9aebc5 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:40.893559 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce898c4f-c739-490f-817a-b2974b9aebc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:40.958565 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:40.966796 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce898c4f-c739-490f-817a-b2974b9aebc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:40.968371 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce898c4f-c739-490f-817a-b2974b9aebc5 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:41.109535 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 618/1241] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:21:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:21:41.116491 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058028db-52ec-41dc-a232-9ab454de39de None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:41.146783 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:41.227867 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058028db-52ec-41dc-a232-9ab454de39de None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:41.267515 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486b8b5-1ded-45a9-b513-d5f8b52a21ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:41.268576 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486b8b5-1ded-45a9-b513-d5f8b52a21ac None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:41.268860 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486b8b5-1ded-45a9-b513-d5f8b52a21ac None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:41.269406 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486b8b5-1ded-45a9-b513-d5f8b52a21ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:41.282858 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486b8b5-1ded-45a9-b513-d5f8b52a21ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:41.284335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486b8b5-1ded-45a9-b513-d5f8b52a21ac None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:41.306588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-5486b8b5-1ded-45a9-b513-d5f8b52a21ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:41.306588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:41.306588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:41.306588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:41.306588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:41.306588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:41.306588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:41.306588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:41.306588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:41.306588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:41.306588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:41.306588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:41.306588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:21:41.306588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:21:41.306588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:41.307764 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:41.308844 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:21:41.308844 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:21:41.308844 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:41.308844 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:41.308844 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 624/1242] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:41.313786 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce898c4f-c739-490f-817a-b2974b9aebc5 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:41.428072 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:41.442831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce898c4f-c739-490f-817a-b2974b9aebc5 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:41.481697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac3206-5891-4c85-9205-41090cc78f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:41.482312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac3206-5891-4c85-9205-41090cc78f83 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:41.482597 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac3206-5891-4c85-9205-41090cc78f83 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:41.482885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac3206-5891-4c85-9205-41090cc78f83 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:41.483707 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac3206-5891-4c85-9205-41090cc78f83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:41.484067 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac3206-5891-4c85-9205-41090cc78f83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:41.485591 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac3206-5891-4c85-9205-41090cc78f83 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:41.497959 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 619/1243] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:41.504054 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486b8b5-1ded-45a9-b513-d5f8b52a21ac None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:41.590400 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:41.615075 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486b8b5-1ded-45a9-b513-d5f8b52a21ac None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:41.654703 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b504a03-a9e4-4011-9d51-96ea6f10d053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:41.655520 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b504a03-a9e4-4011-9d51-96ea6f10d053 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:41.655800 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b504a03-a9e4-4011-9d51-96ea6f10d053 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:41.657083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b504a03-a9e4-4011-9d51-96ea6f10d053 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:41.664748 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b504a03-a9e4-4011-9d51-96ea6f10d053 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:41.667378 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b504a03-a9e4-4011-9d51-96ea6f10d053 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:41.684741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-1b504a03-a9e4-4011-9d51-96ea6f10d053 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:21:41.684741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:41.684741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:41.684741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:41.684741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:41.684741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:41.684741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:41.684741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:41.684741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:41.684741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:41.684741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:41.684741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:41.684741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:21:41.684741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:21:41.684741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:41.686221 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:41.687463 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:21:41.687463 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:21:41.687463 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:21:41.687463 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:21:41.687463 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:21:41.687463 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:21:41.687463 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:21:41.687463 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:41.687463 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 625/1244] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:21:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:41.692896 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac3206-5891-4c85-9205-41090cc78f83 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:41.804278 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac3206-5891-4c85-9205-41090cc78f83 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:41.842673 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2640b2-8971-4c10-bf7b-e57a5cc9bb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:41.843379 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2640b2-8971-4c10-bf7b-e57a5cc9bb99 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:41.843671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2640b2-8971-4c10-bf7b-e57a5cc9bb99 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:41.844110 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2640b2-8971-4c10-bf7b-e57a5cc9bb99 None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:41.844986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2640b2-8971-4c10-bf7b-e57a5cc9bb99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:41.845296 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2640b2-8971-4c10-bf7b-e57a5cc9bb99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:41.846924 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2640b2-8971-4c10-bf7b-e57a5cc9bb99 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:41.860054 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 620/1245] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:21:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:41.866742 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b504a03-a9e4-4011-9d51-96ea6f10d053 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:41.989500 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b504a03-a9e4-4011-9d51-96ea6f10d053 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:42.029203 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c30c1-fed0-4757-8af1-3dbcb952048b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:42.029897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c30c1-fed0-4757-8af1-3dbcb952048b None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:42.030177 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c30c1-fed0-4757-8af1-3dbcb952048b None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:42.030479 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c30c1-fed0-4757-8af1-3dbcb952048b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:42.031300 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c30c1-fed0-4757-8af1-3dbcb952048b None admin] RBAC: Adding query filter params (role.id=c9206a4a0d49473f87a45d6c8abb2e51, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=8cd315b75b1f4be5a788dc70ae07194d) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:42.031733 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c30c1-fed0-4757-8af1-3dbcb952048b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:42.033289 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c30c1-fed0-4757-8af1-3dbcb952048b None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:42.040371 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 626/1246] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:21:41 2022] GET /identity/v3/role_assignments?user.id=8cd315b75b1f4be5a788dc70ae07194d&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=c9206a4a0d49473f87a45d6c8abb2e51 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:21:44.395875 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e5ead-7746-4eb3-aa44-ba7de86b554e None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:44.396514 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e5ead-7746-4eb3-aa44-ba7de86b554e None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:44.396514 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e5ead-7746-4eb3-aa44-ba7de86b554e None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:44.397806 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 621/1247] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:21:44.403197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b67ca-7b56-403e-a954-cbf8a909d963 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:44.403678 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b67ca-7b56-403e-a954-cbf8a909d963 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:44.403793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b67ca-7b56-403e-a954-cbf8a909d963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:44.408906 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:44.422393 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-0b8b67ca-7b56-403e-a954-cbf8a909d963 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:21:44.423586 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-0b8b67ca-7b56-403e-a954-cbf8a909d963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:21:44.500039 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 627/1248] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:21:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:21:44.524624 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e5ead-7746-4eb3-aa44-ba7de86b554e None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:44.544765 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:44.640636 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e5ead-7746-4eb3-aa44-ba7de86b554e None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:44.682357 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fc30b-aaa0-4cd5-9479-b60d1dc4c1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:44.683344 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fc30b-aaa0-4cd5-9479-b60d1dc4c1c5 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:44.683683 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fc30b-aaa0-4cd5-9479-b60d1dc4c1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:44.683949 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fc30b-aaa0-4cd5-9479-b60d1dc4c1c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:44.695314 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57fc30b-aaa0-4cd5-9479-b60d1dc4c1c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-f57fc30b-aaa0-4cd5-9479-b60d1dc4c1c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:21:44.703342 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:44.704596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:44.705915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:44.705915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:44.705915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:44.705915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:44.705915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:21:44.705915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:21:44.705915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:21:44.705915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:21:44.705915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:21:44.705915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:21:44.705915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:21:44.705915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:44.705915 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 622/1249] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:21:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:44.711547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b67ca-7b56-403e-a954-cbf8a909d963 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:44.823090 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b67ca-7b56-403e-a954-cbf8a909d963 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:44.860130 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:44.866510 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934f3a1-4fa1-431e-88d5-66e57bd316be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:44.867576 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934f3a1-4fa1-431e-88d5-66e57bd316be None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:44.868051 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934f3a1-4fa1-431e-88d5-66e57bd316be None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:44.868325 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934f3a1-4fa1-431e-88d5-66e57bd316be None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:44.869169 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934f3a1-4fa1-431e-88d5-66e57bd316be None admin] RBAC: Adding query filter params (name=admin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:44.869492 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934f3a1-4fa1-431e-88d5-66e57bd316be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:44.871100 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934f3a1-4fa1-431e-88d5-66e57bd316be None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:44.880931 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 628/1250] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:21:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:44.886666 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fc30b-aaa0-4cd5-9479-b60d1dc4c1c5 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:44.985122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:45.016671 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fc30b-aaa0-4cd5-9479-b60d1dc4c1c5 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:45.057378 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb00e06-3e5a-4b57-8d23-67a88cd54b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:45.058294 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb00e06-3e5a-4b57-8d23-67a88cd54b27 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:45.058581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb00e06-3e5a-4b57-8d23-67a88cd54b27 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:45.058861 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb00e06-3e5a-4b57-8d23-67a88cd54b27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:45.076912 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb00e06-3e5a-4b57-8d23-67a88cd54b27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:45.078754 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb00e06-3e5a-4b57-8d23-67a88cd54b27 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:45.098594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-abb00e06-3e5a-4b57-8d23-67a88cd54b27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:45.098594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:45.098594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:45.098594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:45.098594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:45.098594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:45.098594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:45.098594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:45.098594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:45.098594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:45.098594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:45.098594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:45.098594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:21:45.098594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:21:45.098594 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:45.099987 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:45.101477 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:21:45.101477 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:21:45.101477 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:45.101477 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:45.101477 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 623/1251] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:45.107084 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934f3a1-4fa1-431e-88d5-66e57bd316be None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:45.230913 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934f3a1-4fa1-431e-88d5-66e57bd316be None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:45.267260 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e8aa0-5324-4da7-a733-1d7e9d3a4b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:45.267867 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e8aa0-5324-4da7-a733-1d7e9d3a4b41 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:45.268154 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e8aa0-5324-4da7-a733-1d7e9d3a4b41 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:45.268436 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e8aa0-5324-4da7-a733-1d7e9d3a4b41 None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:45.269261 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e8aa0-5324-4da7-a733-1d7e9d3a4b41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:45.270238 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e8aa0-5324-4da7-a733-1d7e9d3a4b41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:45.271662 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e8aa0-5324-4da7-a733-1d7e9d3a4b41 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:45.284038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 629/1252] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:45.290583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb00e06-3e5a-4b57-8d23-67a88cd54b27 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:45.406111 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb00e06-3e5a-4b57-8d23-67a88cd54b27 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:45.437415 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:45.447246 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1a36f-100d-44e4-8e76-2ee28a4040d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:45.448101 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1a36f-100d-44e4-8e76-2ee28a4040d7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:45.448395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1a36f-100d-44e4-8e76-2ee28a4040d7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:45.448677 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1a36f-100d-44e4-8e76-2ee28a4040d7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:45.457431 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1a36f-100d-44e4-8e76-2ee28a4040d7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:45.459264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1a36f-100d-44e4-8e76-2ee28a4040d7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-afe1a36f-100d-44e4-8e76-2ee28a4040d7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:21:45.477203 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:45.483612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:45.484798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:45.484798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:45.484798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:21:45.484798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:21:45.484798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:21:45.484798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:21:45.484798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:21:45.484798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:21:45.484798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:21:45.484798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:45.484798 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 624/1253] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:45.485570 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e8aa0-5324-4da7-a733-1d7e9d3a4b41 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:45.546662 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:45.598111 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e8aa0-5324-4da7-a733-1d7e9d3a4b41 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:45.635947 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46311837-207a-4a1f-9a5e-afef74310965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:45.636787 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46311837-207a-4a1f-9a5e-afef74310965 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:45.637146 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46311837-207a-4a1f-9a5e-afef74310965 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:45.637438 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46311837-207a-4a1f-9a5e-afef74310965 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:45.638456 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46311837-207a-4a1f-9a5e-afef74310965 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:45.638770 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46311837-207a-4a1f-9a5e-afef74310965 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:45.640409 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46311837-207a-4a1f-9a5e-afef74310965 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:45.664212 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 630/1254] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 20:21:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:21:45.671127 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1a36f-100d-44e4-8e76-2ee28a4040d7 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:45.796027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1a36f-100d-44e4-8e76-2ee28a4040d7 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:45.834882 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2385ff-7bb1-47b6-9bb7-2403cab603a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:45.835797 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2385ff-7bb1-47b6-9bb7-2403cab603a7 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:45.835954 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2385ff-7bb1-47b6-9bb7-2403cab603a7 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:45.836175 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2385ff-7bb1-47b6-9bb7-2403cab603a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:45.900531 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:45.914744 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2385ff-7bb1-47b6-9bb7-2403cab603a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:45.916221 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2385ff-7bb1-47b6-9bb7-2403cab603a7 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:46.041755 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 625/1255] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:21:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:21:46.047990 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46311837-207a-4a1f-9a5e-afef74310965 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:46.164136 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46311837-207a-4a1f-9a5e-afef74310965 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:46.179521 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:46.202932 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9150983-b4a1-40c7-9158-c7e4245018fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:46.203878 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9150983-b4a1-40c7-9158-c7e4245018fe None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:46.204160 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9150983-b4a1-40c7-9158-c7e4245018fe None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:46.204533 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9150983-b4a1-40c7-9158-c7e4245018fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:46.216766 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9150983-b4a1-40c7-9158-c7e4245018fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:46.218470 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9150983-b4a1-40c7-9158-c7e4245018fe None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:46.238127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-d9150983-b4a1-40c7-9158-c7e4245018fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:46.238127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:46.238127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:46.238127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:46.238127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:46.238127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:46.238127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:46.238127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:46.238127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:46.238127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:46.238127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:46.238127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:46.238127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:21:46.238127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:21:46.238127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:46.239488 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:46.240904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:21:46.240904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:21:46.240904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:46.240904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:46.240904 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 631/1256] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:21:46.245608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2385ff-7bb1-47b6-9bb7-2403cab603a7 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:46.340821 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:46.355225 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2385ff-7bb1-47b6-9bb7-2403cab603a7 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:46.394116 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dc1c1-b3f3-47ce-975c-35af67dd2700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:46.394883 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dc1c1-b3f3-47ce-975c-35af67dd2700 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:46.395230 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dc1c1-b3f3-47ce-975c-35af67dd2700 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:46.395449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dc1c1-b3f3-47ce-975c-35af67dd2700 None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:46.396338 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320dc1c1-b3f3-47ce-975c-35af67dd2700 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:46.396684 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320dc1c1-b3f3-47ce-975c-35af67dd2700 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:46.398376 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320dc1c1-b3f3-47ce-975c-35af67dd2700 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:46.411897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 626/1257] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:46.419015 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9150983-b4a1-40c7-9158-c7e4245018fe None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:46.539408 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9150983-b4a1-40c7-9158-c7e4245018fe None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:46.577774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f57d63-a343-481f-af2d-83e99c9abea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:46.578617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f57d63-a343-481f-af2d-83e99c9abea9 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:46.578961 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f57d63-a343-481f-af2d-83e99c9abea9 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:46.579387 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f57d63-a343-481f-af2d-83e99c9abea9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:46.587313 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f57d63-a343-481f-af2d-83e99c9abea9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:46.588730 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f57d63-a343-481f-af2d-83e99c9abea9 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:46.604857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-91f57d63-a343-481f-af2d-83e99c9abea9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:21:46.604857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:46.604857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:46.604857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:46.604857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:46.604857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:46.604857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:46.604857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:46.604857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:46.604857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:46.604857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:46.604857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:46.604857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:21:46.604857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:21:46.604857 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:46.606124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:46.607692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:21:46.607692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:21:46.607692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:21:46.607692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:21:46.607692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:21:46.607692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:21:46.607692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:21:46.607692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:46.607692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 632/1258] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:21:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:46.612658 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dc1c1-b3f3-47ce-975c-35af67dd2700 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:46.731418 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dc1c1-b3f3-47ce-975c-35af67dd2700 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:46.770512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad1380-1de4-43cd-9298-b58b87a23e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:46.771190 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad1380-1de4-43cd-9298-b58b87a23e26 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:46.771474 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad1380-1de4-43cd-9298-b58b87a23e26 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:46.771750 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad1380-1de4-43cd-9298-b58b87a23e26 None admin] PATH_INFO: `/v3/projects` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:46.772579 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad1380-1de4-43cd-9298-b58b87a23e26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:46.772924 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad1380-1de4-43cd-9298-b58b87a23e26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:46.774566 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad1380-1de4-43cd-9298-b58b87a23e26 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:46.789413 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 627/1259] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:21:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:46.796331 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f57d63-a343-481f-af2d-83e99c9abea9 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:46.826005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:46.910826 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f57d63-a343-481f-af2d-83e99c9abea9 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:46.949256 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14593f78-d863-4b74-a258-7ac19028ae91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:46.950082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14593f78-d863-4b74-a258-7ac19028ae91 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:46.950453 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14593f78-d863-4b74-a258-7ac19028ae91 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:46.950920 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14593f78-d863-4b74-a258-7ac19028ae91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:46.952298 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14593f78-d863-4b74-a258-7ac19028ae91 None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=8cd315b75b1f4be5a788dc70ae07194d) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:46.952774 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14593f78-d863-4b74-a258-7ac19028ae91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:46.954574 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14593f78-d863-4b74-a258-7ac19028ae91 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:46.973695 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 633/1260] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:21:46 2022] GET /identity/v3/role_assignments?user.id=8cd315b75b1f4be5a788dc70ae07194d&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=b239f941cb724331bf28dc489b70ad1d => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:49.175724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363b095-3b50-4720-b06a-c2409664dc70 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:49.176610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363b095-3b50-4720-b06a-c2409664dc70 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:49.176610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363b095-3b50-4720-b06a-c2409664dc70 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:49.178805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 628/1261] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:21:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:21:49.185115 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f95ea4-2261-4e4a-808f-e563a3a954df None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:49.185897 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f95ea4-2261-4e4a-808f-e563a3a954df None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:49.186066 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f95ea4-2261-4e4a-808f-e563a3a954df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:49.212486 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-c1f95ea4-2261-4e4a-808f-e563a3a954df None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:21:49.213683 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-c1f95ea4-2261-4e4a-808f-e563a3a954df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:21:49.287319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 634/1262] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:21:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:21:49.311900 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363b095-3b50-4720-b06a-c2409664dc70 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:49.325765 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:49.441223 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363b095-3b50-4720-b06a-c2409664dc70 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:49.477763 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86fd47-b05f-4a67-b573-065aec1daf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:49.478599 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86fd47-b05f-4a67-b573-065aec1daf95 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:49.478878 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86fd47-b05f-4a67-b573-065aec1daf95 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:49.479153 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86fd47-b05f-4a67-b573-065aec1daf95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:49.489994 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86fd47-b05f-4a67-b573-065aec1daf95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-eb86fd47-b05f-4a67-b573-065aec1daf95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:21:49.497762 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:49.499139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:49.500364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:49.500364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:49.500364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:49.500364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:49.500364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:21:49.500364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:21:49.500364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:21:49.500364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:21:49.500364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:21:49.500364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:21:49.500364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:21:49.500364 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:49.502449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 629/1263] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:21:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:49.505363 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f95ea4-2261-4e4a-808f-e563a3a954df None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:49.524643 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:49.618124 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f95ea4-2261-4e4a-808f-e563a3a954df None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:49.655176 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb046708-3b12-40ab-b372-a3251616f03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:49.655847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb046708-3b12-40ab-b372-a3251616f03c None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:49.656127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb046708-3b12-40ab-b372-a3251616f03c None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:49.656402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb046708-3b12-40ab-b372-a3251616f03c None admin] PATH_INFO: `/v3/roles` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:49.657151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb046708-3b12-40ab-b372-a3251616f03c None admin] RBAC: Adding query filter params (name=admin) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:49.657461 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb046708-3b12-40ab-b372-a3251616f03c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:49.658881 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb046708-3b12-40ab-b372-a3251616f03c None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:49.667345 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 635/1264] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:21:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:49.673542 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86fd47-b05f-4a67-b573-065aec1daf95 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:49.743699 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:49.779277 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86fd47-b05f-4a67-b573-065aec1daf95 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:49.814412 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed829d-7439-4d91-ae37-3217e01368e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:49.815205 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed829d-7439-4d91-ae37-3217e01368e5 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:49.815486 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed829d-7439-4d91-ae37-3217e01368e5 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:49.815759 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed829d-7439-4d91-ae37-3217e01368e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:49.829002 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed829d-7439-4d91-ae37-3217e01368e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:49.830957 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed829d-7439-4d91-ae37-3217e01368e5 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:49.848783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-3aed829d-7439-4d91-ae37-3217e01368e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:49.848783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:49.848783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:49.848783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:49.848783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:49.848783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:49.848783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:49.848783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:49.848783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:49.848783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:49.848783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:49.848783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:49.848783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:21:49.848783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:21:49.848783 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:49.850095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:49.851463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:21:49.851463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:21:49.851463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:49.851463 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:49.854707 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 630/1265] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:49.855872 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb046708-3b12-40ab-b372-a3251616f03c None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:49.947573 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:49.969969 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb046708-3b12-40ab-b372-a3251616f03c None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:50.007466 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc846abb-0988-49df-b187-24996a70f93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:50.008439 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc846abb-0988-49df-b187-24996a70f93b None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:50.008721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc846abb-0988-49df-b187-24996a70f93b None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:50.009059 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc846abb-0988-49df-b187-24996a70f93b None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:50.009726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc846abb-0988-49df-b187-24996a70f93b None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:50.010058 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc846abb-0988-49df-b187-24996a70f93b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:50.011473 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc846abb-0988-49df-b187-24996a70f93b None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:50.024003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 636/1266] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:21:50.029884 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed829d-7439-4d91-ae37-3217e01368e5 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:50.136696 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed829d-7439-4d91-ae37-3217e01368e5 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:50.159643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:50.177365 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc016f-addd-4a2c-ab84-47bc89514984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:50.178172 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc016f-addd-4a2c-ab84-47bc89514984 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:50.178498 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc016f-addd-4a2c-ab84-47bc89514984 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:50.178774 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc016f-addd-4a2c-ab84-47bc89514984 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:50.187129 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc016f-addd-4a2c-ab84-47bc89514984 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:50.188606 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc016f-addd-4a2c-ab84-47bc89514984 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-59cc016f-addd-4a2c-ab84-47bc89514984 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:21:50.203789 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:50.205006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:50.206315 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:50.206315 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:50.206315 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:21:50.206315 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:21:50.206315 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:21:50.206315 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:21:50.206315 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:21:50.206315 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:21:50.206315 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:21:50.206315 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:50.206315 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 631/1267] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:50.211170 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc846abb-0988-49df-b187-24996a70f93b None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:50.319354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc846abb-0988-49df-b187-24996a70f93b None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:50.360916 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434698c4-653e-44e1-a29a-11e803d3dab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:50.361984 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434698c4-653e-44e1-a29a-11e803d3dab4 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:50.362501 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434698c4-653e-44e1-a29a-11e803d3dab4 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:50.362943 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434698c4-653e-44e1-a29a-11e803d3dab4 None admin] PATH_INFO: `/v3/users` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:50.363809 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434698c4-653e-44e1-a29a-11e803d3dab4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:50.364120 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434698c4-653e-44e1-a29a-11e803d3dab4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:50.365569 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434698c4-653e-44e1-a29a-11e803d3dab4 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:50.375104 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:50.384334 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 637/1268] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 20:21:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:50.390074 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc016f-addd-4a2c-ab84-47bc89514984 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:50.520491 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc016f-addd-4a2c-ab84-47bc89514984 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:50.556227 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d04163-a481-420a-87c4-392f4dc96309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:50.557021 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d04163-a481-420a-87c4-392f4dc96309 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:50.557303 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d04163-a481-420a-87c4-392f4dc96309 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:50.557585 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d04163-a481-420a-87c4-392f4dc96309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:50.569224 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d04163-a481-420a-87c4-392f4dc96309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:50.570788 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d04163-a481-420a-87c4-392f4dc96309 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:50.589488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-71d04163-a481-420a-87c4-392f4dc96309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:50.589488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:50.589488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:50.589488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:50.589488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:50.589488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:50.589488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:50.589488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:50.589488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:50.589488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:50.589488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:50.589488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:50.589488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:21:50.589488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:21:50.589488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:50.590679 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:50.591913 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:21:50.591913 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:21:50.591913 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:50.591913 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:50.591913 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 632/1269] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:50.598626 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434698c4-653e-44e1-a29a-11e803d3dab4 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:50.707369 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434698c4-653e-44e1-a29a-11e803d3dab4 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:50.743377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94e68b-0e46-4fd2-85b9-1925e47ac9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:50.744039 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94e68b-0e46-4fd2-85b9-1925e47ac9cb None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:50.744319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94e68b-0e46-4fd2-85b9-1925e47ac9cb None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:50.744593 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94e68b-0e46-4fd2-85b9-1925e47ac9cb None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:50.745314 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94e68b-0e46-4fd2-85b9-1925e47ac9cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:50.745617 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94e68b-0e46-4fd2-85b9-1925e47ac9cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:50.747012 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94e68b-0e46-4fd2-85b9-1925e47ac9cb None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:50.760873 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 638/1270] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:50.769327 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d04163-a481-420a-87c4-392f4dc96309 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:50.805261 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:50.881371 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d04163-a481-420a-87c4-392f4dc96309 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:50.917222 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4c432-e13c-4080-976e-3018ef7772c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:50.918009 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4c432-e13c-4080-976e-3018ef7772c3 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:50.918312 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4c432-e13c-4080-976e-3018ef7772c3 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:50.918596 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4c432-e13c-4080-976e-3018ef7772c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:50.926251 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4c432-e13c-4080-976e-3018ef7772c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:50.927745 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4c432-e13c-4080-976e-3018ef7772c3 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:50.943259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-8fe4c432-e13c-4080-976e-3018ef7772c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:21:50.943259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:50.943259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:50.943259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:50.943259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:50.943259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:50.943259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:50.943259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:50.943259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:50.943259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:50.943259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:50.943259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:50.943259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:21:50.943259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:21:50.943259 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:50.944617 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:50.945905 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:21:50.945905 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:21:50.945905 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:21:50.945905 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:21:50.945905 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:21:50.945905 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:21:50.945905 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:21:50.945905 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:50.945905 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 633/1271] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:21:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:50.950568 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94e68b-0e46-4fd2-85b9-1925e47ac9cb None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:51.023093 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:51.073081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94e68b-0e46-4fd2-85b9-1925e47ac9cb None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:51.112094 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bed4eb-f7a9-40fa-8b15-ff66f3bc6476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:51.112799 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bed4eb-f7a9-40fa-8b15-ff66f3bc6476 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:51.113098 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bed4eb-f7a9-40fa-8b15-ff66f3bc6476 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:51.113374 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bed4eb-f7a9-40fa-8b15-ff66f3bc6476 None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:51.114163 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bed4eb-f7a9-40fa-8b15-ff66f3bc6476 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:51.114508 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bed4eb-f7a9-40fa-8b15-ff66f3bc6476 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:51.115953 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bed4eb-f7a9-40fa-8b15-ff66f3bc6476 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:51.129023 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 639/1272] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:21:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:51.138229 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4c432-e13c-4080-976e-3018ef7772c3 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:51.246795 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:51.256365 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4c432-e13c-4080-976e-3018ef7772c3 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:51.291808 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ef83d-cc0c-4d87-8a4c-8ef1b857826e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:51.292691 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ef83d-cc0c-4d87-8a4c-8ef1b857826e None admin] REQUEST_METHOD: `PUT` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:51.292970 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ef83d-cc0c-4d87-8a4c-8ef1b857826e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:51.293245 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ef83d-cc0c-4d87-8a4c-8ef1b857826e None admin] PATH_INFO: `/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/8cd315b75b1f4be5a788dc70ae07194d/roles/b239f941cb724331bf28dc489b70ad1d` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:51.295328 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ef83d-cc0c-4d87-8a4c-8ef1b857826e None admin] RBAC: Authorizing `identity:create_grant(project_id=59555e7adac54eddb47d1ef18ef870c4, user_id=8cd315b75b1f4be5a788dc70ae07194d, role_id=b239f941cb724331bf28dc489b70ad1d)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:51.296767 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ef83d-cc0c-4d87-8a4c-8ef1b857826e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:51.311244 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 634/1273] 10.222.0.84 () {58 vars in 1439 bytes} [Thu May 19 20:21:51 2022] PUT /identity/v3/projects/59555e7adac54eddb47d1ef18ef870c4/users/8cd315b75b1f4be5a788dc70ae07194d/roles/b239f941cb724331bf28dc489b70ad1d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:21:53.497390 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc346f-17f1-40e7-ab92-fcb9b0f5d087 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:53.498290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc346f-17f1-40e7-ab92-fcb9b0f5d087 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:53.498290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc346f-17f1-40e7-ab92-fcb9b0f5d087 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:53.500581 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 640/1274] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:21:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:21:53.507152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cb88c-fc30-49e0-b80c-5cee80797c2d None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:53.507940 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cb88c-fc30-49e0-b80c-5cee80797c2d None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:53.508143 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cb88c-fc30-49e0-b80c-5cee80797c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:53.530449 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-117cb88c-fc30-49e0-b80c-5cee80797c2d None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:21:53.531657 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-117cb88c-fc30-49e0-b80c-5cee80797c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:21:53.605372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 635/1275] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:21:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:21:53.632008 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc346f-17f1-40e7-ab92-fcb9b0f5d087 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:53.744395 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc346f-17f1-40e7-ab92-fcb9b0f5d087 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:53.762062 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:53.787740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0e6d0-6284-4a63-a5a4-eac6a8a0fc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:53.788552 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0e6d0-6284-4a63-a5a4-eac6a8a0fc77 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:53.788838 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0e6d0-6284-4a63-a5a4-eac6a8a0fc77 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:53.789117 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0e6d0-6284-4a63-a5a4-eac6a8a0fc77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:53.800369 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0e6d0-6284-4a63-a5a4-eac6a8a0fc77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-84b0e6d0-6284-4a63-a5a4-eac6a8a0fc77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:21:53.808401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:53.809639 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:53.811078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:53.811078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:53.811078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:53.811078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:53.811078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:21:53.811078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:21:53.811078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:21:53.811078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:21:53.811078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:21:53.811078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:21:53.811078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:21:53.811078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:53.811078 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 641/1276] 10.222.0.84 () {56 vars in 1096 bytes} [Thu May 19 20:21:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:53.815985 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117cb88c-fc30-49e0-b80c-5cee80797c2d None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:53.910590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:53.933131 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117cb88c-fc30-49e0-b80c-5cee80797c2d None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:53.970687 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575eec3-b9a5-4591-9140-29297381aa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:53.971576 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575eec3-b9a5-4591-9140-29297381aa7d None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:53.971886 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575eec3-b9a5-4591-9140-29297381aa7d None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:53.972231 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575eec3-b9a5-4591-9140-29297381aa7d None admin] PATH_INFO: `/v3/roles` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:53.973060 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575eec3-b9a5-4591-9140-29297381aa7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:53.973408 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575eec3-b9a5-4591-9140-29297381aa7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:53.975020 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575eec3-b9a5-4591-9140-29297381aa7d None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:53.983855 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 636/1277] 10.222.0.84 () {56 vars in 1099 bytes} [Thu May 19 20:21:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:53.990195 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0e6d0-6284-4a63-a5a4-eac6a8a0fc77 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:54.107302 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0e6d0-6284-4a63-a5a4-eac6a8a0fc77 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:54.148933 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1965dd-21ac-4882-ab66-8abe780d954e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:54.149746 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1965dd-21ac-4882-ab66-8abe780d954e None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:54.150025 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1965dd-21ac-4882-ab66-8abe780d954e None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:54.150307 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1965dd-21ac-4882-ab66-8abe780d954e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:54.169033 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1965dd-21ac-4882-ab66-8abe780d954e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:54.170544 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1965dd-21ac-4882-ab66-8abe780d954e None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:54.190888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-6a1965dd-21ac-4882-ab66-8abe780d954e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:54.190888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:54.190888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:54.190888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:54.190888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:54.190888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:54.190888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:54.190888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:54.190888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:54.190888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:54.190888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:54.190888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:54.190888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:21:54.190888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:21:54.190888 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:54.192252 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:54.193424 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:21:54.193424 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:21:54.193424 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:54.193424 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:54.193424 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 642/1278] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:21:54.199301 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575eec3-b9a5-4591-9140-29297381aa7d None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:54.318751 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575eec3-b9a5-4591-9140-29297381aa7d None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:54.355308 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d817163-f467-4671-bd91-12b6bfe3b0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:54.356307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d817163-f467-4671-bd91-12b6bfe3b0fd None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:54.356718 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d817163-f467-4671-bd91-12b6bfe3b0fd None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:54.357099 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d817163-f467-4671-bd91-12b6bfe3b0fd None admin] PATH_INFO: `/v3/domains` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:54.358348 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d817163-f467-4671-bd91-12b6bfe3b0fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:54.358696 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d817163-f467-4671-bd91-12b6bfe3b0fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:54.360902 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d817163-f467-4671-bd91-12b6bfe3b0fd None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:54.374603 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 637/1279] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:54.380566 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1965dd-21ac-4882-ab66-8abe780d954e None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:54.413373 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:54.504762 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1965dd-21ac-4882-ab66-8abe780d954e None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:54.543747 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717269f6-6001-4622-b9b6-b51b0fadf10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:54.544558 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717269f6-6001-4622-b9b6-b51b0fadf10b None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:54.544851 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717269f6-6001-4622-b9b6-b51b0fadf10b None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:54.545082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717269f6-6001-4622-b9b6-b51b0fadf10b None admin] PATH_INFO: `/v3/users/placement` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:54.552817 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717269f6-6001-4622-b9b6-b51b0fadf10b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:54.554390 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717269f6-6001-4622-b9b6-b51b0fadf10b None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-717269f6-6001-4622-b9b6-b51b0fadf10b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:21:54.569752 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:54.571081 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:54.573435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:54.573435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:54.573435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:21:54.573435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:21:54.573435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:21:54.573435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:21:54.573435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:21:54.573435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:21:54.573435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:21:54.573435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:54.573435 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 643/1280] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:21:54.579629 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d817163-f467-4671-bd91-12b6bfe3b0fd None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:54.584514 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:54.691822 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d817163-f467-4671-bd91-12b6bfe3b0fd None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:54.728592 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c9e6c-0dfb-4617-8525-ab77b158c7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:54.729256 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c9e6c-0dfb-4617-8525-ab77b158c7a9 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:54.729532 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c9e6c-0dfb-4617-8525-ab77b158c7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:54.729804 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c9e6c-0dfb-4617-8525-ab77b158c7a9 None admin] PATH_INFO: `/v3/users` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:54.730670 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c9e6c-0dfb-4617-8525-ab77b158c7a9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:54.730979 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c9e6c-0dfb-4617-8525-ab77b158c7a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:54.732427 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c9e6c-0dfb-4617-8525-ab77b158c7a9 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:54.749642 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 638/1281] 10.222.0.84 () {56 vars in 1143 bytes} [Thu May 19 20:21:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:54.757489 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717269f6-6001-4622-b9b6-b51b0fadf10b None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:54.868572 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:54.881523 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717269f6-6001-4622-b9b6-b51b0fadf10b None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:54.921172 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbcd851-87c8-48e0-9be3-e1c39461aedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:54.921822 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcd851-87c8-48e0-9be3-e1c39461aedc None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:54.922107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcd851-87c8-48e0-9be3-e1c39461aedc None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:54.922418 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcd851-87c8-48e0-9be3-e1c39461aedc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:54.999013 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbcd851-87c8-48e0-9be3-e1c39461aedc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:55.000393 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbcd851-87c8-48e0-9be3-e1c39461aedc None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:55.102975 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:55.114104 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 644/1282] 10.222.0.84 () {58 vars in 1282 bytes} [Thu May 19 20:21:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:21:55.120112 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c9e6c-0dfb-4617-8525-ab77b158c7a9 None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:55.189996 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:55.238004 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c9e6c-0dfb-4617-8525-ab77b158c7a9 None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:55.279838 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb710e-5360-4f46-9572-5e60329b9a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:55.280561 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb710e-5360-4f46-9572-5e60329b9a8e None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:55.280846 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb710e-5360-4f46-9572-5e60329b9a8e None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:55.281122 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb710e-5360-4f46-9572-5e60329b9a8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:55.292307 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb710e-5360-4f46-9572-5e60329b9a8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:55.293784 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb710e-5360-4f46-9572-5e60329b9a8e None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:55.311724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-06eb710e-5360-4f46-9572-5e60329b9a8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:55.311724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:55.311724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:55.311724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:55.311724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:55.311724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:55.311724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:55.311724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:55.311724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:55.311724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:55.311724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:55.311724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:55.311724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:21:55.311724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:21:55.311724 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:55.312904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:55.314001 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:21:55.314001 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:21:55.314001 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:21:55.314001 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:55.314001 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 639/1283] 10.222.0.84 () {56 vars in 1108 bytes} [Thu May 19 20:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:55.320169 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbcd851-87c8-48e0-9be3-e1c39461aedc None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:55.440767 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbcd851-87c8-48e0-9be3-e1c39461aedc None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:55.477025 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ae7b5-1d2a-4c6c-87c2-f05607d587cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:55.477692 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ae7b5-1d2a-4c6c-87c2-f05607d587cd None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:55.477973 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ae7b5-1d2a-4c6c-87c2-f05607d587cd None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:55.478250 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ae7b5-1d2a-4c6c-87c2-f05607d587cd None admin] PATH_INFO: `/v3/domains` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:55.478999 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ae7b5-1d2a-4c6c-87c2-f05607d587cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:55.479306 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ae7b5-1d2a-4c6c-87c2-f05607d587cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:55.480673 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ae7b5-1d2a-4c6c-87c2-f05607d587cd None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:55.492109 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 645/1284] 10.222.0.84 () {56 vars in 1109 bytes} [Thu May 19 20:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:55.500816 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb710e-5360-4f46-9572-5e60329b9a8e None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:55.610526 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb710e-5360-4f46-9572-5e60329b9a8e None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:55.625144 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:55.656264 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a9ba6-ae38-483c-93fd-90697d0b462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:55.657059 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a9ba6-ae38-483c-93fd-90697d0b462f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:55.657341 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a9ba6-ae38-483c-93fd-90697d0b462f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:55.657610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a9ba6-ae38-483c-93fd-90697d0b462f None admin] PATH_INFO: `/v3/projects/service` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:55.665598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a9ba6-ae38-483c-93fd-90697d0b462f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:55.667096 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a9ba6-ae38-483c-93fd-90697d0b462f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:55.681660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application [None req-066a9ba6-ae38-483c-93fd-90697d0b462f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:21:55.681660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:55.681660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:55.681660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:55.681660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:55.681660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:55.681660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:55.681660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:55.681660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:55.681660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:55.681660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:55.681660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:55.681660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:21:55.681660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:21:55.681660 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application with Lock( May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:55.682986 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:55.684533 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:21:55.684533 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:21:55.684533 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:21:55.684533 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:21:55.684533 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:21:55.684533 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:21:55.684533 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:21:55.684533 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: ERROR keystone.server.flask.application May 19 20:21:55.684533 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 640/1285] 10.222.0.84 () {56 vars in 1111 bytes} [Thu May 19 20:21:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:21:55.688961 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ae7b5-1d2a-4c6c-87c2-f05607d587cd None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:55.748102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:55.800023 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ae7b5-1d2a-4c6c-87c2-f05607d587cd None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:55.839695 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00524745-85dd-4c2e-b037-bddee1ac70d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:55.840354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00524745-85dd-4c2e-b037-bddee1ac70d7 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:55.840631 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00524745-85dd-4c2e-b037-bddee1ac70d7 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:55.840907 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00524745-85dd-4c2e-b037-bddee1ac70d7 None admin] PATH_INFO: `/v3/projects` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:55.841694 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00524745-85dd-4c2e-b037-bddee1ac70d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:55.842003 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00524745-85dd-4c2e-b037-bddee1ac70d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:55.843458 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00524745-85dd-4c2e-b037-bddee1ac70d7 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:55.856508 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 646/1286] 10.222.0.84 () {56 vars in 1148 bytes} [Thu May 19 20:21:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:21:55.863043 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a9ba6-ae38-483c-93fd-90697d0b462f None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:55.973381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a9ba6-ae38-483c-93fd-90697d0b462f None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:56.010844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a34846-87f3-4541-8d0a-034768785f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:56.011499 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a34846-87f3-4541-8d0a-034768785f97 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:56.011777 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a34846-87f3-4541-8d0a-034768785f97 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:56.012049 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a34846-87f3-4541-8d0a-034768785f97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:56.012885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a34846-87f3-4541-8d0a-034768785f97 None admin] RBAC: Adding query filter params (role.id=b239f941cb724331bf28dc489b70ad1d, scope.project.id=59555e7adac54eddb47d1ef18ef870c4, user.id=8cd315b75b1f4be5a788dc70ae07194d) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:56.013192 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a34846-87f3-4541-8d0a-034768785f97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:56.014676 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a34846-87f3-4541-8d0a-034768785f97 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:56.021314 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 641/1287] 10.222.0.84 () {56 vars in 1374 bytes} [Thu May 19 20:21:55 2022] GET /identity/v3/role_assignments?user.id=8cd315b75b1f4be5a788dc70ae07194d&scope.project.id=59555e7adac54eddb47d1ef18ef870c4&role.id=b239f941cb724331bf28dc489b70ad1d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:21:58.226952 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bf149-3ec9-4496-80d6-71ea88a80ceb None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:58.227533 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bf149-3ec9-4496-80d6-71ea88a80ceb None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:58.227533 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bf149-3ec9-4496-80d6-71ea88a80ceb None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:58.228900 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 647/1288] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:21:58.234441 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041acaab-a304-4ce9-9e44-090df6c45a47 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:58.234908 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041acaab-a304-4ce9-9e44-090df6c45a47 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:58.235020 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041acaab-a304-4ce9-9e44-090df6c45a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:58.253769 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-041acaab-a304-4ce9-9e44-090df6c45a47 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:21:58.254965 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-041acaab-a304-4ce9-9e44-090df6c45a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:21:58.285875 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:58.337685 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 642/1289] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:21:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:21:58.359772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bf149-3ec9-4496-80d6-71ea88a80ceb None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:58.479958 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bf149-3ec9-4496-80d6-71ea88a80ceb None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:58.491410 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:21:58.517835 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedbec6-8fdf-4184-90d4-61551c5ae8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:58.518716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedbec6-8fdf-4184-90d4-61551c5ae8a7 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:58.518987 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedbec6-8fdf-4184-90d4-61551c5ae8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:58.519280 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedbec6-8fdf-4184-90d4-61551c5ae8a7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:58.519964 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bedbec6-8fdf-4184-90d4-61551c5ae8a7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:58.521333 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bedbec6-8fdf-4184-90d4-61551c5ae8a7 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-3bedbec6-8fdf-4184-90d4-61551c5ae8a7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:21:58.533555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:21:58.535481 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:21:58.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:21:58.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:21:58.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:21:58.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:21:58.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:21:58.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:21:58.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:21:58.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:21:58.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:21:58.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:21:58.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:21:58.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:21:58.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:21:58.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:21:58.537304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 648/1290] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:21:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:21:58.544354 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041acaab-a304-4ce9-9e44-090df6c45a47 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:58.657904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041acaab-a304-4ce9-9e44-090df6c45a47 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:58.693637 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ee907-77a4-4c06-b209-c1fe18bd647b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:58.694308 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ee907-77a4-4c06-b209-c1fe18bd647b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:58.694595 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ee907-77a4-4c06-b209-c1fe18bd647b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:58.694869 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ee907-77a4-4c06-b209-c1fe18bd647b None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:58.695564 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ee907-77a4-4c06-b209-c1fe18bd647b None admin] RBAC: Adding query filter params (name=placement) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:58.695873 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ee907-77a4-4c06-b209-c1fe18bd647b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:58.697242 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ee907-77a4-4c06-b209-c1fe18bd647b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:58.702218 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 643/1291] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 20:21:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:21:58.708853 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedbec6-8fdf-4184-90d4-61551c5ae8a7 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:21:58.832915 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedbec6-8fdf-4184-90d4-61551c5ae8a7 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:21:58.878246 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a61606-f9f0-4e26-bc97-25b2dd77eb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:21:58.878987 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a61606-f9f0-4e26-bc97-25b2dd77eb0d None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:21:58.879276 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a61606-f9f0-4e26-bc97-25b2dd77eb0d None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:21:58.879555 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a61606-f9f0-4e26-bc97-25b2dd77eb0d None admin] PATH_INFO: `/v3/services` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:21:58.880267 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a61606-f9f0-4e26-bc97-25b2dd77eb0d None admin] RBAC: Adding query filter params (type=placement) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:21:58.880577 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a61606-f9f0-4e26-bc97-25b2dd77eb0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:21:58.882024 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a61606-f9f0-4e26-bc97-25b2dd77eb0d None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:21:58.886936 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 649/1292] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 20:21:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:22:01.095445 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9be6f5-f196-4f11-8adc-a52d24bda87d None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:22:01.096077 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9be6f5-f196-4f11-8adc-a52d24bda87d None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:22:01.096077 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9be6f5-f196-4f11-8adc-a52d24bda87d None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:22:01.097541 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 644/1293] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:22:01.104030 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb7b25-4375-4a26-9f50-2fea01a7791f None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:22:01.104763 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb7b25-4375-4a26-9f50-2fea01a7791f None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:22:01.104978 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb7b25-4375-4a26-9f50-2fea01a7791f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:22:01.126304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-55bb7b25-4375-4a26-9f50-2fea01a7791f None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:22:01.127527 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-55bb7b25-4375-4a26-9f50-2fea01a7791f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:22:01.163461 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:22:01.216856 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 650/1294] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:22:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:22:01.240928 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9be6f5-f196-4f11-8adc-a52d24bda87d None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:22:01.280402 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:22:01.371336 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9be6f5-f196-4f11-8adc-a52d24bda87d None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:22:01.408513 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265dbbb-a44b-4816-88b8-27bbe0c1af20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:22:01.409184 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265dbbb-a44b-4816-88b8-27bbe0c1af20 None admin] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:22:01.410015 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265dbbb-a44b-4816-88b8-27bbe0c1af20 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:22:01.410357 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265dbbb-a44b-4816-88b8-27bbe0c1af20 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:22:01.411094 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265dbbb-a44b-4816-88b8-27bbe0c1af20 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:22:01.412489 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265dbbb-a44b-4816-88b8-27bbe0c1af20 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:22:01.428372 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 645/1295] 10.222.0.84 () {60 vars in 1141 bytes} [Thu May 19 20:22:01 2022] POST /identity/v3/services => generated 246 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:22:03.698292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85738388-8d63-4f6b-95f7-50fb062ad390 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:22:03.698972 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85738388-8d63-4f6b-95f7-50fb062ad390 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:22:03.698972 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85738388-8d63-4f6b-95f7-50fb062ad390 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:22:03.700778 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 651/1296] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:22:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:22:03.709887 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2c61b-3e92-486c-bc8d-d8439a595ebf None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:22:03.710361 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2c61b-3e92-486c-bc8d-d8439a595ebf None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:22:03.710496 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2c61b-3e92-486c-bc8d-d8439a595ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:22:03.733061 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-72e2c61b-3e92-486c-bc8d-d8439a595ebf None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:22:03.734286 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-72e2c61b-3e92-486c-bc8d-d8439a595ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:22:03.823885 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 646/1297] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:22:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:22:03.848099 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85738388-8d63-4f6b-95f7-50fb062ad390 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:22:03.990151 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85738388-8d63-4f6b-95f7-50fb062ad390 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:22:04.031037 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6959f02-a650-4145-bea6-fb3cb9d65124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:22:04.031848 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6959f02-a650-4145-bea6-fb3cb9d65124 None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:22:04.032129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6959f02-a650-4145-bea6-fb3cb9d65124 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:22:04.032402 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6959f02-a650-4145-bea6-fb3cb9d65124 None admin] PATH_INFO: `/v3/services/placement` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:22:04.033056 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6959f02-a650-4145-bea6-fb3cb9d65124 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:22:04.034726 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6959f02-a650-4145-bea6-fb3cb9d65124 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-a6959f02-a650-4145-bea6-fb3cb9d65124 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:22:04.047595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:22:04.049083 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:22:04.050598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:22:04.050598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:22:04.050598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:22:04.050598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:22:04.050598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:22:04.050598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:22:04.050598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:22:04.050598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:22:04.050598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:22:04.050598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:22:04.050598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:22:04.050598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:22:04.050598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:22:04.050598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:22:04.050598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 652/1298] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:22:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:22:04.055359 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2c61b-3e92-486c-bc8d-d8439a595ebf None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:22:04.076053 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:22:04.167408 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2c61b-3e92-486c-bc8d-d8439a595ebf None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:22:04.214455 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa6118-2e61-4049-8c37-be9986e7183b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:22:04.215127 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa6118-2e61-4049-8c37-be9986e7183b None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:22:04.215407 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa6118-2e61-4049-8c37-be9986e7183b None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:22:04.215684 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa6118-2e61-4049-8c37-be9986e7183b None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:22:04.216387 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa6118-2e61-4049-8c37-be9986e7183b None admin] RBAC: Adding query filter params (name=placement) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:22:04.216697 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa6118-2e61-4049-8c37-be9986e7183b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:22:04.218092 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa6118-2e61-4049-8c37-be9986e7183b None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:22:04.224097 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 647/1299] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 20:22:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:22:04.233531 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6959f02-a650-4145-bea6-fb3cb9d65124 None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:22:04.274705 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:22:04.360905 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6959f02-a650-4145-bea6-fb3cb9d65124 None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:22:04.397929 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcd916-8ccf-46a2-b1f1-567719f45e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:22:04.398620 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcd916-8ccf-46a2-b1f1-567719f45e9a None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:22:04.398903 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcd916-8ccf-46a2-b1f1-567719f45e9a None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:22:04.399178 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcd916-8ccf-46a2-b1f1-567719f45e9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:22:04.399977 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcd916-8ccf-46a2-b1f1-567719f45e9a None admin] RBAC: Adding query filter params (interface=public, service_id=19b29972e1fc42e1b36f1b2c463440f4, region_id=RegionOne) {{(pid=277682) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:22:04.400290 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcd916-8ccf-46a2-b1f1-567719f45e9a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:22:04.401681 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcd916-8ccf-46a2-b1f1-567719f45e9a None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:22:04.407422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 653/1300] 10.222.0.84 () {56 vars in 1251 bytes} [Thu May 19 20:22:04 2022] GET /identity/v3/endpoints?service_id=19b29972e1fc42e1b36f1b2c463440f4&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:22:04.414188 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa6118-2e61-4049-8c37-be9986e7183b None admin] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:22:04.502288 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:22:04.523527 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa6118-2e61-4049-8c37-be9986e7183b None admin] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:22:04.561598 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c248f-169e-446b-9765-538b1539633f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:22:04.562286 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c248f-169e-446b-9765-538b1539633f None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:22:04.562577 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c248f-169e-446b-9765-538b1539633f None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:22:04.562854 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c248f-169e-446b-9765-538b1539633f None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:22:04.563543 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c248f-169e-446b-9765-538b1539633f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:22:04.564941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c248f-169e-446b-9765-538b1539633f None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:22:04.572479 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 648/1301] 10.222.0.84 () {56 vars in 1087 bytes} [Thu May 19 20:22:04 2022] GET /identity/v3/services => generated 2191 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:22:06.786972 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506b962-8c7e-444e-a486-34c76cba0005 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:22:06.787863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506b962-8c7e-444e-a486-34c76cba0005 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:22:06.787863 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506b962-8c7e-444e-a486-34c76cba0005 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:22:06.790004 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 654/1302] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:22:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:22:06.795664 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f38472-f8ec-429a-80de-e93cb2795181 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:22:06.796121 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f38472-f8ec-429a-80de-e93cb2795181 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:22:06.796234 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f38472-f8ec-429a-80de-e93cb2795181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:22:06.814844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-79f38472-f8ec-429a-80de-e93cb2795181 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:22:06.816509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-79f38472-f8ec-429a-80de-e93cb2795181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:22:06.892968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 649/1303] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:22:06 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:22:06.917206 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506b962-8c7e-444e-a486-34c76cba0005 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:22:07.028071 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:22:07.036540 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506b962-8c7e-444e-a486-34c76cba0005 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:22:07.073387 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414a970-4502-4e26-ba84-57352465b53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:22:07.074121 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414a970-4502-4e26-ba84-57352465b53f None admin] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:22:07.074428 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414a970-4502-4e26-ba84-57352465b53f None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:22:07.074704 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414a970-4502-4e26-ba84-57352465b53f None admin] PATH_INFO: `/v3/services/placement` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:22:07.075354 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414a970-4502-4e26-ba84-57352465b53f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:22:07.076716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414a970-4502-4e26-ba84-57352465b53f None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application [None req-0414a970-4502-4e26-ba84-57352465b53f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:22:07.089464 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:22:07.090847 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:22:07.092156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:22:07.092156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application with Lock( May 19 20:22:07.092156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:22:07.092156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self._enter() May 19 20:22:07.092156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:22:07.092156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:22:07.092156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:22:07.092156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application return self.creator() May 19 20:22:07.092156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:22:07.092156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application created_value = creator( May 19 20:22:07.092156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:22:07.092156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:22:07.092156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:22:07.092156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: ERROR keystone.server.flask.application May 19 20:22:07.092156 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 655/1304] 10.222.0.84 () {56 vars in 1117 bytes} [Thu May 19 20:22:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:22:07.097851 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f38472-f8ec-429a-80de-e93cb2795181 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:22:07.182397 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:22:07.215906 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f38472-f8ec-429a-80de-e93cb2795181 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:22:07.252642 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94228c-9d7f-499f-8d7f-87009f3cc699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:22:07.253591 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94228c-9d7f-499f-8d7f-87009f3cc699 None admin] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:22:07.253874 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94228c-9d7f-499f-8d7f-87009f3cc699 None admin] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:22:07.254151 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94228c-9d7f-499f-8d7f-87009f3cc699 None admin] PATH_INFO: `/v3/services` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:22:07.254886 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94228c-9d7f-499f-8d7f-87009f3cc699 None admin] RBAC: Adding query filter params (name=placement) {{(pid=277681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:22:07.255196 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94228c-9d7f-499f-8d7f-87009f3cc699 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:22:07.256581 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94228c-9d7f-499f-8d7f-87009f3cc699 None admin] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:22:07.262363 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 650/1305] 10.222.0.84 () {56 vars in 1116 bytes} [Thu May 19 20:22:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:22:07.271644 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414a970-4502-4e26-ba84-57352465b53f None admin] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:22:07.401281 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414a970-4502-4e26-ba84-57352465b53f None admin] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:22:07.446659 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921caad-3a7a-4daa-9d31-62f680641851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '470d21b48683452bbca4f78884ff1182', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:22:07.447328 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921caad-3a7a-4daa-9d31-62f680641851 None admin] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:22:07.447610 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921caad-3a7a-4daa-9d31-62f680641851 None admin] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:22:07.447886 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921caad-3a7a-4daa-9d31-62f680641851 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:22:07.448642 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921caad-3a7a-4daa-9d31-62f680641851 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:22:07.450025 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921caad-3a7a-4daa-9d31-62f680641851 None admin] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:22:07.472428 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 656/1306] 10.222.0.84 () {60 vars in 1144 bytes} [Thu May 19 20:22:07 2022] POST /identity/v3/endpoints => generated 331 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:22:17.258808 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=30 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:22:17.467383 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:14.434240 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4244dd0-c096-4a10-ad08-efe64a34a5f9 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:14.434767 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4244dd0-c096-4a10-ad08-efe64a34a5f9 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:14.434860 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4244dd0-c096-4a10-ad08-efe64a34a5f9 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:14.440488 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 651/1307] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:23:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:14.442598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9dbad-8bfa-41ae-b80e-b1c507eeb108 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:14.443102 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9dbad-8bfa-41ae-b80e-b1c507eeb108 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:14.443216 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9dbad-8bfa-41ae-b80e-b1c507eeb108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:14.611739 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-b9b9dbad-8bfa-41ae-b80e-b1c507eeb108 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:14.613375 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-b9b9dbad-8bfa-41ae-b80e-b1c507eeb108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:14.826861 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 657/1308] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:23:14.908279 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909feca0-dd43-46a4-a43b-689f2d51b949 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:14.909062 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909feca0-dd43-46a4-a43b-689f2d51b949 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:14.909665 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909feca0-dd43-46a4-a43b-689f2d51b949 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:14.912297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 652/1309] 10.222.0.84 () {54 vars in 964 bytes} [Thu May 19 20:23:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:14.919714 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54ee1c-84c9-4c8f-b911-60accce3005b None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:14.920061 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54ee1c-84c9-4c8f-b911-60accce3005b None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:14.920282 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54ee1c-84c9-4c8f-b911-60accce3005b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:15.094658 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-bc54ee1c-84c9-4c8f-b911-60accce3005b None None] MFA Rules not processed for user `3d63f8a58ef24de897ab7fd0828ec467`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:15.096087 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-bc54ee1c-84c9-4c8f-b911-60accce3005b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:15.150418 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:15.224849 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 658/1310] 10.222.0.84 () {58 vars in 1061 bytes} [Thu May 19 20:23:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:23:15.234361 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909feca0-dd43-46a4-a43b-689f2d51b949 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:15.236381 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-909feca0-dd43-46a4-a43b-689f2d51b949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:15.359022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909feca0-dd43-46a4-a43b-689f2d51b949 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:15.437337 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47417e3-62e8-4660-b574-26c3e2879ce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d63f8a58ef24de897ab7fd0828ec467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:15.438133 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47417e3-62e8-4660-b574-26c3e2879ce6 service neutron] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:15.438538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47417e3-62e8-4660-b574-26c3e2879ce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:15.438916 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47417e3-62e8-4660-b574-26c3e2879ce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:15.441155 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-e47417e3-62e8-4660-b574-26c3e2879ce6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:15.499013 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:15.555444 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47417e3-62e8-4660-b574-26c3e2879ce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:15.557085 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47417e3-62e8-4660-b574-26c3e2879ce6 service neutron] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:15.576727 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 653/1311] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:23:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:23:18.157484 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2d0a2-b743-4d6a-9a38-bdfcd8565eba None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:18.158053 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2d0a2-b743-4d6a-9a38-bdfcd8565eba None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:18.158503 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2d0a2-b743-4d6a-9a38-bdfcd8565eba None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:18.162096 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 659/1312] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:23:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:18.168041 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6bfe3-3547-45cf-8056-f7b31b9d76f2 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:18.168502 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6bfe3-3547-45cf-8056-f7b31b9d76f2 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:18.168608 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6bfe3-3547-45cf-8056-f7b31b9d76f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:18.257223 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-7ec6bfe3-3547-45cf-8056-f7b31b9d76f2 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:18.258495 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-7ec6bfe3-3547-45cf-8056-f7b31b9d76f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:18.394291 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 654/1313] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:23:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:23:18.421232 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2d0a2-b743-4d6a-9a38-bdfcd8565eba None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:18.422842 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-11e2d0a2-b743-4d6a-9a38-bdfcd8565eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:18.543527 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2d0a2-b743-4d6a-9a38-bdfcd8565eba None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:18.619832 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:18.623462 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2efb51-2187-4d34-9614-af8285353b1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d63f8a58ef24de897ab7fd0828ec467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:18.624285 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2efb51-2187-4d34-9614-af8285353b1b service neutron] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:18.624689 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2efb51-2187-4d34-9614-af8285353b1b service neutron] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:18.625220 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2efb51-2187-4d34-9614-af8285353b1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:18.627065 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-cf2efb51-2187-4d34-9614-af8285353b1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:18.744292 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2efb51-2187-4d34-9614-af8285353b1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:18.745772 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2efb51-2187-4d34-9614-af8285353b1b service neutron] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:18.765549 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 660/1314] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:23:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:23:21.381618 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87630751-3e88-4b5e-859d-c43c83144b1c None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:21.382498 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87630751-3e88-4b5e-859d-c43c83144b1c None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:21.382498 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87630751-3e88-4b5e-859d-c43c83144b1c None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:21.384753 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 655/1315] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:23:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:21.392930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152809b-6da5-4513-a4c4-27e5652dd8eb None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:21.393364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152809b-6da5-4513-a4c4-27e5652dd8eb None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:21.393465 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152809b-6da5-4513-a4c4-27e5652dd8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:21.460000 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-a152809b-6da5-4513-a4c4-27e5652dd8eb None None] MFA Rules not processed for user `d5972d97f8c144e99fb7fb22399f0d94`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:21.461541 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-a152809b-6da5-4513-a4c4-27e5652dd8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:21.520460 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:21.568552 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 661/1316] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:23:21 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:23:21.612546 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87630751-3e88-4b5e-859d-c43c83144b1c None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:21.615006 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-87630751-3e88-4b5e-859d-c43c83144b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:21.630035 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:21.741496 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87630751-3e88-4b5e-859d-c43c83144b1c None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:21.825171 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e4111-4105-492f-99f9-c0e999b29f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d63f8a58ef24de897ab7fd0828ec467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:21.825743 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e4111-4105-492f-99f9-c0e999b29f3d service neutron] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:21.826035 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e4111-4105-492f-99f9-c0e999b29f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:21.826304 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e4111-4105-492f-99f9-c0e999b29f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:21.827981 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-fb8e4111-4105-492f-99f9-c0e999b29f3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:21.892219 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:21.945441 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e4111-4105-492f-99f9-c0e999b29f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:21.946939 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e4111-4105-492f-99f9-c0e999b29f3d service neutron] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:21.966408 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 656/1317] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:23:21 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:23:24.747166 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7ba9a-9767-43a1-90dd-131d59f78c02 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:24.747741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7ba9a-9767-43a1-90dd-131d59f78c02 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:24.747741 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7ba9a-9767-43a1-90dd-131d59f78c02 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:24.749013 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 662/1318] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:24.755646 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbf9cc-8e1f-4eff-af09-e1e1f89b095a None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:24.756367 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbf9cc-8e1f-4eff-af09-e1e1f89b095a None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:24.756509 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbf9cc-8e1f-4eff-af09-e1e1f89b095a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:24.838085 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-cdfbf9cc-8e1f-4eff-af09-e1e1f89b095a None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:24.839459 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-cdfbf9cc-8e1f-4eff-af09-e1e1f89b095a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:24.958121 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 657/1319] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:23:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:23:25.023210 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befc335-1460-4735-8e02-f4e0b62cb585 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:25.023510 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befc335-1460-4735-8e02-f4e0b62cb585 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:25.023782 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befc335-1460-4735-8e02-f4e0b62cb585 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:25.025194 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 663/1320] 10.222.0.84 () {54 vars in 964 bytes} [Thu May 19 20:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:25.034712 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83ceed-a82d-49ba-aec4-0b960d199a42 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:25.035154 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83ceed-a82d-49ba-aec4-0b960d199a42 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:25.035583 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83ceed-a82d-49ba-aec4-0b960d199a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:25.062090 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:25.122311 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-1d83ceed-a82d-49ba-aec4-0b960d199a42 None None] MFA Rules not processed for user `3d63f8a58ef24de897ab7fd0828ec467`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:25.124251 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-1d83ceed-a82d-49ba-aec4-0b960d199a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:25.209021 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 658/1321] 10.222.0.84 () {58 vars in 1061 bytes} [Thu May 19 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:23:25.218647 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befc335-1460-4735-8e02-f4e0b62cb585 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:25.220200 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-8befc335-1460-4735-8e02-f4e0b62cb585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:25.344585 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befc335-1460-4735-8e02-f4e0b62cb585 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:25.426130 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a64c2-f5e9-4484-acd0-9d8d1478ff72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d63f8a58ef24de897ab7fd0828ec467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:25.426823 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a64c2-f5e9-4484-acd0-9d8d1478ff72 service neutron] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:25.427101 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a64c2-f5e9-4484-acd0-9d8d1478ff72 service neutron] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:25.427374 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a64c2-f5e9-4484-acd0-9d8d1478ff72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:25.428991 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-b79a64c2-f5e9-4484-acd0-9d8d1478ff72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:25.437002 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:25.544327 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a64c2-f5e9-4484-acd0-9d8d1478ff72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:25.545777 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a64c2-f5e9-4484-acd0-9d8d1478ff72 service neutron] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:25.564038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 664/1322] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:23:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:23:28.988408 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95996965-eb9c-4c32-8728-37d0f82f94d4 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:28.989022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95996965-eb9c-4c32-8728-37d0f82f94d4 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:28.989022 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95996965-eb9c-4c32-8728-37d0f82f94d4 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:28.990467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 659/1323] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:23:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:28.996217 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d29900-2eb2-4ba0-8a26-5b28bbc8cecc None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:28.996644 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d29900-2eb2-4ba0-8a26-5b28bbc8cecc None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:28.996769 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d29900-2eb2-4ba0-8a26-5b28bbc8cecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:29.065576 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-37d29900-2eb2-4ba0-8a26-5b28bbc8cecc None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:29.066873 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-37d29900-2eb2-4ba0-8a26-5b28bbc8cecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:29.130795 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:29.171727 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 665/1324] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:23:29.207968 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95996965-eb9c-4c32-8728-37d0f82f94d4 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:29.209590 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-95996965-eb9c-4c32-8728-37d0f82f94d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:29.323572 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95996965-eb9c-4c32-8728-37d0f82f94d4 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:29.337593 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:29.402845 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db395c74-396d-4dde-a8be-5603d736998d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d63f8a58ef24de897ab7fd0828ec467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:29.403504 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db395c74-396d-4dde-a8be-5603d736998d service neutron] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:29.403784 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db395c74-396d-4dde-a8be-5603d736998d service neutron] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:29.404060 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db395c74-396d-4dde-a8be-5603d736998d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:29.405733 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-db395c74-396d-4dde-a8be-5603d736998d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:29.530538 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db395c74-396d-4dde-a8be-5603d736998d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:29.531964 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db395c74-396d-4dde-a8be-5603d736998d service neutron] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:29.552578 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 660/1325] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:23:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:23:31.837554 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d131cb8-e2e2-4483-91bc-274409a465b0 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:31.838134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d131cb8-e2e2-4483-91bc-274409a465b0 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:31.838134 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d131cb8-e2e2-4483-91bc-274409a465b0 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:31.839632 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 666/1326] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:31.845073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe78b8-a101-4666-85cb-1e635153d78c None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:31.845832 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe78b8-a101-4666-85cb-1e635153d78c None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:31.845832 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe78b8-a101-4666-85cb-1e635153d78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:31.897226 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:31.916528 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-9afe78b8-a101-4666-85cb-1e635153d78c None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:31.917767 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-9afe78b8-a101-4666-85cb-1e635153d78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:32.016503 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 661/1327] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:23:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:23:32.052332 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d131cb8-e2e2-4483-91bc-274409a465b0 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:32.053960 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-9d131cb8-e2e2-4483-91bc-274409a465b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:32.164169 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d131cb8-e2e2-4483-91bc-274409a465b0 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:32.246482 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12123f-9684-4b7d-b012-38de977f1790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d63f8a58ef24de897ab7fd0828ec467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:32.247145 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12123f-9684-4b7d-b012-38de977f1790 service neutron] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:32.247426 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12123f-9684-4b7d-b012-38de977f1790 service neutron] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:32.247701 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12123f-9684-4b7d-b012-38de977f1790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:32.249364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-6f12123f-9684-4b7d-b012-38de977f1790 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:32.268776 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:32.375773 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12123f-9684-4b7d-b012-38de977f1790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:32.378096 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12123f-9684-4b7d-b012-38de977f1790 service neutron] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:32.403690 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 667/1328] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:23:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:23:34.800410 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c405512-a967-41b8-9db7-2786b1657771 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:34.801237 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c405512-a967-41b8-9db7-2786b1657771 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:34.801447 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c405512-a967-41b8-9db7-2786b1657771 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:34.803831 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 662/1329] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:23:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:34.811714 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21b9c4-41a9-40dc-9e86-83a0fd282e08 None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:34.812183 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21b9c4-41a9-40dc-9e86-83a0fd282e08 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:34.812304 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21b9c4-41a9-40dc-9e86-83a0fd282e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:34.882478 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-6f21b9c4-41a9-40dc-9e86-83a0fd282e08 None None] MFA Rules not processed for user `d5972d97f8c144e99fb7fb22399f0d94`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:34.883716 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-6f21b9c4-41a9-40dc-9e86-83a0fd282e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:34.950883 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:34.986423 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 668/1330] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:23:34 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:23:35.028897 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c405512-a967-41b8-9db7-2786b1657771 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:35.030467 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-3c405512-a967-41b8-9db7-2786b1657771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:35.143780 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c405512-a967-41b8-9db7-2786b1657771 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:35.163880 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:35.224910 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b81f7e-8a81-497a-ab65-d03991c80c8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d63f8a58ef24de897ab7fd0828ec467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:35.225570 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b81f7e-8a81-497a-ab65-d03991c80c8a service neutron] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:35.225854 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b81f7e-8a81-497a-ab65-d03991c80c8a service neutron] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:35.226129 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b81f7e-8a81-497a-ab65-d03991c80c8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:35.227833 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-44b81f7e-8a81-497a-ab65-d03991c80c8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:35.356234 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b81f7e-8a81-497a-ab65-d03991c80c8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:35.357700 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b81f7e-8a81-497a-ab65-d03991c80c8a service neutron] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:35.376320 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 663/1331] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:23:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:23:37.860883 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830eac0-c100-456f-a489-7c2363aeb066 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:37.861636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830eac0-c100-456f-a489-7c2363aeb066 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:37.861636 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830eac0-c100-456f-a489-7c2363aeb066 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:37.863005 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 669/1332] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:37.871756 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef6392-4e3a-4f44-8810-afadc2671216 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:37.872510 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef6392-4e3a-4f44-8810-afadc2671216 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:37.872666 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef6392-4e3a-4f44-8810-afadc2671216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:37.949009 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:37.952636 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-b1ef6392-4e3a-4f44-8810-afadc2671216 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:37.954578 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-b1ef6392-4e3a-4f44-8810-afadc2671216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:38.071804 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 664/1333] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:23:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:23:38.108370 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830eac0-c100-456f-a489-7c2363aeb066 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:38.109930 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-e830eac0-c100-456f-a489-7c2363aeb066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:38.243607 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830eac0-c100-456f-a489-7c2363aeb066 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:38.334652 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705fa55-2aca-41b3-b61a-fc4921549f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d63f8a58ef24de897ab7fd0828ec467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:38.335313 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705fa55-2aca-41b3-b61a-fc4921549f9c service neutron] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:38.335914 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705fa55-2aca-41b3-b61a-fc4921549f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:38.336197 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705fa55-2aca-41b3-b61a-fc4921549f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:38.337865 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-f705fa55-2aca-41b3-b61a-fc4921549f9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:38.371186 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:38.470459 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f705fa55-2aca-41b3-b61a-fc4921549f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:38.471979 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f705fa55-2aca-41b3-b61a-fc4921549f9c service neutron] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:38.493368 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 670/1334] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:23:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:23:40.752921 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b75408-7ee9-4240-bb03-39234f2c34d2 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:40.753799 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b75408-7ee9-4240-bb03-39234f2c34d2 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:40.753799 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b75408-7ee9-4240-bb03-39234f2c34d2 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:40.756017 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 665/1335] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:23:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:40.765323 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f89643-f6b2-498b-9eca-dc51a0aa85ab None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:40.766082 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f89643-f6b2-498b-9eca-dc51a0aa85ab None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:40.766319 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f89643-f6b2-498b-9eca-dc51a0aa85ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:40.836659 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-47f89643-f6b2-498b-9eca-dc51a0aa85ab None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:40.837923 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-47f89643-f6b2-498b-9eca-dc51a0aa85ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:40.926220 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:40.949599 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 671/1336] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:23:40.993108 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b75408-7ee9-4240-bb03-39234f2c34d2 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:40.995575 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-06b75408-7ee9-4240-bb03-39234f2c34d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:41.107643 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b75408-7ee9-4240-bb03-39234f2c34d2 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:41.136673 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:41.183972 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e5b42-f86b-4ea4-882f-599837d7df78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d63f8a58ef24de897ab7fd0828ec467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:41.184633 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e5b42-f86b-4ea4-882f-599837d7df78 service neutron] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:41.184904 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e5b42-f86b-4ea4-882f-599837d7df78 service neutron] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:41.185174 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e5b42-f86b-4ea4-882f-599837d7df78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:41.186844 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-088e5b42-f86b-4ea4-882f-599837d7df78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:41.314056 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e5b42-f86b-4ea4-882f-599837d7df78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:41.316211 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e5b42-f86b-4ea4-882f-599837d7df78 service neutron] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:41.341995 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 666/1337] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:23:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:23:43.883224 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd3dd8-57ce-4077-91cd-9d4f1af11ea8 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:43.883780 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd3dd8-57ce-4077-91cd-9d4f1af11ea8 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:43.883780 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd3dd8-57ce-4077-91cd-9d4f1af11ea8 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:43.885852 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 672/1338] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:23:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:43.894378 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d494a2-5d4d-4fec-a387-0f96a474cd2d None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:43.894825 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d494a2-5d4d-4fec-a387-0f96a474cd2d None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:43.894946 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d494a2-5d4d-4fec-a387-0f96a474cd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:43.969367 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-34d494a2-5d4d-4fec-a387-0f96a474cd2d None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:43.971443 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-34d494a2-5d4d-4fec-a387-0f96a474cd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:43.981402 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:44.090805 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 667/1339] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:23:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:23:44.126721 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd3dd8-57ce-4077-91cd-9d4f1af11ea8 None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:44.128801 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-8fbd3dd8-57ce-4077-91cd-9d4f1af11ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:44.251712 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd3dd8-57ce-4077-91cd-9d4f1af11ea8 None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:44.339893 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b612ba-dba9-4f4c-9b0d-d72c839072c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d63f8a58ef24de897ab7fd0828ec467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:44.340547 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b612ba-dba9-4f4c-9b0d-d72c839072c1 service neutron] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:44.340828 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b612ba-dba9-4f4c-9b0d-d72c839072c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:44.341107 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b612ba-dba9-4f4c-9b0d-d72c839072c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:44.342793 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-94b612ba-dba9-4f4c-9b0d-d72c839072c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:44.379642 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:44.458969 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b612ba-dba9-4f4c-9b0d-d72c839072c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:44.460746 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b612ba-dba9-4f4c-9b0d-d72c839072c1 service neutron] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:44.480558 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 673/1340] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:23:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 20:23:48.885208 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3725dc9-ef57-4431-93f6-90586de59b84 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:48.885818 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3725dc9-ef57-4431-93f6-90586de59b84 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:48.885818 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3725dc9-ef57-4431-93f6-90586de59b84 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:48.887183 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 668/1341] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:48.892791 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88fc15-7504-4a38-aa74-c32bb636ee1d None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:48.893261 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88fc15-7504-4a38-aa74-c32bb636ee1d None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:48.893377 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88fc15-7504-4a38-aa74-c32bb636ee1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:48.968361 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-5d88fc15-7504-4a38-aa74-c32bb636ee1d None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:48.969740 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-5d88fc15-7504-4a38-aa74-c32bb636ee1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:49.065598 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=100 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:49.080111 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 674/1342] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:23:49.119898 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3725dc9-ef57-4431-93f6-90586de59b84 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:49.122361 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-f3725dc9-ef57-4431-93f6-90586de59b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:49.224139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3725dc9-ef57-4431-93f6-90586de59b84 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:49.227994 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a629b5-1268-448c-a33e-d3351aeb368b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d63f8a58ef24de897ab7fd0828ec467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:49.228655 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a629b5-1268-448c-a33e-d3351aeb368b service neutron] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:49.228941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a629b5-1268-448c-a33e-d3351aeb368b service neutron] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:49.229220 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a629b5-1268-448c-a33e-d3351aeb368b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:49.239070 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a629b5-1268-448c-a33e-d3351aeb368b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:49.240540 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a629b5-1268-448c-a33e-d3351aeb368b service neutron] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:49.250584 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 669/1343] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:23:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:23:52.087040 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b1abe-b0a8-46a5-86fd-c690b3f8cc9c None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:52.087588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b1abe-b0a8-46a5-86fd-c690b3f8cc9c None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:52.087588 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b1abe-b0a8-46a5-86fd-c690b3f8cc9c None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:52.088926 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 675/1344] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:52.094536 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc24a05-2571-407f-9666-e6058733a375 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:52.094993 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc24a05-2571-407f-9666-e6058733a375 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:52.095184 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc24a05-2571-407f-9666-e6058733a375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:52.124827 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-9dc24a05-2571-407f-9666-e6058733a375 None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:52.126033 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-9dc24a05-2571-407f-9666-e6058733a375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:52.154788 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:23:52.205887 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 670/1345] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:23:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:23:52.242887 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b1abe-b0a8-46a5-86fd-c690b3f8cc9c None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:52.255457 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b1abe-b0a8-46a5-86fd-c690b3f8cc9c None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:52.259770 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfad465-e3f0-42a6-9809-bd0ff05727bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d63f8a58ef24de897ab7fd0828ec467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:52.260422 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfad465-e3f0-42a6-9809-bd0ff05727bb service neutron] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:52.260709 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfad465-e3f0-42a6-9809-bd0ff05727bb service neutron] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:52.260990 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfad465-e3f0-42a6-9809-bd0ff05727bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:52.271162 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfad465-e3f0-42a6-9809-bd0ff05727bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:52.272624 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfad465-e3f0-42a6-9809-bd0ff05727bb service neutron] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:52.283225 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 676/1346] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:23:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:23:56.659983 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01165762-1a4f-4872-886a-1c1ad30012a0 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:56.660501 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01165762-1a4f-4872-886a-1c1ad30012a0 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:56.660615 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01165762-1a4f-4872-886a-1c1ad30012a0 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:56.662088 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 671/1347] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:56.667811 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473479d-b56e-4a32-bb92-b335037f89bf None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:56.668317 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473479d-b56e-4a32-bb92-b335037f89bf None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:56.668430 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473479d-b56e-4a32-bb92-b335037f89bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:56.687659 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-e473479d-b56e-4a32-bb92-b335037f89bf None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:56.688963 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-e473479d-b56e-4a32-bb92-b335037f89bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:56.698309 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 677/1348] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:23:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:23:56.737295 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01165762-1a4f-4872-886a-1c1ad30012a0 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:56.739095 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-01165762-1a4f-4872-886a-1c1ad30012a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:56.754395 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01165762-1a4f-4872-886a-1c1ad30012a0 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:56.757879 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbd780-8920-4769-8f13-a3e82d811087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d63f8a58ef24de897ab7fd0828ec467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:56.758573 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbd780-8920-4769-8f13-a3e82d811087 service neutron] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:56.758858 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbd780-8920-4769-8f13-a3e82d811087 service neutron] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:56.759139 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbd780-8920-4769-8f13-a3e82d811087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:56.769123 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bbd780-8920-4769-8f13-a3e82d811087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:56.770612 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bbd780-8920-4769-8f13-a3e82d811087 service neutron] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:56.780234 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 672/1349] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:23:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:24:02.246864 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f43845-d11f-44c5-8fe7-63e79b19a380 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:02.247405 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f43845-d11f-44c5-8fe7-63e79b19a380 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:02.247405 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f43845-d11f-44c5-8fe7-63e79b19a380 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:02.248783 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 678/1350] 10.222.0.84 () {54 vars in 921 bytes} [Thu May 19 20:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:02.254451 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10524f4c-ee60-4b6e-beb5-1427f5151189 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:02.262512 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10524f4c-ee60-4b6e-beb5-1427f5151189 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:02.263073 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10524f4c-ee60-4b6e-beb5-1427f5151189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:02.360947 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-10524f4c-ee60-4b6e-beb5-1427f5151189 None None] MFA Rules not processed for user `8cd315b75b1f4be5a788dc70ae07194d`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:02.362638 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-10524f4c-ee60-4b6e-beb5-1427f5151189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:02.450225 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 673/1351] 10.222.0.84 () {58 vars in 1018 bytes} [Thu May 19 20:24:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:24:06.681038 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=13 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:07.029965 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce2989-d6af-4e5c-b4c1-039814e4dfcd None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:07.030364 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce2989-d6af-4e5c-b4c1-039814e4dfcd None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:07.030761 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce2989-d6af-4e5c-b4c1-039814e4dfcd None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:07.032429 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 679/1352] 10.222.0.84 () {54 vars in 921 bytes} [Thu May 19 20:24:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:07.037861 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc884bce-1690-4868-bc76-5da92fd101e4 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:07.038423 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc884bce-1690-4868-bc76-5da92fd101e4 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:07.038701 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc884bce-1690-4868-bc76-5da92fd101e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:07.058244 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-dc884bce-1690-4868-bc76-5da92fd101e4 None None] MFA Rules not processed for user `8cd315b75b1f4be5a788dc70ae07194d`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:07.059610 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-dc884bce-1690-4868-bc76-5da92fd101e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:07.069027 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 674/1353] 10.222.0.84 () {58 vars in 1018 bytes} [Thu May 19 20:24:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:24:08.636152 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8786c-c4cd-4c85-90c5-31421ddd0a14 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:08.636742 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8786c-c4cd-4c85-90c5-31421ddd0a14 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:08.636742 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8786c-c4cd-4c85-90c5-31421ddd0a14 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:08.638127 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 680/1354] 10.222.0.84 () {54 vars in 921 bytes} [Thu May 19 20:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:08.643705 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05c720-6a5a-4db0-b004-ee684700fb98 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:08.644159 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05c720-6a5a-4db0-b004-ee684700fb98 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:08.644293 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05c720-6a5a-4db0-b004-ee684700fb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:08.663574 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-fd05c720-6a5a-4db0-b004-ee684700fb98 None None] MFA Rules not processed for user `8cd315b75b1f4be5a788dc70ae07194d`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:08.664796 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-fd05c720-6a5a-4db0-b004-ee684700fb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:08.675024 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 675/1355] 10.222.0.84 () {58 vars in 1018 bytes} [Thu May 19 20:24:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:24:12.599457 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591f81e-2952-4f94-8319-4d6a270d90b4 None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:12.600040 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591f81e-2952-4f94-8319-4d6a270d90b4 None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:12.600040 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591f81e-2952-4f94-8319-4d6a270d90b4 None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:12.601569 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 681/1356] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:12.606974 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707f07e-fb93-468e-b3bb-68bb046fc65c None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:12.607830 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707f07e-fb93-468e-b3bb-68bb046fc65c None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:12.608575 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707f07e-fb93-468e-b3bb-68bb046fc65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:12.627854 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-a707f07e-fb93-468e-b3bb-68bb046fc65c None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:12.629265 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-a707f07e-fb93-468e-b3bb-68bb046fc65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:12.638369 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 676/1357] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:24:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:24:12.700595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a3839-3d81-442e-813e-8cd44efff71c None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:12.701048 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a3839-3d81-442e-813e-8cd44efff71c None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:12.701525 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a3839-3d81-442e-813e-8cd44efff71c None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:12.703093 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 682/1358] 10.222.0.84 () {54 vars in 961 bytes} [Thu May 19 20:24:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:12.715666 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42070e-256e-4276-b3d7-04f0347585c9 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:12.716374 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42070e-256e-4276-b3d7-04f0347585c9 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:12.717263 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42070e-256e-4276-b3d7-04f0347585c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:12.778992 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=100 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:12.790947 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-5e42070e-256e-4276-b3d7-04f0347585c9 None None] MFA Rules not processed for user `edb1bcb781084f03a4ad2611a09bd77e`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:12.792500 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-5e42070e-256e-4276-b3d7-04f0347585c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:12.890071 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 677/1359] 10.222.0.84 () {58 vars in 1058 bytes} [Thu May 19 20:24:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:24:12.899208 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a3839-3d81-442e-813e-8cd44efff71c None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:12.914927 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a3839-3d81-442e-813e-8cd44efff71c None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:12.920743 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4af69c-8e19-4bfd-b392-f11154b36e51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1bcb781084f03a4ad2611a09bd77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:12.921496 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4af69c-8e19-4bfd-b392-f11154b36e51 service nova] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:12.921877 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4af69c-8e19-4bfd-b392-f11154b36e51 service nova] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:12.922655 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4af69c-8e19-4bfd-b392-f11154b36e51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:12.933233 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4af69c-8e19-4bfd-b392-f11154b36e51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:12.935244 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4af69c-8e19-4bfd-b392-f11154b36e51 service nova] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:12.946595 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 683/1360] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:24:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:24:15.381627 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541aca6-b6fa-4db9-8984-e8632f435dd2 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:15.382161 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541aca6-b6fa-4db9-8984-e8632f435dd2 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:15.382161 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541aca6-b6fa-4db9-8984-e8632f435dd2 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:15.383533 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 678/1361] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:15.388936 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652eb93-1e72-4ed3-a2ba-809ef59f439e None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:15.389401 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652eb93-1e72-4ed3-a2ba-809ef59f439e None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:15.389517 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652eb93-1e72-4ed3-a2ba-809ef59f439e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:15.408736 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-d652eb93-1e72-4ed3-a2ba-809ef59f439e None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:15.409994 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-d652eb93-1e72-4ed3-a2ba-809ef59f439e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:15.421416 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 684/1362] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:24:15.468082 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541aca6-b6fa-4db9-8984-e8632f435dd2 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:15.483722 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541aca6-b6fa-4db9-8984-e8632f435dd2 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:15.490549 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81100b-1d01-45e1-896c-9add4318a90f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1bcb781084f03a4ad2611a09bd77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:15.491220 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81100b-1d01-45e1-896c-9add4318a90f service nova] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:15.491511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81100b-1d01-45e1-896c-9add4318a90f service nova] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:15.491790 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81100b-1d01-45e1-896c-9add4318a90f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:15.501046 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81100b-1d01-45e1-896c-9add4318a90f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:15.502731 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81100b-1d01-45e1-896c-9add4318a90f service nova] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:15.513311 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 679/1363] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:24:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:24:18.107273 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd5d1b-ae41-4ba8-bc08-c3087852da5f None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.107830 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd5d1b-ae41-4ba8-bc08-c3087852da5f None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.107830 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd5d1b-ae41-4ba8-bc08-c3087852da5f None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.109404 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 685/1364] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:18.115667 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469d91b-6b8a-4b35-a0a0-deb141b6b7be None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.116178 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469d91b-6b8a-4b35-a0a0-deb141b6b7be None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.116292 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469d91b-6b8a-4b35-a0a0-deb141b6b7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.136740 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-9469d91b-6b8a-4b35-a0a0-deb141b6b7be None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:18.138297 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-9469d91b-6b8a-4b35-a0a0-deb141b6b7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:18.146941 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 680/1365] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:24:18.193894 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3109e0-6ba0-4099-a8d3-d2680310529d None None] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.194178 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3109e0-6ba0-4099-a8d3-d2680310529d None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.194761 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3109e0-6ba0-4099-a8d3-d2680310529d None None] PATH_INFO: `/` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.196714 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 686/1366] 10.222.0.84 () {54 vars in 961 bytes} [Thu May 19 20:24:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:18.201775 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c761f-decb-4b33-9662-bdb340045225 None None] REQUEST_METHOD: `POST` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.202068 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c761f-decb-4b33-9662-bdb340045225 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.202384 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c761f-decb-4b33-9662-bdb340045225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.222316 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.auth.core [None req-258c761f-decb-4b33-9662-bdb340045225 None None] MFA Rules not processed for user `edb1bcb781084f03a4ad2611a09bd77e`. Rule list: `[]` (Enabled: `True`). {{(pid=277681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:18.223511 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.fernet_utils [None req-258c761f-decb-4b33-9662-bdb340045225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:18.234417 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 681/1367] 10.222.0.84 () {58 vars in 1058 bytes} [Thu May 19 20:24:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:24:18.243180 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3109e0-6ba0-4099-a8d3-d2680310529d None None] Authenticating user token {{(pid=277682) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:18.258129 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3109e0-6ba0-4099-a8d3-d2680310529d None None] Validating token access rules against request {{(pid=277682) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:18.263768 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02dcfe-02d9-474b-adc2-1287842a8405 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1bcb781084f03a4ad2611a09bd77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277682) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:18.264425 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02dcfe-02d9-474b-adc2-1287842a8405 service nova] REQUEST_METHOD: `GET` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.264705 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02dcfe-02d9-474b-adc2-1287842a8405 service nova] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.264982 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02dcfe-02d9-474b-adc2-1287842a8405 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.273979 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02dcfe-02d9-474b-adc2-1287842a8405 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:18.275773 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02dcfe-02d9-474b-adc2-1287842a8405 service nova] RBAC: Authorization granted {{(pid=277682) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:18.286018 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 687/1368] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:24:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:24:19.260700 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863379bc-9743-43d4-80f1-f71fe0d581eb None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:19.261290 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863379bc-9743-43d4-80f1-f71fe0d581eb None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:19.261290 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863379bc-9743-43d4-80f1-f71fe0d581eb None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:19.262742 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 682/1369] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:19.268335 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae11839-e5f3-4694-bcf7-479fa58114ca None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:19.268875 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae11839-e5f3-4694-bcf7-479fa58114ca None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:19.268875 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae11839-e5f3-4694-bcf7-479fa58114ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:19.287946 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-dae11839-e5f3-4694-bcf7-479fa58114ca None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:19.289407 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-dae11839-e5f3-4694-bcf7-479fa58114ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:19.300180 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 688/1370] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:24:20.627152 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8cbf1-1595-4fb4-b0c8-beed480d8a66 None None] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:20.627721 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8cbf1-1595-4fb4-b0c8-beed480d8a66 None None] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:20.627721 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8cbf1-1595-4fb4-b0c8-beed480d8a66 None None] PATH_INFO: `/` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:20.629084 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 683/1371] 10.222.0.84 () {54 vars in 926 bytes} [Thu May 19 20:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:20.634545 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272407aa-a3ec-4fe1-a830-bf188c9c730b None None] REQUEST_METHOD: `POST` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:20.634992 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272407aa-a3ec-4fe1-a830-bf188c9c730b None None] SCRIPT_NAME: `/identity` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:20.635088 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272407aa-a3ec-4fe1-a830-bf188c9c730b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277682) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:20.653412 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.auth.core [None req-272407aa-a3ec-4fe1-a830-bf188c9c730b None None] MFA Rules not processed for user `470d21b48683452bbca4f78884ff1182`. Rule list: `[]` (Enabled: `True`). {{(pid=277682) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:20.654637 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG keystone.common.fernet_utils [None req-272407aa-a3ec-4fe1-a830-bf188c9c730b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277682) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:20.663976 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: [pid: 277682|app: 0|req: 689/1372] 10.222.0.84 () {58 vars in 1023 bytes} [Thu May 19 20:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:24:20.708811 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8cbf1-1595-4fb4-b0c8-beed480d8a66 None None] Authenticating user token {{(pid=277681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:20.723853 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8cbf1-1595-4fb4-b0c8-beed480d8a66 None None] Validating token access rules against request {{(pid=277681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:20.729855 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce64f7e-a593-40ed-b9e2-28073e7b5e40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1bcb781084f03a4ad2611a09bd77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59555e7adac54eddb47d1ef18ef870c4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:20.730533 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce64f7e-a593-40ed-b9e2-28073e7b5e40 service nova] REQUEST_METHOD: `GET` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:20.730809 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce64f7e-a593-40ed-b9e2-28073e7b5e40 service nova] SCRIPT_NAME: `/identity` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:20.731081 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce64f7e-a593-40ed-b9e2-28073e7b5e40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:20.740319 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce64f7e-a593-40ed-b9e2-28073e7b5e40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:20.741920 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce64f7e-a593-40ed-b9e2-28073e7b5e40 service nova] RBAC: Authorization granted {{(pid=277681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:20.751749 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: [pid: 277681|app: 0|req: 684/1373] 10.222.0.84 () {60 vars in 1365 bytes} [Thu May 19 20:24:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:24:30.648306 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: DEBUG dbcounter [-] [277682] Writing DB stats keystone:SELECT=21 {{(pid=277682) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:30.747602 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: DEBUG dbcounter [-] [277681] Writing DB stats keystone:SELECT=54 {{(pid=277681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:47.942464 dv-cinder-835881-13-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 20:24:47.942797 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: Thu May 19 20:24:47 2022 - graceful shutdown triggered... May 19 20:24:47.943519 dv-cinder-835881-13-iscsi devstack@keystone.service[277681]: Gracefully killing worker 1 (pid: 277681)... May 19 20:24:47.944072 dv-cinder-835881-13-iscsi devstack@keystone.service[277682]: Gracefully killing worker 2 (pid: 277682)... May 19 20:24:48.944706 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: worker 1 buried after 1 seconds May 19 20:24:48.944706 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: worker 2 buried after 1 seconds May 19 20:24:48.944706 dv-cinder-835881-13-iscsi devstack@keystone.service[277667]: goodbye to uWSGI. May 19 20:24:48.947221 dv-cinder-835881-13-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 20:24:48.948091 dv-cinder-835881-13-iscsi systemd[1]: Stopped Devstack devstack@keystone.service.